Abstract
In this paper, we introduce the Financial-STS task, a financial domain-specific NLP task designed to measure the nuanced semantic similarity between pairs of financial narratives. These narratives originate from the financial statements of the same company but correspond to different periods, such as year-over-year comparisons. Measuring the subtle semantic differences between these paired narratives enables market stakeholders to gauge changes over time in the company’s financial and operational situations, which is critical for financial decision-making. We find that existing pretrained embedding models and LLM embeddings fall short in discerning these subtle financial narrative shifts. To address this gap, we propose an LLM-augmented pipeline specifically designed for the Financial-STS task. Evaluation on a human-annotated dataset demonstrates that our proposed method outperforms existing methods trained on classic STS tasks and generic LLM embeddings.- Anthology ID:
- 2024.findings-naacl.168
- Volume:
- Findings of the Association for Computational Linguistics: NAACL 2024
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico
- Editors:
- Kevin Duh, Helena Gomez, Steven Bethard
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2641–2652
- Language:
- URL:
- https://aclanthology.org/2024.findings-naacl.168
- DOI:
- Cite (ACL):
- Jiaxin Liu, Yi Yang, and Kar Yan Tam. 2024. Beyond Surface Similarity: Detecting Subtle Semantic Shifts in Financial Narratives. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2641–2652, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- Beyond Surface Similarity: Detecting Subtle Semantic Shifts in Financial Narratives (Liu et al., Findings 2024)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2024.findings-naacl.168.pdf