Transformer-based Models for Long Document Summarisation in Financial Domain
Urvashi Khanna, Samira Ghodratnama, Diego Moll ́a, Amin Beheshti
Abstract
Summarisation of long financial documents is a challenging task due to the lack of large-scale datasets and the need for domain knowledge experts to create human-written summaries. Traditional summarisation approaches that generate a summary based on the content cannot produce summaries comparable to human-written ones and thus are rarely used in practice. In this work, we use the Longformer-Encoder-Decoder (LED) model to handle long financial reports. We describe our experiments and participating systems in the financial narrative summarisation shared task. Multi-stage fine-tuning helps the model generalise better on niche domains and avoids the problem of catastrophic forgetting. We further investigate the effect of the staged fine-tuning approach on the FNS dataset. Our systems achieved promising results in terms of ROUGE scores on the validation dataset.- Anthology ID:
- 2022.fnp-1.10
- Volume:
- Proceedings of the 4th Financial Narrative Processing Workshop @LREC2022
- Month:
- June
- Year:
- 2022
- Address:
- Marseille, France
- Editors:
- Mahmoud El-Haj, Paul Rayson, Nadhem Zmandar
- Venue:
- FNP
- SIG:
- Publisher:
- European Language Resources Association
- Note:
- Pages:
- 73–78
- Language:
- URL:
- https://aclanthology.org/2022.fnp-1.10
- DOI:
- Cite (ACL):
- Urvashi Khanna, Samira Ghodratnama, Diego Moll ́a, and Amin Beheshti. 2022. Transformer-based Models for Long Document Summarisation in Financial Domain. In Proceedings of the 4th Financial Narrative Processing Workshop @LREC2022, pages 73–78, Marseille, France. European Language Resources Association.
- Cite (Informal):
- Transformer-based Models for Long Document Summarisation in Financial Domain (Khanna et al., FNP 2022)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2022.fnp-1.10.pdf