Abstract
This paper introduces our efforts at the FinCasual shared task for modeling causality in financial utterances. Our approach uses the commonly and successfully applied strategy of fine-tuning a transformer-based language model with a twist, i.e. we modified the training and inference mechanism such that our model produces multiple predictions for the same instance. By designing such a model that returns k>1 predictions at the same time, we not only obtain a more resource efficient training (as opposed to fine-tuning some pre-trained language model k independent times), but our results indicate that we are also capable of obtaining comparable or even better evaluation scores that way. We compare multiple strategies for combining the k predictions of our model. Our submissions got ranked third on both subtasks of the shared task.- Anthology ID:
- 2020.fnp-1.13
- Volume:
- Proceedings of the 1st Joint Workshop on Financial Narrative Processing and MultiLing Financial Summarisation
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona, Spain (Online)
- Editors:
- Dr Mahmoud El-Haj, Dr Vasiliki Athanasakou, Dr Sira Ferradans, Dr Catherine Salzedo, Dr Ans Elhag, Dr Houda Bouamor, Dr Marina Litvak, Dr Paul Rayson, Dr George Giannakopoulos, Nikiforos Pittaras
- Venue:
- FNP
- SIG:
- Publisher:
- COLING
- Note:
- Pages:
- 80–84
- Language:
- URL:
- https://aclanthology.org/2020.fnp-1.13
- DOI:
- Cite (ACL):
- Zsolt Szántó and Gábor Berend. 2020. ProsperAMnet at FinCausal 2020, Task 1 & 2: Modeling causality in financial texts using multi-headed transformers. In Proceedings of the 1st Joint Workshop on Financial Narrative Processing and MultiLing Financial Summarisation, pages 80–84, Barcelona, Spain (Online). COLING.
- Cite (Informal):
- ProsperAMnet at FinCausal 2020, Task 1 & 2: Modeling causality in financial texts using multi-headed transformers (Szántó & Berend, FNP 2020)
- PDF:
- https://preview.aclanthology.org/teach-a-man-to-fish/2020.fnp-1.13.pdf