Imaginary Numbers! Evaluating Numerical Referring Expressions by Neural End-to-End Surface Realization Systems
Rossana Cunha, Osuji Chinonso, João Campos, Brian Timoney, Brian Davis, Fabio Cozman, Adriana Pagano, Thiago Castro Ferreira
Abstract
Neural end-to-end surface realizers output more fluent texts than classical architectures. However, they tend to suffer from adequacy problems, in particular hallucinations in numerical referring expression generation. This poses a problem to language generation in sensitive domains, as is the case of robot journalism covering COVID-19 and Amazon deforestation. We propose an approach whereby numerical referring expressions are converted from digits to plain word form descriptions prior to being fed to state-of-the-art Large Language Models. We conduct automatic and human evaluations to report the best strategy to numerical superficial realization. Code and data are publicly available.- Anthology ID:
- 2024.insights-1.10
- Volume:
- Proceedings of the Fifth Workshop on Insights from Negative Results in NLP
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico
- Editors:
- Shabnam Tafreshi, Arjun Akula, João Sedoc, Aleksandr Drozd, Anna Rogers, Anna Rumshisky
- Venues:
- insights | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 73–81
- Language:
- URL:
- https://aclanthology.org/2024.insights-1.10
- DOI:
- Cite (ACL):
- Rossana Cunha, Osuji Chinonso, João Campos, Brian Timoney, Brian Davis, Fabio Cozman, Adriana Pagano, and Thiago Castro Ferreira. 2024. Imaginary Numbers! Evaluating Numerical Referring Expressions by Neural End-to-End Surface Realization Systems. In Proceedings of the Fifth Workshop on Insights from Negative Results in NLP, pages 73–81, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- Imaginary Numbers! Evaluating Numerical Referring Expressions by Neural End-to-End Surface Realization Systems (Cunha et al., insights-WS 2024)
- PDF:
- https://preview.aclanthology.org/jeptaln-2024-ingestion/2024.insights-1.10.pdf