Abstract
Machine translation (MT) requires a wide range of linguistic capabilities, which current end-to-end models are expected to learn implicitly by observing aligned sentences in bilingual corpora. In this work, we ask: How well MT models learn coreference resolution via implicit signal? To answer this question, we develop an evaluation methodology that derives coreference clusters from MT output and evaluates them without requiring annotations in the target language.Following, we evaluate several prominent open-source and commercial MT systems, translating from English to six target languages, and compare them to state-of-the-art coreference resolvers on three challenging benchmarks. Our results show that the monolingual resolvers greatly outperform MT models. Motivated by this result, we experiment with different methods for incorporating the output of coreference resolution models in MT, showing improvement over strong baselines.- Anthology ID:
- 2023.eacl-main.69
- Volume:
- Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
- Month:
- May
- Year:
- 2023
- Address:
- Dubrovnik, Croatia
- Editors:
- Andreas Vlachos, Isabelle Augenstein
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 980–992
- Language:
- URL:
- https://aclanthology.org/2023.eacl-main.69
- DOI:
- 10.18653/v1/2023.eacl-main.69
- Cite (ACL):
- Asaf Yehudai, Arie Cattan, Omri Abend, and Gabriel Stanovsky. 2023. Evaluating and Improving the Coreference Capabilities of Machine Translation Models. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 980–992, Dubrovnik, Croatia. Association for Computational Linguistics.
- Cite (Informal):
- Evaluating and Improving the Coreference Capabilities of Machine Translation Models (Yehudai et al., EACL 2023)
- PDF:
- https://preview.aclanthology.org/revert-3132-ingestion-checklist/2023.eacl-main.69.pdf