Abstract
Despite increasing efforts to improve evaluation of machine translation (MT) by going beyond the sentence level to the document level, the definition of what exactly constitutes a “document level” is still not clear. This work deals with the context span necessary for a more reliable MT evaluation. We report results from a series of surveys involving three domains and 18 target languages designed to identify the necessary context span as well as issues related to it. Our findings indicate that, despite the fact that some issues and spans are strongly dependent on domain and on the target language, a number of common patterns can be observed so that general guidelines for context-aware MT evaluation can be drawn.- Anthology ID:
- 2020.lrec-1.461
- Volume:
- Proceedings of the Twelfth Language Resources and Evaluation Conference
- Month:
- May
- Year:
- 2020
- Address:
- Marseille, France
- Editors:
- Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
- Venue:
- LREC
- SIG:
- Publisher:
- European Language Resources Association
- Note:
- Pages:
- 3735–3742
- Language:
- English
- URL:
- https://aclanthology.org/2020.lrec-1.461
- DOI:
- Cite (ACL):
- Sheila Castilho, Maja Popović, and Andy Way. 2020. On Context Span Needed for Machine Translation Evaluation. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3735–3742, Marseille, France. European Language Resources Association.
- Cite (Informal):
- On Context Span Needed for Machine Translation Evaluation (Castilho et al., LREC 2020)
- PDF:
- https://preview.aclanthology.org/landing_page/2020.lrec-1.461.pdf