DELA Corpus - A Document-Level Corpus Annotated with Context-Related Issues
Sheila Castilho, João Lucas Cavalheiro Camargo, Miguel Menezes, Andy Way
Abstract
Recently, the Machine Translation (MT) community has become more interested in document-level evaluation especially in light of reactions to claims of “human parity”, since examining the quality at the level of the document rather than at the sentence level allows for the assessment of suprasentential context, providing a more reliable evaluation. This paper presents a document-level corpus annotated in English with context-aware issues that arise when translating from English into Brazilian Portuguese, namely ellipsis, gender, lexical ambiguity, number, reference, and terminology, with six different domains. The corpus can be used as a challenge test set for evaluation and as a training/testing corpus for MT as well as for deep linguistic analysis of context issues. To the best of our knowledge, this is the first corpus of its kind.- Anthology ID:
- 2021.wmt-1.63
- Volume:
- Proceedings of the Sixth Conference on Machine Translation
- Month:
- November
- Year:
- 2021
- Address:
- Online
- Editors:
- Loic Barrault, Ondrej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussa, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Tom Kocmi, Andre Martins, Makoto Morishita, Christof Monz
- Venue:
- WMT
- SIG:
- SIGMT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 566–577
- Language:
- URL:
- https://aclanthology.org/2021.wmt-1.63
- DOI:
- Cite (ACL):
- Sheila Castilho, João Lucas Cavalheiro Camargo, Miguel Menezes, and Andy Way. 2021. DELA Corpus - A Document-Level Corpus Annotated with Context-Related Issues. In Proceedings of the Sixth Conference on Machine Translation, pages 566–577, Online. Association for Computational Linguistics.
- Cite (Informal):
- DELA Corpus - A Document-Level Corpus Annotated with Context-Related Issues (Castilho et al., WMT 2021)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2021.wmt-1.63.pdf
- Code
- sheilacastilho/dela-project