Measuring and Increasing Context Usage in Context-Aware Machine Translation
Patrick Fernandes, Kayo Yin, Graham Neubig, André F. T. Martins
Abstract
Recent work in neural machine translation has demonstrated both the necessity and feasibility of using inter-sentential context, context from sentences other than those currently being translated. However, while many current methods present model architectures that theoretically can use this extra context, it is often not clear how much they do actually utilize it at translation time. In this paper, we introduce a new metric, conditional cross-mutual information, to quantify usage of context by these models. Using this metric, we measure how much document-level machine translation systems use particular varieties of context. We find that target context is referenced more than source context, and that including more context has a diminishing affect on results. We then introduce a new, simple training method, context-aware word dropout, to increase the usage of context by context-aware models. Experiments show that our method not only increases context usage, but also improves the translation quality according to metrics such as BLEU and COMET, as well as performance on anaphoric pronoun resolution and lexical cohesion contrastive datasets.- Anthology ID:
- 2021.acl-long.505
- Volume:
- Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
- Month:
- August
- Year:
- 2021
- Address:
- Online
- Editors:
- Chengqing Zong, Fei Xia, Wenjie Li, Roberto Navigli
- Venues:
- ACL | IJCNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 6467–6478
- Language:
- URL:
- https://preview.aclanthology.org/remove-affiliations/2021.acl-long.505/
- DOI:
- 10.18653/v1/2021.acl-long.505
- Cite (ACL):
- Patrick Fernandes, Kayo Yin, Graham Neubig, and André F. T. Martins. 2021. Measuring and Increasing Context Usage in Context-Aware Machine Translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6467–6478, Online. Association for Computational Linguistics.
- Cite (Informal):
- Measuring and Increasing Context Usage in Context-Aware Machine Translation (Fernandes et al., ACL-IJCNLP 2021)
- PDF:
- https://preview.aclanthology.org/remove-affiliations/2021.acl-long.505.pdf
- Code
- neulab/contextual-mt