Generalizing Back-Translation in Neural Machine Translation
Miguel Graça, Yunsu Kim, Julian Schamper, Shahram Khadivi, Hermann Ney
Abstract
Back-translation — data augmentation by translating target monolingual data — is a crucial component in modern neural machine translation (NMT). In this work, we reformulate back-translation in the scope of cross-entropy optimization of an NMT model, clarifying its underlying mathematical assumptions and approximations beyond its heuristic usage. Our formulation covers broader synthetic data generation schemes, including sampling from a target-to-source NMT model. With this formulation, we point out fundamental problems of the sampling-based approaches and propose to remedy them by (i) disabling label smoothing for the target-to-source model and (ii) sampling from a restricted search space. Our statements are investigated on the WMT 2018 German <-> English news translation task.- Anthology ID:
- W19-5205
- Volume:
- Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers)
- Month:
- August
- Year:
- 2019
- Address:
- Florence, Italy
- Editors:
- Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, André Martins, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Matt Post, Marco Turchi, Karin Verspoor
- Venue:
- WMT
- SIG:
- SIGMT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 45–52
- Language:
- URL:
- https://aclanthology.org/W19-5205
- DOI:
- 10.18653/v1/W19-5205
- Cite (ACL):
- Miguel Graça, Yunsu Kim, Julian Schamper, Shahram Khadivi, and Hermann Ney. 2019. Generalizing Back-Translation in Neural Machine Translation. In Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pages 45–52, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- Generalizing Back-Translation in Neural Machine Translation (Graça et al., WMT 2019)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/W19-5205.pdf