Generalizing Back-Translation in Neural Machine Translation
Miguel Graça, Yunsu Kim, Julian Schamper, Shahram Khadivi, Hermann Ney
Abstract
Back-translation — data augmentation by translating target monolingual data — is a crucial component in modern neural machine translation (NMT). In this work, we reformulate back-translation in the scope of cross-entropy optimization of an NMT model, clarifying its underlying mathematical assumptions and approximations beyond its heuristic usage. Our formulation covers broader synthetic data generation schemes, including sampling from a target-to-source NMT model. With this formulation, we point out fundamental problems of the sampling-based approaches and propose to remedy them by (i) disabling label smoothing for the target-to-source model and (ii) sampling from a restricted search space. Our statements are investigated on the WMT 2018 German <-> English news translation task.- Anthology ID:
- W19-5205
- Volume:
- Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers)
- Month:
- August
- Year:
- 2019
- Address:
- Florence, Italy
- Venue:
- WMT
- SIG:
- SIGMT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 45–52
- Language:
- URL:
- https://aclanthology.org/W19-5205
- DOI:
- 10.18653/v1/W19-5205
- Cite (ACL):
- Miguel Graça, Yunsu Kim, Julian Schamper, Shahram Khadivi, and Hermann Ney. 2019. Generalizing Back-Translation in Neural Machine Translation. In Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pages 45–52, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- Generalizing Back-Translation in Neural Machine Translation (Graça et al., WMT 2019)
- PDF:
- https://preview.aclanthology.org/starsem-semeval-split/W19-5205.pdf