Abstract
The field of unsupervised machine translation has seen significant advancement from the marriage of the Transformer and the back-translation algorithm. The Transformer is a powerful generative model, and back-translation leverages Transformer’s high-quality translations for iterative self-improvement. However, the Transformer is encumbered by the run-time of autoregressive inference during back-translation, and back-translation is limited by a lack of synthetic data efficiency. We propose a two-for-one improvement to Transformer back-translation: Quick Back-Translation (QBT). QBT re-purposes the encoder as a generative model, and uses encoder-generated sequences to train the decoder in conjunction with the original autoregressive back-translation step, improving data throughput and utilization. Experiments on various WMT benchmarks demonstrate that a relatively small number of refining steps of QBT improve current unsupervised machine translation models, and that QBT dramatically outperforms standard back-translation only method in terms of training efficiency for comparable translation qualities.- Anthology ID:
- 2023.findings-emnlp.571
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2023
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 8521–8534
- Language:
- URL:
- https://aclanthology.org/2023.findings-emnlp.571
- DOI:
- 10.18653/v1/2023.findings-emnlp.571
- Cite (ACL):
- Benjamin Brimacombe and Jiawei Zhou. 2023. Quick Back-Translation for Unsupervised Machine Translation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8521–8534, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Quick Back-Translation for Unsupervised Machine Translation (Brimacombe & Zhou, Findings 2023)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2023.findings-emnlp.571.pdf