Abstract
Although existing multilingual neural machine translation (MNMT) models have demonstrated remarkable performance to handle multiple translation directions in a single model and achieved zero-shot translation between language pairs unseen in training, they still suffer from relatively poor translation qualities for some language pairs. A practical scenario is that how to continually update MNMT models for both supervised and zero-shot translations when limited new data arrives. To this end, we propose a two-stage approach that encourages original models to acquire language-agnostic multilingual representations from new data, and preserves the model architecture without introducing parameters. Experimental results and further analysis demonstrate that our method can efficiently improve performance of existing MNMT models in translation directions where they are initially weak, and mitigates the degeneration in the original well-performing translation directions, offering flexibility in the real-world scenario.- Anthology ID:
- 2023.emnlp-main.860
- Volume:
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 13938–13951
- Language:
- URL:
- https://aclanthology.org/2023.emnlp-main.860
- DOI:
- 10.18653/v1/2023.emnlp-main.860
- Cite (ACL):
- Kaiyu Huang, Peng Li, Junpeng Liu, Maosong Sun, and Yang Liu. 2023. Learn and Consolidate: Continual Adaptation for Zero-Shot and Multilingual Neural Machine Translation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13938–13951, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Learn and Consolidate: Continual Adaptation for Zero-Shot and Multilingual Neural Machine Translation (Huang et al., EMNLP 2023)
- PDF:
- https://preview.aclanthology.org/landing_page/2023.emnlp-main.860.pdf