Abstract
The many-to-many multilingual neural machine translation can translate between language pairs unseen during training, i.e., zero-shot translation. Improving zero-shot translation requires the model to learn universal representations and cross-mapping relationships to transfer the knowledge learned on the supervised directions to the zero-shot directions. In this work, we propose the state mover’s distance based on the optimal theory to model the difference of the representations output by the encoder. Then, we bridge the gap between the semantic-equivalent representations of different languages at the token level by minimizing the proposed distance to learn universal representations. Besides, we propose an agreement-based training scheme, which can help the model make consistent predictions based on the semantic-equivalent sentences to learn universal cross-mapping relationships for all translation directions. The experimental results on diverse multilingual datasets show that our method can improve consistently compared with the baseline system and other contrast methods. The analysis proves that our method can better align the semantic space and improve the prediction consistency.- Anthology ID:
- 2022.findings-emnlp.485
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2022
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates
- Editors:
- Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 6492–6504
- Language:
- URL:
- https://aclanthology.org/2022.findings-emnlp.485
- DOI:
- 10.18653/v1/2022.findings-emnlp.485
- Cite (ACL):
- Shuhao Gu and Yang Feng. 2022. Improving Zero-Shot Multilingual Translation with Universal Representations and Cross-Mapping. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6492–6504, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Cite (Informal):
- Improving Zero-Shot Multilingual Translation with Universal Representations and Cross-Mapping (Gu & Feng, Findings 2022)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2022.findings-emnlp.485.pdf