Abstract
Mutual learning, where multiple agents learn collaboratively and teach one another, has been shown to be an effective way to distill knowledge for image classification tasks. In this paper, we extend mutual learning to the machine translation task and operate at both the sentence-level and the token-level. Firstly, we co-train multiple agents by using the same parallel corpora. After convergence, each agent selects and learns its poorly predicted tokens from other agents. The poorly predicted tokens are determined by the acceptance-rejection sampling algorithm. Our experiments show that sequential mutual learning at the sentence-level and the token-level improves the results cumulatively. Absolute improvements compared to strong baselines are obtained on various translation tasks. On the IWSLT’14 German-English task, we get a new state-of-the-art BLEU score of 37.0. We also report a competitive result, 29.9 BLEU score, on the WMT’14 English-German task.- Anthology ID:
- 2020.findings-emnlp.155
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2020
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Trevor Cohn, Yulan He, Yang Liu
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1715–1724
- Language:
- URL:
- https://aclanthology.org/2020.findings-emnlp.155
- DOI:
- 10.18653/v1/2020.findings-emnlp.155
- Cite (ACL):
- Baohao Liao, Yingbo Gao, and Hermann Ney. 2020. Multi-Agent Mutual Learning at Sentence-Level and Token-Level for Neural Machine Translation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1715–1724, Online. Association for Computational Linguistics.
- Cite (Informal):
- Multi-Agent Mutual Learning at Sentence-Level and Token-Level for Neural Machine Translation (Liao et al., Findings 2020)
- PDF:
- https://preview.aclanthology.org/add_acl24_videos/2020.findings-emnlp.155.pdf