Unsupervised Neural Machine Translation with Universal Grammar

Zuchao Li, Masao Utiyama, Eiichiro Sumita, Hai Zhao


Abstract
Machine translation usually relies on parallel corpora to provide parallel signals for training. The advent of unsupervised machine translation has brought machine translation away from this reliance, though performance still lags behind traditional supervised machine translation. In unsupervised machine translation, the model seeks symmetric language similarities as a source of weak parallel signal to achieve translation. Chomsky’s Universal Grammar theory postulates that grammar is an innate form of knowledge to humans and is governed by universal principles and constraints. Therefore, in this paper, we seek to leverage such shared grammar clues to provide more explicit language parallel signals to enhance the training of unsupervised machine translation models. Through experiments on multiple typical language pairs, we demonstrate the effectiveness of our proposed approaches.
Anthology ID:
2021.emnlp-main.261
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3249–3264
Language:
URL:
https://aclanthology.org/2021.emnlp-main.261
DOI:
10.18653/v1/2021.emnlp-main.261
Bibkey:
Cite (ACL):
Zuchao Li, Masao Utiyama, Eiichiro Sumita, and Hai Zhao. 2021. Unsupervised Neural Machine Translation with Universal Grammar. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3249–3264, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Unsupervised Neural Machine Translation with Universal Grammar (Li et al., EMNLP 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/2021.emnlp-main.261.pdf
Data
Penn Treebank