Enriching Non-Autoregressive Transformer with Syntactic and Semantic Structures for Neural Machine Translation

Ye Liu, Yao Wan, Jianguo Zhang, Wenting Zhao, Philip Yu

Abstract
The non-autoregressive models have boosted the efficiency of neural machine translation through parallelized decoding at the cost of effectiveness, when comparing with the autoregressive counterparts. In this paper, we claim that the syntactic and semantic structures among natural language are critical for non-autoregressive machine translation and can further improve the performance. However, these structures are rarely considered in the existing non-autoregressive models. Inspired by this intuition, we propose to incorporate the explicit syntactic and semantic structure of languages into a non-autoregressive Transformer, for the task of neural machine translation. Moreover, we also consider the intermediate latent alignment within target sentences to better learn the long-term token dependencies. Experimental results on two real-world datasets (i.e., WMT14 En-De and WMT16 En- Ro) show that our model achieves a significantly faster speed, as well as keeps the translation quality when compared with several state-of-the-art non-autoregressive models.
Anthology ID:
2021.eacl-main.105
Volume:
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
Month:
April
Year:
2021
Address:
Online
Editors:
Paola Merlo, Jorg Tiedemann, Reut Tsarfaty
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1235–1244
Language:
URL:
https://preview.aclanthology.org/build-pipeline-with-new-library/2021.eacl-main.105/
DOI:
10.18653/v1/2021.eacl-main.105
Bibkey:
Cite (ACL):
Ye Liu, Yao Wan, Jianguo Zhang, Wenting Zhao, and Philip Yu. 2021. Enriching Non-Autoregressive Transformer with Syntactic and Semantic Structures for Neural Machine Translation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1235–1244, Online. Association for Computational Linguistics.
Cite (Informal):
Enriching Non-Autoregressive Transformer with Syntactic and Semantic Structures for Neural Machine Translation (Liu et al., EACL 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/build-pipeline-with-new-library/2021.eacl-main.105.pdf
PDF Search Fix metadata