Context-Aware Smoothing for Neural Machine Translation

Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, Tiejun Zhao


Abstract
In Neural Machine Translation (NMT), each word is represented as a low-dimension, real-value vector for encoding its syntax and semantic information. This means that even if the word is in a different sentence context, it is represented as the fixed vector to learn source representation. Moreover, a large number of Out-Of-Vocabulary (OOV) words, which have different syntax and semantic information, are represented as the same vector representation of “unk”. To alleviate this problem, we propose a novel context-aware smoothing method to dynamically learn a sentence-specific vector for each word (including OOV words) depending on its local context words in a sentence. The learned context-aware representation is integrated into the NMT to improve the translation performance. Empirical results on NIST Chinese-to-English translation task show that the proposed approach achieves 1.78 BLEU improvements on average over a strong attentional NMT, and outperforms some existing systems.
Anthology ID:
I17-1002
Volume:
Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
November
Year:
2017
Address:
Taipei, Taiwan
Venue:
IJCNLP
SIG:
Publisher:
Asian Federation of Natural Language Processing
Note:
Pages:
11–20
Language:
URL:
https://aclanthology.org/I17-1002
DOI:
Bibkey:
Cite (ACL):
Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, and Tiejun Zhao. 2017. Context-Aware Smoothing for Neural Machine Translation. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 11–20, Taipei, Taiwan. Asian Federation of Natural Language Processing.
Cite (Informal):
Context-Aware Smoothing for Neural Machine Translation (Chen et al., IJCNLP 2017)
Copy Citation:
PDF:
https://preview.aclanthology.org/author-url/I17-1002.pdf