Combining Character and Word Information in Neural Machine Translation Using a Multi-Level Attention

Huadong Chen, Shujian Huang, David Chiang, Xinyu Dai, Jiajun Chen

Abstract
Natural language sentences, being hierarchical, can be represented at different levels of granularity, like words, subwords, or characters. But most neural machine translation systems require the sentence to be represented as a sequence at a single level of granularity. It can be difficult to determine which granularity is better for a particular translation task. In this paper, we improve the model by incorporating multiple levels of granularity. Specifically, we propose (1) an encoder with character attention which augments the (sub)word-level representation with character-level information; (2) a decoder with multiple attentions that enable the representations from different levels of granularity to control the translation cooperatively. Experiments on three translation tasks demonstrate that our proposed models outperform the standard word-based model, the subword-based model, and a strong character-based model.
Anthology ID:
N18-1116
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)
Month:
June
Year:
2018
Address:
New Orleans, Louisiana
Editors:
Marilyn Walker, Heng Ji, Amanda Stent
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1284–1293
Language:
URL:
https://aclanthology.org/N18-1116
DOI:
10.18653/v1/N18-1116
Bibkey:
Cite (ACL):
Huadong Chen, Shujian Huang, David Chiang, Xinyu Dai, and Jiajun Chen. 2018. Combining Character and Word Information in Neural Machine Translation Using a Multi-Level Attention. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1284–1293, New Orleans, Louisiana. Association for Computational Linguistics.
Cite (Informal):
Combining Character and Word Information in Neural Machine Translation Using a Multi-Level Attention (Chen et al., NAACL 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/ml4al-ingestion/N18-1116.pdf
PDF Search