Bag-of-Words as Target for Neural Machine Translation

Shuming Ma, Xu Sun, Yizhong Wang, Junyang Lin


Abstract
A sentence can be translated into more than one correct sentences. However, most of the existing neural machine translation models only use one of the correct translations as the targets, and the other correct sentences are punished as the incorrect sentences in the training stage. Since most of the correct translations for one sentence share the similar bag-of-words, it is possible to distinguish the correct translations from the incorrect ones by the bag-of-words. In this paper, we propose an approach that uses both the sentences and the bag-of-words as targets in the training stage, in order to encourage the model to generate the potentially correct sentences that are not appeared in the training set. We evaluate our model on a Chinese-English translation dataset, and experiments show our model outperforms the strong baselines by the BLEU score of 4.55.
Anthology ID:
P18-2053
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Editors:
Iryna Gurevych, Yusuke Miyao
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
332–338
Language:
URL:
https://aclanthology.org/P18-2053
DOI:
10.18653/v1/P18-2053
Bibkey:
Cite (ACL):
Shuming Ma, Xu Sun, Yizhong Wang, and Junyang Lin. 2018. Bag-of-Words as Target for Neural Machine Translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 332–338, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Bag-of-Words as Target for Neural Machine Translation (Ma et al., ACL 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-1/P18-2053.pdf
Code
 lancopku/bag-of-words