Saliency-driven Word Alignment Interpretation for Neural Machine Translation

Shuoyang Ding, Hainan Xu, Philipp Koehn


Abstract
Despite their original goal to jointly learn to align and translate, Neural Machine Translation (NMT) models, especially Transformer, are often perceived as not learning interpretable word alignments. In this paper, we show that NMT models do learn interpretable word alignments, which could only be revealed with proper interpretation methods. We propose a series of such methods that are model-agnostic, are able to be applied either offline or online, and do not require parameter update or architectural change. We show that under the force decoding setup, the alignments induced by our interpretation method are of better quality than fast-align for some systems, and when performing free decoding, they agree well with the alignments induced by automatic alignment tools.
Anthology ID:
W19-5201
Volume:
Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers)
Month:
August
Year:
2019
Address:
Florence, Italy
Venue:
WMT
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–12
Language:
URL:
https://aclanthology.org/W19-5201
DOI:
10.18653/v1/W19-5201
Bibkey:
Cite (ACL):
Shuoyang Ding, Hainan Xu, and Philipp Koehn. 2019. Saliency-driven Word Alignment Interpretation for Neural Machine Translation. In Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pages 1–12, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Saliency-driven Word Alignment Interpretation for Neural Machine Translation (Ding et al., WMT 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/starsem-semeval-split/W19-5201.pdf
Presentation:
 W19-5201.Presentation.pdf
Code
 shuoyangd/meerkat