Guiding Neural Machine Translation with Semantic Kernels
Ping Guo, Yue Hu, Xiangpeng Wei, Yubing Ren, Yunpeng Li, Luxi Xing, Yuqiang Xie
Abstract
Machine Translation task has made great progress with the help of auto-regressive decoding paradigm and Transformer architecture. In this paradigm, though the encoder can obtain global source representations, the decoder can only use translation history to determine the current word. Previous promising works attempted to address this issue by applying a draft or a fixed-length semantic embedding as target-side global information. However, these methods either degrade model efficiency or show limitations in expressing semantics. Motivated by Functional Equivalence Theory, we extract several semantic kernels from a source sentence, each of which can express one semantic segment of the original sentence. Together, these semantic kernels can capture global semantic information, and we project them into target embedding space to guide target sentence generation. We further force our model to use semantic kernels at each decoding step through an adaptive mask algorithm. Empirical studies on various machine translation benchmarks show that our approach gains approximately an improvement of 1 BLEU score on most benchmarks over the Transformer baseline and about 1.7 times faster than previous works on average at inference time.- Anthology ID:
- 2022.findings-emnlp.541
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2022
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates
- Editors:
- Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 7316–7327
- Language:
- URL:
- https://aclanthology.org/2022.findings-emnlp.541
- DOI:
- 10.18653/v1/2022.findings-emnlp.541
- Cite (ACL):
- Ping Guo, Yue Hu, Xiangpeng Wei, Yubing Ren, Yunpeng Li, Luxi Xing, and Yuqiang Xie. 2022. Guiding Neural Machine Translation with Semantic Kernels. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 7316–7327, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Cite (Informal):
- Guiding Neural Machine Translation with Semantic Kernels (Guo et al., Findings 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-1/2022.findings-emnlp.541.pdf