Abstract
Beam search decoding is the de-facto method for decoding auto-regressive Neural Machine Translation (NMT) models, including multilingual NMT where the target language is specified as an input. However, decoding multilingual NMT models commonly produces off-target translations – yielding translation outputs not in the intended language.In this paper, we first conduct an error analysis of off-target translations for a strong multilingual NMT model and identify how these decodings are produced during beam search. We then propose Language-informed Beam Search (LiBS), a general decoding algorithm incorporating an off-the-shelf Language Identification (LiD) model into beam search decoding to reduce off-target translations. LiBS is an inference-time procedure that is NMT-model agnostic and does not require any additional parallel data. Results show that our proposed LiBS algorithm on average improves +1.1 BLEU and +0.9 BLEU on WMT and OPUS datasets, and reduces off-target rates from 22.9% to 7.7% and 65.8% to 25.3% respectively.- Anthology ID:
- 2024.findings-acl.932
- Volume:
- Findings of the Association for Computational Linguistics ACL 2024
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand and virtual meeting
- Editors:
- Lun-Wei Ku, Andre Martins, Vivek Srikumar
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 15761–15772
- Language:
- URL:
- https://aclanthology.org/2024.findings-acl.932
- DOI:
- Cite (ACL):
- Yilin Yang, Stefan Lee, and Prasad Tadepalli. 2024. Language-Informed Beam Search Decoding for Multilingual Machine Translation. In Findings of the Association for Computational Linguistics ACL 2024, pages 15761–15772, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
- Cite (Informal):
- Language-Informed Beam Search Decoding for Multilingual Machine Translation (Yang et al., Findings 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.findings-acl.932.pdf