Improving Beam Search by Removing Monotonic Constraint for Neural Machine Translation

Raphael Shu, Hideki Nakayama


Abstract
To achieve high translation performance, neural machine translation models usually rely on the beam search algorithm for decoding sentences. The beam search finds good candidate translations by considering multiple hypotheses of translations simultaneously. However, as the algorithm produces hypotheses in a monotonic left-to-right order, a hypothesis can not be revisited once it is discarded. We found such monotonicity forces the algorithm to sacrifice some good decoding paths. To mitigate this problem, we relax the monotonic constraint of the beam search by maintaining all found hypotheses in a single priority queue and using a universal score function for hypothesis selection. The proposed algorithm allows discarded hypotheses to be recovered in a later step. Despite its simplicity, we show that the proposed decoding algorithm enhances the quality of selected hypotheses and improve the translations even for high-performance models in English-Japanese translation task.
Anthology ID:
P18-2054
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
339–344
Language:
URL:
https://aclanthology.org/P18-2054
DOI:
10.18653/v1/P18-2054
Bibkey:
Cite (ACL):
Raphael Shu and Hideki Nakayama. 2018. Improving Beam Search by Removing Monotonic Constraint for Neural Machine Translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 339–344, Melbourne, Australia. Association for Computational Linguistics.
Cite (Informal):
Improving Beam Search by Removing Monotonic Constraint for Neural Machine Translation (Shu & Nakayama, ACL 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/P18-2054.pdf
Note:
 P18-2054.Notes.pdf
Poster:
 P18-2054.Poster.pdf