Top-Rank Enhanced Listwise Optimization for Statistical Machine Translation

Huadong Chen, Shujian Huang, David Chiang, Xinyu Dai, Jiajun Chen

Abstract
Pairwise ranking methods are the most widely used discriminative training approaches for structure prediction problems in natural language processing (NLP). Decomposing the problem of ranking hypotheses into pairwise comparisons enables simple and efficient solutions. However, neglecting the global ordering of the hypothesis list may hinder learning. We propose a listwise learning framework for structure prediction problems such as machine translation. Our framework directly models the entire translation list’s ordering to learn parameters which may better fit the given listwise samples. Furthermore, we propose top-rank enhanced loss functions, which are more sensitive to ranking errors at higher positions. Experiments on a large-scale Chinese-English translation task show that both our listwise learning framework and top-rank enhanced listwise losses lead to significant improvements in translation quality.
Anthology ID:
K17-1011
Volume:
Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)
Month:
August
Year:
2017
Address:
Vancouver, Canada
Editors:
Roger Levy, Lucia Specia
Venue:
CoNLL
SIG:
SIGNLL
Publisher:
Association for Computational Linguistics
Note:
Pages:
90–99
Language:
URL:
https://aclanthology.org/K17-1011
DOI:
10.18653/v1/K17-1011
Bibkey:
Cite (ACL):
Huadong Chen, Shujian Huang, David Chiang, Xinyu Dai, and Jiajun Chen. 2017. Top-Rank Enhanced Listwise Optimization for Statistical Machine Translation. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 90–99, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
Top-Rank Enhanced Listwise Optimization for Statistical Machine Translation (Chen et al., CoNLL 2017)
Copy Citation:
PDF:
https://preview.aclanthology.org/add_acl24_videos/K17-1011.pdf
PDF Search