Are Multiple Reference Translations Necessary? Investigating the Value of Paraphrased Reference Translations in Parameter Optimization

Nitin Madnani, Philip Resnik, Bonnie J. Dorr, Richard Schwartz

Abstract
Most state-of-the-art statistical machine translation systems use log-linear models, which are defined in terms of hypothesis features and weights for those features. It is standard to tune the feature weights in order to maximize a translation quality metric, using held-out test sentences and their corresponding reference translations. However, obtaining reference translations is expensive. In our earlier work (Madnani et al., 2007), we introduced a new full-sentence paraphrase technique, based on English-to-English decoding with an MT system, and demonstrated that the resulting paraphrases can be used to cut the number of human reference translations needed in half. In this paper, we take the idea a step further, asking how far it is possible to get with just a single good reference translation for each item in the development set. Our analysis suggests that it is necessary to invest in four or more human translations in order to significantly improve on a single translation augmented by monolingual paraphrases.
Anthology ID:
2008.amta-papers.13
Volume:
Proceedings of the 8th Conference of the Association for Machine Translation in the Americas: Research Papers
Month:
October 21-25
Year:
2008
Address:
Waikiki, USA
Venue:
AMTA
SIG:
Publisher:
Association for Machine Translation in the Americas
Note:
Pages:
143–152
Language:
URL:
https://aclanthology.org/2008.amta-papers.13
DOI:
Bibkey:
Cite (ACL):
Nitin Madnani, Philip Resnik, Bonnie J. Dorr, and Richard Schwartz. 2008. Are Multiple Reference Translations Necessary? Investigating the Value of Paraphrased Reference Translations in Parameter Optimization. In Proceedings of the 8th Conference of the Association for Machine Translation in the Americas: Research Papers, pages 143–152, Waikiki, USA. Association for Machine Translation in the Americas.
Cite (Informal):
Are Multiple Reference Translations Necessary? Investigating the Value of Paraphrased Reference Translations in Parameter Optimization (Madnani et al., AMTA 2008)
Copy Citation:
PDF:
https://preview.aclanthology.org/add_acl24_videos/2008.amta-papers.13.pdf
PDF Search