Automated Paraphrase Lattice Creation for HyTER Machine Translation Evaluation

Marianna Apidianaki, Guillaume Wisniewski, Anne Cocos, Chris Callison-Burch

Abstract
We propose a variant of a well-known machine translation (MT) evaluation metric, HyTER (Dreyer and Marcu, 2012), which exploits reference translations enriched with meaning equivalent expressions. The original HyTER metric relied on hand-crafted paraphrase networks which restricted its applicability to new data. We test, for the first time, HyTER with automatically built paraphrase lattices. We show that although the metric obtains good results on small and carefully curated data with both manually and automatically selected substitutes, it achieves medium performance on much larger and noisier datasets, demonstrating the limits of the metric for tuning and evaluation of current MT systems.
Anthology ID:
N18-2077
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)
Month:
June
Year:
2018
Address:
New Orleans, Louisiana
Editors:
Marilyn Walker, Heng Ji, Amanda Stent
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
480–485
Language:
URL:
https://aclanthology.org/N18-2077
DOI:
10.18653/v1/N18-2077
Bibkey:
Cite (ACL):
Marianna Apidianaki, Guillaume Wisniewski, Anne Cocos, and Chris Callison-Burch. 2018. Automated Paraphrase Lattice Creation for HyTER Machine Translation Evaluation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 480–485, New Orleans, Louisiana. Association for Computational Linguistics.
Cite (Informal):
Automated Paraphrase Lattice Creation for HyTER Machine Translation Evaluation (Apidianaki et al., NAACL 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/fix-dup-bibkey/N18-2077.pdf
PDF Search