Automated Paraphrase Lattice Creation for HyTER Machine Translation Evaluation
Marianna Apidianaki, Guillaume Wisniewski, Anne Cocos, Chris Callison-Burch
Abstract
We propose a variant of a well-known machine translation (MT) evaluation metric, HyTER (Dreyer and Marcu, 2012), which exploits reference translations enriched with meaning equivalent expressions. The original HyTER metric relied on hand-crafted paraphrase networks which restricted its applicability to new data. We test, for the first time, HyTER with automatically built paraphrase lattices. We show that although the metric obtains good results on small and carefully curated data with both manually and automatically selected substitutes, it achieves medium performance on much larger and noisier datasets, demonstrating the limits of the metric for tuning and evaluation of current MT systems.- Anthology ID:
- N18-2077
- Volume:
- Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)
- Month:
- June
- Year:
- 2018
- Address:
- New Orleans, Louisiana
- Venue:
- NAACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 480–485
- Language:
- URL:
- https://aclanthology.org/N18-2077
- DOI:
- 10.18653/v1/N18-2077
- Cite (ACL):
- Marianna Apidianaki, Guillaume Wisniewski, Anne Cocos, and Chris Callison-Burch. 2018. Automated Paraphrase Lattice Creation for HyTER Machine Translation Evaluation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 480–485, New Orleans, Louisiana. Association for Computational Linguistics.
- Cite (Informal):
- Automated Paraphrase Lattice Creation for HyTER Machine Translation Evaluation (Apidianaki et al., NAACL 2018)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/N18-2077.pdf