Abstract
We propose WMDO, a metric based on distance between distributions in the semantic vector space. Matching in the semantic space has been investigated for translation evaluation, but the constraints of a translation’s word order have not been fully explored. Building on the Word Mover’s Distance metric and various word embeddings, we introduce a fragmentation penalty to account for fluency of a translation. This word order extension is shown to perform better than standard WMD, with promising results against other types of metrics.- Anthology ID:
- W19-5356
- Volume:
- Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)
- Month:
- August
- Year:
- 2019
- Address:
- Florence, Italy
- Editors:
- Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, André Martins, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Matt Post, Marco Turchi, Karin Verspoor
- Venue:
- WMT
- SIG:
- SIGMT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 494–500
- Language:
- URL:
- https://aclanthology.org/W19-5356
- DOI:
- 10.18653/v1/W19-5356
- Cite (ACL):
- Julian Chow, Lucia Specia, and Pranava Madhyastha. 2019. WMDO: Fluency-based Word Mover’s Distance for Machine Translation Evaluation. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 494–500, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- WMDO: Fluency-based Word Mover’s Distance for Machine Translation Evaluation (Chow et al., WMT 2019)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-1/W19-5356.pdf