Abstract
We propose an automatic evaluation method of machine translation that uses source language sentences regarded as additional pseudo references. The proposed method evaluates a translation hypothesis in a regression model. The model takes the paired source, reference, and hypothesis sentence all together as an input. A pretrained large scale cross-lingual language model encodes the input to sentence-pair vectors, and the model predicts a human evaluation score with those vectors. Our experiments show that our proposed method using Cross-lingual Language Model (XLM) trained with a translation language modeling (TLM) objective achieves a higher correlation with human judgments than a baseline method that uses only hypothesis and reference sentences. Additionally, using source sentences in our proposed method is confirmed to improve the evaluation performance.- Anthology ID:
- 2020.acl-main.327
- Volume:
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Month:
- July
- Year:
- 2020
- Address:
- Online
- Editors:
- Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3553–3558
- Language:
- URL:
- https://aclanthology.org/2020.acl-main.327
- DOI:
- 10.18653/v1/2020.acl-main.327
- Cite (ACL):
- Kosuke Takahashi, Katsuhito Sudoh, and Satoshi Nakamura. 2020. Automatic Machine Translation Evaluation using Source Language Inputs and Cross-lingual Language Model. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3553–3558, Online. Association for Computational Linguistics.
- Cite (Informal):
- Automatic Machine Translation Evaluation using Source Language Inputs and Cross-lingual Language Model (Takahashi et al., ACL 2020)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2020.acl-main.327.pdf