Quality Estimation via Backtranslation at the WMT 2022 Quality Estimation Task
Sweta Agrawal, Nikita Mehandru, Niloufar Salehi, Marine Carpuat
Abstract
This paper describes submission to the WMT 2022 Quality Estimation shared task (Task 1: sentence-level quality prediction). We follow a simple and intuitive approach, which consists of estimating MT quality by automatically back-translating hypotheses into the source language using a multilingual MT system. We then compare the resulting backtranslation with the original source using standard MT evaluation metrics. We find that even the best-performing backtranslation-based scores perform substantially worse than supervised QE systems, including the organizers’ baseline. However, combining backtranslation-based metrics with off-the-shelf QE scorers improves correlation with human judgments, suggesting that they can indeed complement a supervised QE system.- Anthology ID:
- 2022.wmt-1.54
- Original:
- 2022.wmt-1.54v1
- Version 2:
- 2022.wmt-1.54v2
- Volume:
- Proceedings of the Seventh Conference on Machine Translation (WMT)
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates (Hybrid)
- Venue:
- WMT
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 593–596
- Language:
- URL:
- https://aclanthology.org/2022.wmt-1.54
- DOI:
- Cite (ACL):
- Sweta Agrawal, Nikita Mehandru, Niloufar Salehi, and Marine Carpuat. 2022. Quality Estimation via Backtranslation at the WMT 2022 Quality Estimation Task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 593–596, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
- Cite (Informal):
- Quality Estimation via Backtranslation at the WMT 2022 Quality Estimation Task (Agrawal et al., WMT 2022)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2022.wmt-1.54.pdf