Abstract
This report describes the Minimum Bayes Risk Quality Estimation (MBR-QE) submission to the Workshop on Machine Translation’s 2023 Metrics Shared Task. MBR decoding with neural utility metrics like BLEURT is known to be effective in generating high quality machine translations. We use the underlying technique of MBR decoding and develop an MBR based reference-free quality estimation metric. Our method uses an evaluator machine translation system and a reference-based utility metric (specifically BLEURT and MetricX) to calculate a quality estimation score of a model. We report results related to comparing different MBR configurations and utility metrics.- Anthology ID:
- 2023.wmt-1.67
- Volume:
- Proceedings of the Eighth Conference on Machine Translation
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Philipp Koehn, Barry Haddow, Tom Kocmi, Christof Monz
- Venue:
- WMT
- SIG:
- SIGMT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 806–811
- Language:
- URL:
- https://aclanthology.org/2023.wmt-1.67
- DOI:
- 10.18653/v1/2023.wmt-1.67
- Cite (ACL):
- Subhajit Naskar, Daniel Deutsch, and Markus Freitag. 2023. Quality Estimation Using Minimum Bayes Risk. In Proceedings of the Eighth Conference on Machine Translation, pages 806–811, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Quality Estimation Using Minimum Bayes Risk (Naskar et al., WMT 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2023.wmt-1.67.pdf