Abstract
We describe our submission of a new metric, SLIDE (Raunak et al., 2023), to the WMT 2023 metrics task. SLIDE is a reference-free quality-estimation metric that works by constructing a fixed sentence-length window over the documents in a test set, concatenating chunks and then sending them for scoring as a single unit by COMET (Rei et al, 2022). We find that SLIDE improves dramatically over its context-less counterpart on the two WMT22 evaluation campaigns (MQM and DA+SQM).- Anthology ID:
- 2023.wmt-1.68
- Volume:
- Proceedings of the Eighth Conference on Machine Translation
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Philipp Koehn, Barry Haddow, Tom Kocmi, Christof Monz
- Venue:
- WMT
- SIG:
- SIGMT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 812–814
- Language:
- URL:
- https://aclanthology.org/2023.wmt-1.68
- DOI:
- 10.18653/v1/2023.wmt-1.68
- Cite (ACL):
- Vikas Raunak, Tom Kocmi, and Matt Post. 2023. Evaluating Metrics for Document-context Evaluation in Machine Translation. In Proceedings of the Eighth Conference on Machine Translation, pages 812–814, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Evaluating Metrics for Document-context Evaluation in Machine Translation (Raunak et al., WMT 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-1/2023.wmt-1.68.pdf