REUSE: REference-free UnSupervised Quality Estimation Metric

Ananya Mukherjee, Manish Shrivastava

Abstract
This paper describes our submission to the WMT2022 shared metrics task. Our unsupervised metric estimates the translation quality at chunk-level and sentence-level. Source and target sentence chunks are retrieved by using a multi-lingual chunker. The chunk-level similarity is computed by leveraging BERT contextual word embeddings and sentence similarity scores are calculated by leveraging sentence embeddings of Language-Agnostic BERT models. The final quality estimation score is obtained by mean pooling the chunk-level and sentence-level similarity scores. This paper outlines our experiments and also reports the correlation with human judgements for en-de, en-ru and zh-en language pairs of WMT17, WMT18 and WMT19 test sets.
Anthology ID:
2022.wmt-1.50
Volume:
Proceedings of the Seventh Conference on Machine Translation (WMT)
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates (Hybrid)
Editors:
Philipp Koehn, Loïc Barrault, Ondřej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Tom Kocmi, André Martins, Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Marco Turchi, Marcos Zampieri
Venue:
WMT
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
564–568
Language:
URL:
https://aclanthology.org/2022.wmt-1.50
DOI:
Bibkey:
Cite (ACL):
Ananya Mukherjee and Manish Shrivastava. 2022. REUSE: REference-free UnSupervised Quality Estimation Metric. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 564–568, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Cite (Informal):
REUSE: REference-free UnSupervised Quality Estimation Metric (Mukherjee & Shrivastava, WMT 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl24-info/2022.wmt-1.50.pdf
PDF Search