Abstract
This paper describes our participating system in the shared task Explainable quality estimation of 2nd Workshop on Evaluation & Comparison of NLP Systems. The task of quality estimation (QE, a.k.a. reference-free evaluation) is to predict the quality of MT output at inference time without access to reference translations. In this proposed work, we first build a word-level quality estimation model, then we finetune this model for sentence-level QE. Our proposed models achieve near state-of-the-art results. In the word-level QE, we place 2nd and 3rd on the supervised Ro-En and Et-En test sets. In the sentence-level QE, we achieve a relative improvement of 8.86% (Ro-En) and 10.6% (Et-En) in terms of the Pearson correlation coefficient over the baseline model.- Anthology ID:
- 2021.eval4nlp-1.24
- Volume:
- Proceedings of the 2nd Workshop on Evaluation and Comparison of NLP Systems
- Month:
- November
- Year:
- 2021
- Address:
- Punta Cana, Dominican Republic
- Editors:
- Yang Gao, Steffen Eger, Wei Zhao, Piyawat Lertvittayakumjorn, Marina Fomicheva
- Venue:
- Eval4NLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 250–255
- Language:
- URL:
- https://aclanthology.org/2021.eval4nlp-1.24
- DOI:
- 10.18653/v1/2021.eval4nlp-1.24
- Cite (ACL):
- Peter Polák, Muskaan Singh, and Ondřej Bojar. 2021. Explainable Quality Estimation: CUNI Eval4NLP Submission. In Proceedings of the 2nd Workshop on Evaluation and Comparison of NLP Systems, pages 250–255, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Explainable Quality Estimation: CUNI Eval4NLP Submission (Polák et al., Eval4NLP 2021)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2021.eval4nlp-1.24.pdf
- Code
- pe-trik/eval4nlp-2021