BERGAMOT-LATTE Submissions for the WMT20 Quality Estimation Shared Task
Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Vishrav Chaudhary, Mark Fishel, Francisco Guzmán, Lucia Specia
Abstract
This paper presents our submission to the WMT2020 Shared Task on Quality Estimation (QE). We participate in Task and Task 2 focusing on sentence-level prediction. We explore (a) a black-box approach to QE based on pre-trained representations; and (b) glass-box approaches that leverage various indicators that can be extracted from the neural MT systems. In addition to training a feature-based regression model using glass-box quality indicators, we also test whether they can be used to predict MT quality directly with no supervision. We assess our systems in a multi-lingual setting and show that both types of approaches generalise well across languages. Our black-box QE models tied for the winning submission in four out of seven language pairs inTask 1, thus demonstrating very strong performance. The glass-box approaches also performed competitively, representing a light-weight alternative to the neural-based models.- Anthology ID:
- 2020.wmt-1.116
- Volume:
- Proceedings of the Fifth Conference on Machine Translation
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Venue:
- WMT
- SIG:
- SIGMT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1010–1017
- Language:
- URL:
- https://aclanthology.org/2020.wmt-1.116
- DOI:
- Cite (ACL):
- Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Vishrav Chaudhary, Mark Fishel, Francisco Guzmán, and Lucia Specia. 2020. BERGAMOT-LATTE Submissions for the WMT20 Quality Estimation Shared Task. In Proceedings of the Fifth Conference on Machine Translation, pages 1010–1017, Online. Association for Computational Linguistics.
- Cite (Informal):
- BERGAMOT-LATTE Submissions for the WMT20 Quality Estimation Shared Task (Fomicheva et al., WMT 2020)
- PDF:
- https://preview.aclanthology.org/nodalida-main-page/2020.wmt-1.116.pdf