OpenKiwi: An Open Source Framework for Quality Estimation

Fabio Kepler, Jonay Trénous, Marcos Treviso, Miguel Vera, André F. T. Martins


Abstract
We introduce OpenKiwi, a Pytorch-based open source framework for translation quality estimation. OpenKiwi supports training and testing of word-level and sentence-level quality estimation systems, implementing the winning systems of the WMT 2015–18 quality estimation campaigns. We benchmark OpenKiwi on two datasets from WMT 2018 (English-German SMT and NMT), yielding state-of-the-art performance on the word-level tasks and near state-of-the-art in the sentence-level tasks.
Anthology ID:
P19-3020
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations
Month:
July
Year:
2019
Address:
Florence, Italy
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
117–122
Language:
URL:
https://aclanthology.org/P19-3020
DOI:
10.18653/v1/P19-3020
Award:
 Best Demo Paper
Bibkey:
Cite (ACL):
Fabio Kepler, Jonay Trénous, Marcos Treviso, Miguel Vera, and André F. T. Martins. 2019. OpenKiwi: An Open Source Framework for Quality Estimation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 117–122, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
OpenKiwi: An Open Source Framework for Quality Estimation (Kepler et al., ACL 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/P19-3020.pdf
Code
 Unbabel/OpenKiwi