A Quality Estimation and Quality Evaluation Tool for the Translation Industry
Elena Murgolo, Javad Pourmostafa Roshan Sharami, Dimitar Shterionov
Abstract
With the increase in machine translation (MT) quality over the latest years, it has now become a common practice to integrate MT in the workflow of language service providers (LSPs) and other actors in the translation industry. With MT having a direct impact on the translation workflow, it is important not only to use high-quality MT systems, but also to understand the quality dimension so that the humans involved in the translation workflow can make informed decisions. The evaluation and monitoring of MT output quality has become one of the essential aspects of language technology management in LSPs’ workflows. First, a general practice is to carry out human tests to evaluate MT output quality before deployment. Second, a quality estimate of the translated text, thus after deployment, can inform post editors or even represent post-editing effort. In the former case, based on the quality assessment of a candidate engine, an informed decision can be made whether the engine would be deployed for production or not. In the latter, a quality estimate of the translation output can guide the human post-editor or even make rough approximations of the post-editing effort. Quality of an MT engine can be assessed on document- or on sentence-level. A tool to jointly provide all these functionalities does not exist yet. The overall objective of the project presented in this paper is to develop an MT quality assessment (MTQA) tool that simplifies the quality assessment of MT engines, combining quality evaluation and quality estimation on document- and sentence- level.- Anthology ID:
- 2022.eamt-1.43
- Volume:
- Proceedings of the 23rd Annual Conference of the European Association for Machine Translation
- Month:
- June
- Year:
- 2022
- Address:
- Ghent, Belgium
- Editors:
- Helena Moniz, Lieve Macken, Andrew Rufener, Loïc Barrault, Marta R. Costa-jussà, Christophe Declercq, Maarit Koponen, Ellie Kemp, Spyridon Pilos, Mikel L. Forcada, Carolina Scarton, Joachim Van den Bogaert, Joke Daems, Arda Tezcan, Bram Vanroy, Margot Fonteyne
- Venue:
- EAMT
- SIG:
- Publisher:
- European Association for Machine Translation
- Note:
- Pages:
- 307–308
- Language:
- URL:
- https://aclanthology.org/2022.eamt-1.43
- DOI:
- Cite (ACL):
- Elena Murgolo, Javad Pourmostafa Roshan Sharami, and Dimitar Shterionov. 2022. A Quality Estimation and Quality Evaluation Tool for the Translation Industry. In Proceedings of the 23rd Annual Conference of the European Association for Machine Translation, pages 307–308, Ghent, Belgium. European Association for Machine Translation.
- Cite (Informal):
- A Quality Estimation and Quality Evaluation Tool for the Translation Industry (Murgolo et al., EAMT 2022)
- PDF:
- https://preview.aclanthology.org/ingest-2024-clasp/2022.eamt-1.43.pdf