Adapting Machine Translation Education to the Neural Era: A Case Study of MT Quality Assessment

Lieve Macken, Bram Vanroy, Arda Tezcan

Abstract
The use of automatic evaluation metrics to assess Machine Translation (MT) quality is well established in the translation industry. Whereas it is relatively easy to cover the word- and character-based metrics in an MT course, it is less obvious to integrate the newer neural metrics. In this paper we discuss how we introduced the topic of MT quality assessment in a course for translation students. We selected three English source texts, each having a different difficulty level and style, and let the students translate the texts into their L1 and reflect upon translation difficulty. Afterwards, the students were asked to assess MT quality for the same texts using different methods and to critically reflect upon obtained results. The students had access to the MATEO web interface, which contains word- and character-based metrics as well as neural metrics. The students used two different reference translations: their own translations and professional translations of the three texts. We not only synthesise the comments of the students, but also present the results of some cross-lingual analyses on nine different language pairs.
Anthology ID:
2023.eamt-1.30
Volume:
Proceedings of the 24th Annual Conference of the European Association for Machine Translation
Month:
June
Year:
2023
Address:
Tampere, Finland
Editors:
Mary Nurminen, Judith Brenner, Maarit Koponen, Sirkku Latomaa, Mikhail Mikhailov, Frederike Schierl, Tharindu Ranasinghe, Eva Vanmassenhove, Sergi Alvarez Vidal, Nora Aranberri, Mara Nunziatini, Carla Parra Escartín, Mikel Forcada, Maja Popovic, Carolina Scarton, Helena Moniz
Venue:
EAMT
SIG:
Publisher:
European Association for Machine Translation
Note:
Pages:
305–314
Language:
URL:
https://aclanthology.org/2023.eamt-1.30
DOI:
Bibkey:
Cite (ACL):
Lieve Macken, Bram Vanroy, and Arda Tezcan. 2023. Adapting Machine Translation Education to the Neural Era: A Case Study of MT Quality Assessment. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pages 305–314, Tampere, Finland. European Association for Machine Translation.
Cite (Informal):
Adapting Machine Translation Education to the Neural Era: A Case Study of MT Quality Assessment (Macken et al., EAMT 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/add_acl24_videos/2023.eamt-1.30.pdf
PDF Search