Relations between comprehensibility and adequacy errors in machine translation output

Maja Popović


Abstract
This work presents a detailed analysis of translation errors perceived by readers as comprehensibility and/or adequacy issues. The main finding is that good comprehensibility, similarly to good fluency, can mask a number of adequacy errors. Of all major adequacy errors, 30% were fully comprehensible, thus fully misleading the reader to accept the incorrect information. Another 25% of major adequacy errors were perceived as almost comprehensible, thus being potentially misleading. Also, a vast majority of omissions (about 70%) is hidden by comprehensibility. Further analysis of misleading translations revealed that the most frequent error types are ambiguity, mistranslation, noun phrase error, word-by-word translation, untranslated word, subject-verb agreement, and spelling error in the source text. However, none of these error types appears exclusively in misleading translations, but are also frequent in fully incorrect (incomprehensible inadequate) and discarded correct (incomprehensible adequate) translations. Deeper analysis is needed to potentially detect underlying phenomena specifically related to misleading translations.
Anthology ID:
2020.conll-1.19
Volume:
Proceedings of the 24th Conference on Computational Natural Language Learning
Month:
November
Year:
2020
Address:
Online
Venue:
CoNLL
SIG:
SIGNLL
Publisher:
Association for Computational Linguistics
Note:
Pages:
256–264
Language:
URL:
https://aclanthology.org/2020.conll-1.19
DOI:
10.18653/v1/2020.conll-1.19
Bibkey:
Cite (ACL):
Maja Popović. 2020. Relations between comprehensibility and adequacy errors in machine translation output. In Proceedings of the 24th Conference on Computational Natural Language Learning, pages 256–264, Online. Association for Computational Linguistics.
Cite (Informal):
Relations between comprehensibility and adequacy errors in machine translation output (Popović, CoNLL 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/2020.conll-1.19.pdf