Linguistic Evaluation of German-English Machine Translation Using a Test Suite

Eleftherios Avramidis, Vivien Macketanz, Ursula Strohriegel, Hans Uszkoreit

Abstract
We present the results of the application of a grammatical test suite for German-to-English MT on the systems submitted at WMT19, with a detailed analysis for 107 phenomena organized in 14 categories. The systems still translate wrong one out of four test items in average. Low performance is indicated for idioms, modals, pseudo-clefts, multi-word expressions and verb valency. When compared to last year, there has been a improvement of function words, non verbal agreement and punctuation. More detailed conclusions about particular systems and phenomena are also presented.
Anthology ID:
W19-5351
Volume:
Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)
Month:
August
Year:
2019
Address:
Florence, Italy
Venue:
WMT
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
445–454
Language:
URL:
https://aclanthology.org/W19-5351
DOI:
10.18653/v1/W19-5351
Bibkey:
Cite (ACL):
Eleftherios Avramidis, Vivien Macketanz, Ursula Strohriegel, and Hans Uszkoreit. 2019. Linguistic Evaluation of German-English Machine Translation Using a Test Suite. In Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pages 445–454, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Linguistic Evaluation of German-English Machine Translation Using a Test Suite (Avramidis et al., WMT 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/paclic-22-ingestion/W19-5351.pdf
Supplementary:
 W19-5351.Supplementary.tgz
PDF Search Supplementary