A Comparison of MT Errors and ESL Errors

Homa B. Hashemi, Rebecca Hwa

Abstract
Generating fluent and grammatical sentences is a major goal for both Machine Translation (MT) and second-language Grammar Error Correction (GEC), but there have not been a lot of cross-fertilization between the two research communities. Arguably, an automatic translate-to-English system might be seen as an English as a Second Language (ESL) writer whose native language is the source language. This paper investigates whether research findings from the GEC community may help with characterizing MT error analysis. We describe a method for the automatic classification of MT errors according to English as a Second Language (ESL) error categories and conduct a large comparison experiment that includes both high-performing and low-performing translate-to-English MT systems for several source languages. Comparing the distribution of MT error types for all the systems suggests that MT systems have fairly similar distributions regardless of their source languages, and the high-performing MT systems have error distributions that are more similar to those of the low-performing MT systems than to those of ESL learners with the same L1.
Anthology ID:
L14-1695
Volume:
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Month:
May
Year:
2014
Address:
Reykjavik, Iceland
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
2696–2700
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/911_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Homa B. Hashemi and Rebecca Hwa. 2014. A Comparison of MT Errors and ESL Errors. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 2696–2700, Reykjavik, Iceland. European Language Resources Association (ELRA).
Cite (Informal):
A Comparison of MT Errors and ESL Errors (Hashemi & Hwa, LREC 2014)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/911_Paper.pdf
PDF Search