On the origin of errors: A fine-grained analysis of MT and PE errors and their relationship

Joke Daems, Lieve Macken, Sonia Vandepitte

Abstract
In order to improve the symbiosis between machine translation (MT) system and post-editor, it is not enough to know that the output of one system is better than the output of another system. A fine-grained error analysis is needed to provide information on the type and location of errors occurring in MT and the corresponding errors occurring after post-editing (PE). This article reports on a fine-grained translation quality assessment approach which was applied to machine translated-texts and the post-edited versions of these texts, made by student post-editors. By linking each error to the corresponding source text-passage, it is possible to identify passages that were problematic in MT, but not after PE, or passages that were problematic even after PE. This method provides rich data on the origin and impact of errors, which can be used to improve post-editor training as well as machine translation systems. We present the results of a pilot experiment on the post-editing of newspaper articles and highlight the advantages of our approach.
Anthology ID:
L14-1441
Volume:
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
Month:
May
Year:
2014
Address:
Reykjavik, Iceland
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
62–66
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/532_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Joke Daems, Lieve Macken, and Sonia Vandepitte. 2014. On the origin of errors: A fine-grained analysis of MT and PE errors and their relationship. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 62–66, Reykjavik, Iceland. European Language Resources Association (ELRA).
Cite (Informal):
On the origin of errors: A fine-grained analysis of MT and PE errors and their relationship (Daems et al., LREC 2014)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2014/pdf/532_Paper.pdf
PDF Search