Audrey Le

Also published as: Audrey N. Le


Document Image Collection Using Amazon’s Mechanical Turk
Audrey Le | Jerome Ajot | Mark Przybocki | Stephanie Strassel
Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk


Edit Distance: A Metric for Machine Translation Evaluation
Mark Przybocki | Gregory Sanders | Audrey Le
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)

NIST has coordinated machine translation (MT) evaluations for several years using an automatic and repeatable evaluation measure. Under the Global Autonomous Language Exploitation (GALE) program, NIST is tasked with implementing an edit-distance-based evaluation of MT. Here “edit distance” is defined to be the number of modifications a human editor is required to make to a system translation such that the resulting edited translation contains the complete meaning in easily understandable English, as a single high-quality human reference translation. In preparation for this change in evaluation paradigm, NIST conducted two proof-of-concept exercises specifically designed to probe the data space, to answer questions related to editor agreement, and to establish protocols for the formal GALE evaluations. We report here our experimental design, the data used, and our findings for these exercises.


NIST Language Technology Evaluation Cookbook
Alvin F. Martin | John S. Garofolo | Jonathan C. Fiscus | Audrey N. Le | David S. Pallett | Mark A. Przybocki | Gregory A. Sanders
Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)