Yanli Sun


Deploying MT into a Localisation Workflow: Pains and Gains
Yanli Sun | Juan Liu | Yi Li
Proceedings of Machine Translation Summit XIII: Papers


A Novel Statistical Pre-Processing Model for Rule-Based Machine Translation System
Yanli Sun | Sharon O’Brien | Minako O’Hagan | Fred Hollowood
Proceedings of the 14th Annual conference of the European Association for Machine Translation

Mining the Correlation between Human and Automatic Evaluation at Sentence Level
Yanli Sun
Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)

Automatic evaluation metrics are fast and cost-effective measurements of the quality of a Machine Translation (MT) system. However, as humans are the end-user of MT output, human judgement is the benchmark to assess the usefulness of automatic evaluation metrics. While most studies report the correlation between human evaluation and automatic evaluation at corpus level, our study examines their correlation at sentence level. In addition to the statistical correlation scores, such as Spearman's rank-order correlation coefficient, a finer-grained and detailed examination of the sensitivity of automatic metrics compared to human evaluation is also reported in this study. The results show that the threshold for human evaluators to agree with the judgements of automatic metrics varies with the automatic metrics at sentence level. While the automatic scores for two translations are greatly different, human evaluators may consider the translations to be qualitatively similar and vice versa. The detailed analysis of the correlation between automatic and human evaluation allows us determine with increased confidence whether an increase in the automatic scores will be agreed by human evaluators or not.


Improving Word Alignment Using Syntactic Dependencies
Yanjun Ma | Sylwia Ozdowska | Yanli Sun | Andy Way
Proceedings of the ACL-08: HLT Second Workshop on Syntax and Structure in Statistical Translation (SSST-2)