Abstract
This paper gives a general review and detailed analysis of China Workshop on Machine Translation (CWMT) Evaluation. Compared with the past CWMT evaluation campaigns, CWMT2013 evaluation is characterized as follows: first, adopting gray-box evaluation which makes the results more replicable and controllable; second, adding one rule-based system as a counterpart; third, carrying out manual evaluations on some specific tasks to give a more comprehensive analysis of the translation errors. Boosted by those new features, our analysis and case study on the evaluation results shows the pros and cons of both rule-based and statistical systems, and reveals some interesting correlations bewteen automatic and manual evaluation metrics on different translation systems.- Anthology ID:
- 2014.amta-researchers.16
- Volume:
- Proceedings of the 11th Conference of the Association for Machine Translation in the Americas: MT Researchers Track
- Month:
- October 22-26
- Year:
- 2014
- Address:
- Vancouver, Canada
- Editors:
- Yaser Al-Onaizan, Michel Simard
- Venue:
- AMTA
- SIG:
- Publisher:
- Association for Machine Translation in the Americas
- Note:
- Pages:
- 208–221
- Language:
- URL:
- https://preview.aclanthology.org/build-pipeline-with-new-library/2014.amta-researchers.16/
- DOI:
- Cite (ACL):
- Sitong Yang, Heng Yu, Hongmei Zhao, Qun Liu, and Yajuan Lü. 2014. Review and analysis of China workshop on machine translation 2013 evaluation. In Proceedings of the 11th Conference of the Association for Machine Translation in the Americas: MT Researchers Track, pages 208–221, Vancouver, Canada. Association for Machine Translation in the Americas.
- Cite (Informal):
- Review and analysis of China workshop on machine translation 2013 evaluation (Yang et al., AMTA 2014)
- PDF:
- https://preview.aclanthology.org/build-pipeline-with-new-library/2014.amta-researchers.16.pdf