Abstract
This paper shows the applicability of language testing techniques to machine translation (MT) evaluation through one of a set of related experiments. One straightforward experiment is to use language testing exams and scoring on MT output with little or no adaptation. This paper describes one such experiment, the first in a set. After an initial test (Vanni and Reeder, 2000), we expanded the experiment to include multiple raters and a more detailed analysis of the surprising results. Namely that unlike with humans, MT systems perform more poorly at both level zero and one than at level two and three. This paper presents these results as an illustration of both the applicability of language testing techniques and also the caution that needs to be applied.- Anthology ID:
- 2006.amta-papers.19
- Volume:
- Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers
- Month:
- August 8-12
- Year:
- 2006
- Address:
- Cambridge, Massachusetts, USA
- Venue:
- AMTA
- SIG:
- Publisher:
- Association for Machine Translation in the Americas
- Note:
- Pages:
- 166–175
- Language:
- URL:
- https://aclanthology.org/2006.amta-papers.19
- DOI:
- Cite (ACL):
- Florence Reeder. 2006. Direct Application of a Language Learner Test to MT Evaluation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, pages 166–175, Cambridge, Massachusetts, USA. Association for Machine Translation in the Americas.
- Cite (Informal):
- Direct Application of a Language Learner Test to MT Evaluation (Reeder, AMTA 2006)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/2006.amta-papers.19.pdf