Abstract
This paper addresses the challenging problem of automatically evaluating output from machine translation (MT) systems in order to support the developers of these systems. Conventional approaches to the problem include methods that automatically assign a rank such as A, B, C, or D to MT output according to a single edit distance between this output and a correct translation example. The single edit distance can be differently designed, but changing its design makes assigning a certain rank more accurate, but another rank less accurate. This inhibits improving accuracy of rank assignment. To overcome this obstacle, this paper proposes an automatic ranking method that, by using multiple edit distances, encodes machine-translated sentences with a rank assigned by humans into multi-dimensional vectors from which a classifier of ranks is learned in the form of a decision tree (DT). The proposed method assigns a rank to MT output through the learned DT. The proposed method is evaluated using transcribed texts of real conversations in the travel arrangement domain. Experimental results show that the proposed method is more accurate than the single-edit-distance-based ranking methods, in both closed and open tests. Moreover, the proposed method could estimate MT quality within 3% error in some cases.- Anthology ID:
- 2001.mtsummit-papers.3
- Volume:
- Proceedings of Machine Translation Summit VIII
- Month:
- September 18-22
- Year:
- 2001
- Address:
- Santiago de Compostela, Spain
- Editor:
- Bente Maegaard
- Venue:
- MTSummit
- SIG:
- Publisher:
- Note:
- Pages:
- Language:
- URL:
- https://aclanthology.org/2001.mtsummit-papers.3
- DOI:
- Cite (ACL):
- Yasuhiro Akiba, Kenji Imamura, and Eiichiro Sumita. 2001. Using multiple edit distances to automatically rank machine translation output. In Proceedings of Machine Translation Summit VIII, Santiago de Compostela, Spain.
- Cite (Informal):
- Using multiple edit distances to automatically rank machine translation output (Akiba et al., MTSummit 2001)
- PDF:
- https://preview.aclanthology.org/improve-issue-templates/2001.mtsummit-papers.3.pdf