A test suite for evaluation of English-to-Korean machine translation systems
Sungryong Koh, Jinee Maeng, Ji-Young Lee, Young-Sook Chae, Key-Sun Choi
Abstract
This paper describes KORTERM’s test suite and their practicability. The test-sets have been being constructed on the basis of fine-grained classification of linguistic phenomena to evaluate the technical status of English-to-Korean MT systems systematically. They consist of about 5000 test-sets and are growing. Each test-set contains an English sentence, a model Korean translation, a linguistic phenomenon category, and a yes/no question about the linguistic phenomenon. Two commercial systems were evaluated with a yes/no test of prepared questions. Total accuracy rates of the two systems were different (50% vs. 66%). In addition, a comprehension test was carried out. We found that one system was more comprehensible than the other system. These results seem to show that our test suite is practicable.- Anthology ID:
- 2001.mtsummit-papers.35
- Volume:
- Proceedings of Machine Translation Summit VIII
- Month:
- September 18-22
- Year:
- 2001
- Address:
- Santiago de Compostela, Spain
- Editor:
- Bente Maegaard
- Venue:
- MTSummit
- SIG:
- Publisher:
- Note:
- Pages:
- Language:
- URL:
- https://aclanthology.org/2001.mtsummit-papers.35
- DOI:
- Cite (ACL):
- Sungryong Koh, Jinee Maeng, Ji-Young Lee, Young-Sook Chae, and Key-Sun Choi. 2001. A test suite for evaluation of English-to-Korean machine translation systems. In Proceedings of Machine Translation Summit VIII, Santiago de Compostela, Spain.
- Cite (Informal):
- A test suite for evaluation of English-to-Korean machine translation systems (Koh et al., MTSummit 2001)
- PDF:
- https://preview.aclanthology.org/landing_page/2001.mtsummit-papers.35.pdf