Abstract
Attempts to formulate methods of automatically evaluating machine translation (MT) have generally looked at some attrinbute of translation and then tried, explicitly or implicitly, to extrapolate the measurement to cover a broader class of attributes. In particular, some studies have focused on measuring fidelity of translation, and inferring intelligibility from that, and others have taken the opposite approach. In this paper we examine the more fundamental question of whether, and to what extent, the one attribute can be predicted by the other. As a starting point we use the 1994 DARPA MT corpus, which has measures for both attributes, and perform a simple comparison of the behavior of each. Two hypotheses about a predictable inference between fidelity and intelligibility are compared with the comparative behavior across all language pairs and all documents in the corpus.- Anthology ID:
- 2001.mtsummit-eval.10
- Volume:
- Workshop on MT Evaluation
- Month:
- September 18-22
- Year:
- 2001
- Address:
- Santiago de Compostela, Spain
- Venue:
- MTSummit
- SIG:
- Publisher:
- Note:
- Pages:
- Language:
- URL:
- https://aclanthology.org/2001.mtsummit-eval.10
- DOI:
- Cite (ACL):
- John White. 2001. Predicting intelligibility from fidelity in MT evaluation. In Workshop on MT Evaluation, Santiago de Compostela, Spain.
- Cite (Informal):
- Predicting intelligibility from fidelity in MT evaluation (White, MTSummit 2001)
- PDF:
- https://preview.aclanthology.org/nodalida-main-page/2001.mtsummit-eval.10.pdf