Abstract
We present an automated, system-internal evaluation technique for linguistic representations in a large-scale, multilingual MT system. We use machine-learned classifiers to recognize the differences between linguistic representations generated from transfer in an MT context from representations that are produced by "native" analysis of the target language. In the MT scenario, convergence of the two is the desired result. Holding the feature set and the learning algorithm constant, the accuracy of the classifiers provides a measure of the overall difference between the two sets of linguistic representations: classifiers with higher accuracy correspond to more pronounced differences between representations. More importantly, the classifiers yield the basis for error-analysis by providing a ranking of the importance of linguistic features. The more salient a linguistic criterion is in discriminating transferred representations from "native" representations, the more work will be needed in order to get closer to the goal of producing native-like MT. We present results from using this approach on the Microsoft MT system and discuss its advantages and possible extensions.- Anthology ID:
- 2001.mtsummit-papers.21
- Volume:
- Proceedings of Machine Translation Summit VIII
- Month:
- September 18-22
- Year:
- 2001
- Address:
- Santiago de Compostela, Spain
- Editor:
- Bente Maegaard
- Venue:
- MTSummit
- SIG:
- Publisher:
- Note:
- Pages:
- Language:
- URL:
- https://aclanthology.org/2001.mtsummit-papers.21
- DOI:
- Cite (ACL):
- Michael Gamon, Hisami Suzuki, and Simon Corston-Oliver. 2001. Using machine learning for system-internal evaluation of transferred linguistic representations. In Proceedings of Machine Translation Summit VIII, Santiago de Compostela, Spain.
- Cite (Informal):
- Using machine learning for system-internal evaluation of transferred linguistic representations (Gamon et al., MTSummit 2001)
- PDF:
- https://preview.aclanthology.org/emnlp22-frontmatter/2001.mtsummit-papers.21.pdf