HIT-MI&T Lab’s Submission to Eval4NLP 2023 Shared Task
Rui Zhang, Fuhai Song, Hui Huang, Jinghao Yuan, Muyun Yang, Tiejun Zhao
Abstract
Recently, Large Language Models (LLMs) have boosted the research in natural language processing and shown impressive capabilities across numerous domains, including machine translation evaluation. This paper presents our methods developed for the machine translation evaluation sub-task of the Eval4NLP 2023 Shared Task. Based on the provided LLMs, we propose a generation-based method as well as a probability-based method to perform evaluation, explore different strategies when selecting the demonstrations for in-context learning, and try different ensemble methods to further improve the evaluation accuracy. The experiment results on the development set and test set demonstrate the effectiveness of our proposed method.- Anthology ID:
- 2023.eval4nlp-1.11
- Volume:
- Proceedings of the 4th Workshop on Evaluation and Comparison of NLP Systems
- Month:
- November
- Year:
- 2023
- Address:
- Bali, Indonesia
- Editors:
- Daniel Deutsch, Rotem Dror, Steffen Eger, Yang Gao, Christoph Leiter, Juri Opitz, Andreas Rücklé
- Venues:
- Eval4NLP | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 139–148
- Language:
- URL:
- https://aclanthology.org/2023.eval4nlp-1.11
- DOI:
- 10.18653/v1/2023.eval4nlp-1.11
- Cite (ACL):
- Rui Zhang, Fuhai Song, Hui Huang, Jinghao Yuan, Muyun Yang, and Tiejun Zhao. 2023. HIT-MI&T Lab’s Submission to Eval4NLP 2023 Shared Task. In Proceedings of the 4th Workshop on Evaluation and Comparison of NLP Systems, pages 139–148, Bali, Indonesia. Association for Computational Linguistics.
- Cite (Informal):
- HIT-MI&T Lab’s Submission to Eval4NLP 2023 Shared Task (Zhang et al., Eval4NLP-WS 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2023.eval4nlp-1.11.pdf