Abstract
This paper describes the system description for the HinglishEval challenge at INLG 2022. The goal of this task was to investigate the factors influencing the quality of the code- mixed text generation system. The task was divided into two subtasks, quality rating pre- diction and annotators’ disagreement predic- tion of the synthetic Hinglish dataset. We at- tempted to solve these tasks using sentence- level embeddings, which are obtained from mean pooling the contextualized word embed- dings for all input tokens in our text. We experimented with various classifiers on top of the embeddings produced for respective tasks. Our best-performing system ranked 1st on subtask B and 3rd on subtask A. We make our code available here: https://github. com/nikhilbyte/Hinglish-qEval- Anthology ID:
- 2022.inlg-genchal.5
- Volume:
- Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges
- Month:
- July
- Year:
- 2022
- Address:
- Waterville, Maine, USA and virtual meeting
- Venue:
- INLG
- SIG:
- SIGGEN
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 31–34
- Language:
- URL:
- https://aclanthology.org/2022.inlg-genchal.5
- DOI:
- Cite (ACL):
- Nikhil Singh. 2022. niksss at HinglishEval: Language-agnostic BERT-based Contextual Embeddings with Catboost for Quality Evaluation of the Low-Resource Synthetically Generated Code-Mixed Hinglish Text. In Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges, pages 31–34, Waterville, Maine, USA and virtual meeting. Association for Computational Linguistics.
- Cite (Informal):
- niksss at HinglishEval: Language-agnostic BERT-based Contextual Embeddings with Catboost for Quality Evaluation of the Low-Resource Synthetically Generated Code-Mixed Hinglish Text (Singh, INLG 2022)
- PDF:
- https://preview.aclanthology.org/paclic-22-ingestion/2022.inlg-genchal.5.pdf