Abstract
We hosted a shared task to investigate the factors influencing the quality of the code- mixed text generation systems. The teams experimented with two systems that gener- ate synthetic code-mixed Hinglish sentences. They also experimented with human ratings that evaluate the generation quality of the two systems. The first-of-its-kind, proposed sub- tasks, (i) quality rating prediction and (ii) an- notators’ disagreement prediction of the syn- thetic Hinglish dataset made the shared task quite popular among the multilingual research community. A total of 46 participants com- prising 23 teams from 18 institutions reg- istered for this shared task. The detailed description of the task and the leaderboard is available at https://codalab.lisn.upsaclay.fr/competitions/1688.- Anthology ID:
- 2022.inlg-genchal.3
- Volume:
- Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges
- Month:
- July
- Year:
- 2022
- Address:
- Waterville, Maine, USA and virtual meeting
- Editors:
- Samira Shaikh, Thiago Ferreira, Amanda Stent
- Venue:
- INLG
- SIG:
- SIGGEN
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 19–25
- Language:
- URL:
- https://aclanthology.org/2022.inlg-genchal.3
- DOI:
- Cite (ACL):
- Vivek Srivastava and Mayank Singh. 2022. HinglishEval Generation Challenge on Quality Estimation of Synthetic Code-Mixed Text: Overview and Results. In Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges, pages 19–25, Waterville, Maine, USA and virtual meeting. Association for Computational Linguistics.
- Cite (Informal):
- HinglishEval Generation Challenge on Quality Estimation of Synthetic Code-Mixed Text: Overview and Results (Srivastava & Singh, INLG 2022)
- PDF:
- https://preview.aclanthology.org/improve-issue-templates/2022.inlg-genchal.3.pdf