Abstract
In this system paper, we describe our submission for the 11th task of SemEval2023: Learning with Disagreements, or Le-Wi-Di for short. In the task, the assumption that there is a single gold label in NLP tasks such as hate speech or misogyny detection is challenged, and instead the opinions of multiple annotators are considered. The goal is instead to capture the agreements/disagreements of the annotators. For our system, we utilize the capabilities of modern large-language models as our backbone and investigate various techniques built on top, such as ensemble learning, multi-task learning, or Gaussian processes. Our final submission shows promising results and we achieve an upper-half finish.- Anthology ID:
- 2023.semeval-1.141
- Volume:
- Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Atul Kr. Ojha, A. Seza Doğruöz, Giovanni Da San Martino, Harish Tayyar Madabushi, Ritesh Kumar, Elisa Sartori
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1030–1036
- Language:
- URL:
- https://aclanthology.org/2023.semeval-1.141
- DOI:
- 10.18653/v1/2023.semeval-1.141
- Cite (ACL):
- Dennis Grötzinger, Simon Heuschkel, and Matthias Drews. 2023. CICL_DMS at SemEval-2023 Task 11: Learning With Disagreements (Le-Wi-Di). In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1030–1036, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- CICL_DMS at SemEval-2023 Task 11: Learning With Disagreements (Le-Wi-Di) (Grötzinger et al., SemEval 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-3/2023.semeval-1.141.pdf