Sabancigroup4 at SemEval-2026 Task 5: Uncertainty-Aware Semantic Plausibility Scoring via GNLL Regression and LLM Rationales

Salih Büyükbaş, Doruk Benli, Osman Kara, Dilara Keküllüoğlu


Abstract
SemEval-2026 Task 5 is a shared task on rating the plausibility of an ambiguous homonym in a predetermined context. The dataset of this task consists of a precontext & sentence & ending combinations for each homonym, and the plausibility of the sample is manually rated by 5 annotators. The task of participating teams was to automatically predict the plausibility with respect to the mean rate given by the annotators. Unlike traditional models that rely on single-label selection, this task frames disambiguation as a probabilistic distribution over multiple plausible meanings. To this end, we propose an uncertainty-aware training strategy using GNLL regression, and semantic context enrichment through POS tags and LLM rationales. Our system exhibits competitive performance, achieving 90% accuracy within standard deviation and 81% Spearman correlation, and placing us in the ninth place in the leaderboard.
Anthology ID:
2026.semeval-1.144
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1056–1062
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.144/
DOI:
Bibkey:
Cite (ACL):
Salih Büyükbaş, Doruk Benli, Osman Kara, and Dilara Keküllüoğlu. 2026. Sabancigroup4 at SemEval-2026 Task 5: Uncertainty-Aware Semantic Plausibility Scoring via GNLL Regression and LLM Rationales. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 1056–1062, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
Sabancigroup4 at SemEval-2026 Task 5: Uncertainty-Aware Semantic Plausibility Scoring via GNLL Regression and LLM Rationales (Büyükbaş et al., SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.144.pdf