@inproceedings{ortega-riba-etal-2026-cuclasic,
    title = "{CUCLASIC} at {S}em{E}val-2026 Task 5: {LLM} Prompting Strategies for Rating Ambiguous Word Senses",
    author = "Ortega Riba, Federico  and
      Wilkerson, Jasper  and
      Lafreniere Adams, Kelsey",
    editor = "Kochmar, Ekaterina  and
      Ghosh, Debanjan  and
      North, Kai  and
      Komachi, Mamoru",
    booktitle = "Proceedings of the 20th {I}nternational {W}orkshop on {S}emantic {E}valuation (2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.121/",
    pages = "886--893",
    ISBN = "979-8-89176-414-9",
    abstract = "Word sense disambiguation has been a foundational task in computational semantics since the 1990s, but remains an unsolved problem when it comes to bridging human and computational evaluation of ambiguity. The SemEval-2026 Task 5 attempts to address this gap. We test six Large Language Models (LLMs) from the Llama and Gemini families in order to evaluate LLMs' ratings of ambiguous textual excerpts, experimenting with zero- and few-shot variants of prompts and analyzing how simple linguistic cues improve performance. We propose a methodology of eliciting human-like ratings from language models by using examples with low and high standard deviations between human ratings. We further evaluate and compare the prediction patterns of different models and how they align with the human generated ratings. Our best model (Gemini 3-Flash) achieves a 75{\%} score combining Spearman correlation and accuracy within one standard deviation."
}