@inproceedings{staliunaite-vlachos-2025-uncertain,
    title = "Uncertain (Mis)Takes at {L}e{W}i{D}i-2025: Modeling Human Label Variation With Semantic Entropy",
    author = "Stali{\={u}}nait{\.{e}}, Ieva Raminta  and
      Vlachos, Andreas",
    editor = "Abercrombie, Gavin  and
      Basile, Valerio  and
      Frenda, Simona  and
      Tonelli, Sara  and
      Dudy, Shiran",
    booktitle = "Proceedings of the The 4th Workshop on Perspectivist Approaches to NLP",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.nlperspectives-1.23/",
    pages = "256--262",
    ISBN = "979-8-89176-350-0",
    abstract = "The VariErrNLI task requires detecting the degree to which each Natural Language Inference (NLI) label is acceptable to a group of annotators. This paper presents an approach to VariErrNLI which incorporates measures of uncertainty, namely Semantic Entropy (SE), to model human label variation. Our method is based on the assumption that if two labels are plausible alternatives, then their explanations must be non-contradictory. We measure SE over Large Language Model (LLM)-generated explanations for a given NLI label, which represents the model uncertainty over the semantic space of possible explanations for that label. The system employs SE scores combined with an encoding of the inputs and generated explanations, and reaches a 0.31 Manhattan distance score on the test set, ranking joint first in the soft evaluation of VariErrNLI."
}