@inproceedings{vazquez-etal-2025-semeval,
    title = "{S}em{E}val-2025 Task 3: Mu-{SHROOM}, the Multilingual Shared-task on Hallucinations and Related Observable Overgeneration Mistakes",
    author = {Vazquez, Raul  and
      Mickus, Timothee  and
      Zosa, Elaine  and
      Vahtola, Teemu  and
      Tiedemann, J{\"o}rg  and
      Sinha, Aman  and
      Segonne, Vincent  and
      Sanchez - Vega, Fernando  and
      Raganato, Alessandro  and
      Libovick{\'y}, Jind{\v{r}}ich  and
      Karlgren, Jussi  and
      Ji, Shaoxiong  and
      Helcl, Jind{\v{r}}ich  and
      Guillou, Liane  and
      De Gibert, Ona  and
      Bengoetxea, Jaione  and
      Attieh, Joseph  and
      Apidianaki, Marianna},
    editor = "Rosenthal, Sara  and
      Ros{\'a}, Aiala  and
      Ghosh, Debanjan  and
      Zampieri, Marcos",
    booktitle = "Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/transition-to-people-yaml/2025.semeval-1.322/",
    pages = "2472--2497",
    ISBN = "979-8-89176-273-2",
    abstract = "We present the Mu-SHROOM shared task which is focused on detecting hallucinations and other overgeneration mistakes in the output of instruction-tuned large language models (LLMs).Mu-SHROOM addresses general-purpose LLMs in 14 languages, and frames the hallucination detection problem as a span-labeling task. We received 2,618 submissions from 43 participating teams employing diverse methodologies. The very high number of submissions highlights the interest of the community in hallucination detection. We present the results of the participating systems and provide an empirical analysis in order to better understand the factors that can lead to strong performance in this task. We also underscore current challenges, notably the varying degree of hallucinations across languages and the high annotator disagreement when labeling hallucination spans."
}