@inproceedings{bala-etal-2025-hausanlp,
    title = "{H}ausa{NLP} at {S}em{E}val-2025 Task 3: Towards a Fine-Grained Model-Aware Hallucination Detection",
    author = "Bala, Maryam  and
      Abubakar, Amina  and
      Abubakar, Abdulhamid  and
      Bichi, Abdulkadir  and
      Ahmad, Hafsa  and
      Sani, Sani Abdullahi  and
      Abdulmumin, Idris  and
      Muhammad, Shamsuddeen Hassan  and
      Ahmad, Ibrahim Said",
    editor = "Rosenthal, Sara  and
      Ros{\'a}, Aiala  and
      Ghosh, Debanjan  and
      Zampieri, Marcos",
    booktitle = "Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/transition-to-people-yaml/2025.semeval-1.227/",
    pages = "1737--1741",
    ISBN = "979-8-89176-273-2",
    abstract = "This paper presents our findings of the Multilingual Shared Task on Hallucinations and Related Observable Overgeneration Mistakes, MU-SHROOM, which focuses on identifying hallucinations and related overgeneration errors in large language models (LLMs). The shared task involves detecting specific text spans that constitute hallucinations in the outputs generated by LLMs in 14 languages. To address this task, we aim to provide a nuanced, model-aware understanding of hallucination occurrences and severity in English. We used natural language inference and fine-tuned a ModernBERT model using a synthetic dataset of 400 samples, achieving an Intersection over Union (IoU) score of 0.032 and a correlation score of 0.422. These results indicate a moderately positive correlation between the model{'}s confidence scores and the actual presence of hallucinations. The IoU score indicates that our modelhas a relatively low overlap between the predicted hallucination span and the truth annotation. The performance is unsurprising, given the intricate nature of hallucination detection. Hallucinations often manifest subtly, relying on context, making pinpointing their exact boundaries formidable."
}