@inproceedings{stack-sanchez-etal-2025-nlp,
    title = "{NLP}{\_}{CIMAT} at {S}em{E}val-2025 Task 3: Just Ask {GPT} or look Inside. A prompt and Neural Networks Approach to Hallucination Detection",
    author = "Stack - S{\'a}nchez, Jaime  and
      Alvarez - Carmona, Miguel  and
      Lopez Monroy, Adrian Pastor",
    editor = "Rosenthal, Sara  and
      Ros{\'a}, Aiala  and
      Ghosh, Debanjan  and
      Zampieri, Marcos",
    booktitle = "Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/transition-to-people-yaml/2025.semeval-1.221/",
    pages = "1683--1689",
    ISBN = "979-8-89176-273-2",
    abstract = "This paper presents NLP{\_}CIMAT{'}s participation in SemEval-2025 Task 3, which focuses on hallucination detection in large language models (LLMs) at character level across multiple languages. Hallucinations{---}outputs that are coherent and well-formed but contain inaccurate or fabricated information{---}pose significant challenges in real-world NLP applications. We explore two primary approaches: (1) a prompt-based method that leverages LLMs' own reasoning capabilities and knowledge, with and without external knowledge through a Retrieval-Augmented Generation (RAG)-like framework, and (2) a neural network approach that utilizes the hidden states of a LLM to predict hallucinated tokens. We analyze various factors in the neural approach, such as multilingual training, informing about the language, and hidden state selection. Our findings highlight that incorporating external information, like wikipedia articles, improves hallucination detection, particularly for smaller LLMs. Moreover, our best prompt-based technique secured second place in the Spanish category, demonstrating the effectiveness of in-context learning for this task."
}