GIL-IIMAS UNAM at SemEval-2025 Task 3: MeSSI: A Multilmodule System to detect hallucinated Segments in trivia-like Inquiries.
Francisco Lopez - Ponce, Karla Salas - Jimenez, Adrián Juárez - Pérez, Diego Hernández - Bustamente, Gemma Bel - Enguix, Helena Gomez - Adorno
Abstract
We present MeSSI, a multi-module system applied to SemEval 2025’s task 3: Mu-SHROOM. Our system tags questions in order to obtain semantic relevant terms that are used as information retrieval characteristics. Said characteristics serve as extraction terms for Wikipedia pages that are in turn processed to generate gold standard texts used in a hallucination evaluation system. A PoST-based entity comparison was implemented to contrast the test dataset sentences with the corresponding generated gold standards, wich in turn was the main criteria to tag hallucinations, partitioned in soft labels and hard labels. This method was tested in Spanish and English, finishing 18th and 19th respectively on the IoU based ranking.- Anthology ID:
- 2025.semeval-1.208
- Volume:
- Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
- Month:
- July
- Year:
- 2025
- Address:
- Vienna, Austria
- Editors:
- Sara Rosenthal, Aiala Rosá, Debanjan Ghosh, Marcos Zampieri
- Venues:
- SemEval | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1577–1584
- Language:
- URL:
- https://preview.aclanthology.org/corrections-2025-08/2025.semeval-1.208/
- DOI:
- Cite (ACL):
- Francisco Lopez - Ponce, Karla Salas - Jimenez, Adrián Juárez - Pérez, Diego Hernández - Bustamente, Gemma Bel - Enguix, and Helena Gomez - Adorno. 2025. GIL-IIMAS UNAM at SemEval-2025 Task 3: MeSSI: A Multilmodule System to detect hallucinated Segments in trivia-like Inquiries.. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 1577–1584, Vienna, Austria. Association for Computational Linguistics.
- Cite (Informal):
- GIL-IIMAS UNAM at SemEval-2025 Task 3: MeSSI: A Multilmodule System to detect hallucinated Segments in trivia-like Inquiries. (Lopez - Ponce et al., SemEval 2025)
- PDF:
- https://preview.aclanthology.org/corrections-2025-08/2025.semeval-1.208.pdf