Logic Matters in Lightweight Hallucination Classification for RAG System

Ningyuan Yang, Kaizhu Huang


Abstract
We propose a lightweight, modular framework for hallucination detection in Retrieval-Augmented Generation (RAG) systems, addressing the critical challenge where logical dependencies span across fragmented retrieval results. To address the inherent limitations of compact models in processing long-context information and performing multi-hop reasoning, our approach systematically analyzes the logical relationships among retrieved documents within the vector space. By capturing these geometric patterns through a novel feature extraction framework, the proposed classifier significantly enhances context-aware hallucination detection without requiring complex architectures or pre-training on datasets. Meanwhile, to evaluate multi-document reasoning, we release HotPotQA-derived, a hallucination dataset preserving separate retrieved texts. Experimental results on HotPotQA-derived and several open-source datasets demonstrate that our framework can achieve results comparable to or even surpassing those of large language models (LLMs) on the task of hallucination detection.
Anthology ID:
2026.acl-long.73
Volume:
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1605–1617
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.73/
DOI:
Bibkey:
Cite (ACL):
Ningyuan Yang and Kaizhu Huang. 2026. Logic Matters in Lightweight Hallucination Classification for RAG System. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1605–1617, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Logic Matters in Lightweight Hallucination Classification for RAG System (Yang & Huang, ACL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.73.pdf
Checklist:
 2026.acl-long.73.checklist.pdf