Enhancing Reliability in Community Question Answering with an Expert-Oriented RAG System

Seyyede Zahra Aftabi, Saeed Farzi


Abstract
In recent years, pre-trained large language models (LLMs) have become a cornerstone for automatically generating answers in question-and-answer (Q A) communities, significantly reducing user wait times and improving response quality. However, these models require substantial computational resources and are prone to generating hallucinated or unreliable content. To overcome these limitations, we propose an advanced expert-oriented Retrieval-Augmented Generation (RAG) framework as a cost-effective and reliable alternative. Central to our approach is a user-aware question entailment recognition module, which leverages user modeling to identify archived questions with answers that fully or partially address the user’s new query. This user modeling significantly improves retrieval relevance, resulting in reduced hallucination and enhanced answer quality. The framework synthesizes expert-written answers from similar questions to generate unified responses. Experimental results on the CQADupStack and SE-PQA datasets show the superiority of our user-aware approach over its user-agnostic counterpart, with ROUGE-1 gains of 3.6% and 0.9%. Both human and AI evaluations confirm the effectiveness of incorporating user modeling in minimizing hallucination and delivering contextually appropriate answers, demonstrating its potential for real-world Q A systems. The code and data are available on a GitHub repository at https://anonymous.4open.science/r/User-Oriented-RAG-CQA.
Anthology ID:
2026.findings-eacl.132
Volume:
Findings of the Association for Computational Linguistics: EACL 2026
Month:
March
Year:
2026
Address:
Rabat, Morocco
Editors:
Vera Demberg, Kentaro Inui, Lluís Marquez
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2551–2569
Language:
URL:
https://preview.aclanthology.org/ingest-eacl/2026.findings-eacl.132/
DOI:
Bibkey:
Cite (ACL):
Seyyede Zahra Aftabi and Saeed Farzi. 2026. Enhancing Reliability in Community Question Answering with an Expert-Oriented RAG System. In Findings of the Association for Computational Linguistics: EACL 2026, pages 2551–2569, Rabat, Morocco. Association for Computational Linguistics.
Cite (Informal):
Enhancing Reliability in Community Question Answering with an Expert-Oriented RAG System (Aftabi & Farzi, Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-eacl/2026.findings-eacl.132.pdf
Checklist:
 2026.findings-eacl.132.checklist.pdf