@inproceedings{wu-etal-2026-reflective,
    title = "Reflective {RAG}: Self-Evaluation Driven Strategy Optimization in Agentic Retrieval-Augmented Generation",
    author = "Wu, Haiyan  and
      Wang, Chenchen  and
      Sun, Chaoqun  and
      Lu, Chengxiong  and
      Chen, Yan-Hong  and
      Zhang, Zhiqiang  and
      Feng, Xiaoqing",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.648/",
    pages = "13252--13267",
    ISBN = "979-8-89176-395-1",
    abstract = "Retrieval-Augmented Generation (RAG) has emerged as a widely adopted paradigm for grounding Large Language Models (LLMs) in external knowledge. Recent agentic RAG systems introduce multi-turn reasoning, but they often lack the capacity to evaluate the utility of retrieved information, leading to brittle reasoning and suboptimal decision-making. We propose Reflective RAG, an agentic framework that incorporates self-evaluation to dynamically optimize retrieval and generation strategy. At its core, Reflective RAG employs a reflection tagging mechanism that allows the model to critique the relevance of retrieved content, thereby explicitly guiding its subsequent policy. To ensure robust learning, we introduce a two-stage training procedure that partially decouples evaluation semantics from strategy optimization. First, during supervised fine-tuning (SFT), the model learns to generate accurate reflection signals by self-correcting labels based on internal uncertainty. Second, a reinforcement learning (RL) stage optimizes the agent{'}s strategy using these reflections, stabilized by dynamic KL regularization. Evaluations across five knowledge-intensive QA benchmarks demonstrate that Reflective RAG consistently outperforms strong agentic baselines. Further analysis demonstrates its improved training stability and stronger generalization to complex multi-hop reasoning tasks."
}