@inproceedings{wen-etal-2025-policy,
    title = "On-Policy Self-Alignment with Fine-grained Knowledge Feedback for Hallucination Mitigation",
    author = "Wen, Xueru  and
      Lou, Jie  and
      Lu, Xinyu  and
      Ji, Yuqiu  and
      Guan, Xinyan  and
      Lu, Yaojie  and
      Lin, Hongyu  and
      He, Ben  and
      Han, Xianpei  and
      Zhang, Debing  and
      Sun, Le",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/display_plenaries/2025.findings-acl.271/",
    pages = "5215--5231",
    ISBN = "979-8-89176-256-5",
    abstract = "Hallucination occurs when large language models exhibit behavior that deviates from the boundaries of their knowledge during response generation. To address this critical issue, previous learning-based methods attempt to finetune models but are limited by off-policy sampling and coarse-grained feedback. In this paper, we present \textit{Reinforcement Learning for Hallucination} (RLFH), an on-policy self-alignment approach that enables LLMs to actively explore their knowledge boundaries and self-correct generation behavior through fine-grained feedback signals. RLFH introduces a self-assessment framework where the policy serves as its own judge. Through this framework, responses are automatically decomposed into atomic facts and their truthfulness and informativeness are assessed against external knowledge sources. The resulting fine-grained feedback at the statement level are then converted into token-level dense reward signals. This enables online reinforcement learning to achieve precise and timely optimization without human intervention. Comprehensive evaluations on HotpotQA, SQuADv2, and Biography benchmarks validate RLFH{'}s effectiveness in hallucination mitigation."
}