@inproceedings{ren-etal-2026-knowrl,
    title = "{K}now{RL}: Exploring Knowledgeable Reinforcement Learning for Factuality",
    author = "Ren, Baochang  and
      Qiao, Shuofei  and
      Zhang, Ningyu  and
      Zheng, Da  and
      Chen, Huajun",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1840/",
    pages = "39640--39658",
    ISBN = "979-8-89176-390-6",
    abstract = "Slow-thinking Large Language Models (LLMs) have demonstrated strong reasoning capabilities but often suffer from severe hallucinations due to an inability to recognize their knowledge boundaries. Existing Reinforcement Learning (RL) approaches typically rely on outcome-oriented rewards, which can inadvertently reinforce fabricated reasoning paths when the final answer is correct. To address this, we propose **Know**ledge-enhanced **RL**, **KnowRL**, a framework that integrates factual supervision directly into the reasoning process. By decomposing the chain of thought into atomic facts and verifying them against the corresponding ground-truth knowledge, KnowRL performs fine-grained checks to encourage models to reason faithfully. Crucially, this process-oriented supervision teaches the model to identify its knowledge boundaries, learning to say ``I don{'}t know'' instead of fabricating answers when information is missing. Experimental results demonstrate that KnowRL effectively mitigates hallucinations{---}reducing the Incorrect Rate on SimpleQA by 20.3{\%} for distillation-based slow-thinking models while maintaining strong performance on complex reasoning benchmarks like GPQA and AIME 2025. Furthermore, our method shows robust transferability to out-of-distribution tasks, indicating that the model learns a generalizable verification behavior."
}