@inproceedings{zhang-etal-2025-lets,
    title = "{L}e{TS}: Learning to Think-and-Search via Process-and-Outcome Reward Hybridization",
    author = "Zhang, Qi  and
      Yang, Shouqing  and
      Gao, Lirong  and
      Chen, Hao  and
      Hu, Xiaomeng  and
      Chen, Jinglei  and
      Wang, Jiexiang  and
      Guo, Sheng  and
      Zheng, Bo  and
      Wang, Haobo  and
      Zhao, Junbo",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.257/",
    pages = "5109--5122",
    ISBN = "979-8-89176-332-6",
    abstract = "Large language models (LLMs) have demonstrated impressive capabilities in reasoning with the emergence of reasoning models like OpenAI-o1 and DeepSeek-R1. Recent research focuses on integrating reasoning capabilities into the realm of retrieval-augmented generation (RAG) via outcome-supervised reinforcement learning (RL) approaches, while the correctness of intermediate think-and-search steps is usually neglected. To address this issue, we design a process-level reward module to mitigate the unawareness of intermediate reasoning steps in outcome-level supervision without additional annotation. Grounded on this, we propose **Le**arning to **T**hink-and-**S**earch (**LeTS**), a novel framework that hybridizes stepwise process reward and outcome-based reward to current RL methods for RAG. Extensive experiments demonstrate the generalization and inference efficiency of **LeTS** across various RAG benchmarks. In addition, these results reveal the potential of process- and outcome-level reward hybridization in boosting LLMs' reasoning ability via RL under other scenarios."
}