@inproceedings{gao-etal-2025-towards,
    title = "Towards A Better Initial Policy Model For Scalable Long-{C}o{T} Reinforcement Learning",
    author = "Gao, Bofei  and
      Wang, Yejie  and
      Miao, Yibo  and
      Wu, Ruoyu  and
      Song, Feifan  and
      Yu, Longhui  and
      Liu, Tianyu  and
      Chang, Baobao",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/mtsummit-25-ingestion/2025.findings-acl.397/",
    doi = "10.18653/v1/2025.findings-acl.397",
    pages = "7652--7665",
    ISBN = "979-8-89176-256-5",
    abstract = "Long-CoT reasoning combined with reinforcement learning for large language models demonstrates remarkable performance and scalability. However, we observe that the initial policy model could significantly influence the final performance as well as the token efficiency. Additionally, there is a lack of systematic guidelines for obtaining a better initial policy model. To bridge this gap, we initiate a comprehensive investigation by activating the initial model using a variety of datasets with different data volumes and reasoning patterns. Then, we conduct a thorough analysis and comparison of the RL process for different initial models from the perspectives of upper bounds, diversity, and token efficiency, providing a deeper understanding and insight into the long-CoT RL. Based on our empirical results, we propose a systematic guideline and a novel Re-RFT method for constructing a better RL start point. Our experiment results based on the 14B model surpass the DeepSeek-R1-Distill-Qwen-14B by an average of 4.6{\%}, demonstrating our approach{'}s effectiveness and superiority."
}