@inproceedings{li-etal-2026-reinforcement,
    title = "Reinforcement Learning on Pre-Training Data",
    author = "Li, Siheng  and
      Li, Kejiao  and
      Xu, Zenan  and
      Huang, Guanhua  and
      Li, Kun  and
      Wu, Haoyuan  and
      Wujiajia  and
      Zheng, Zihao  and
      Zhang, Chenchen  and
      Shi, Kun  and
      Gong, Xue  and
      Yi, Qi  and
      Xiong, Ruibin  and
      Xu, Tingqiang  and
      Jiang, Yuhao  and
      Yan, Jianfeng  and
      Zeng, Yuyuan  and
      Xu, Guanghui  and
      Xue, Jinbao  and
      xu, Zhijiang  and
      Fang, Zheng  and
      LI, Shuai  and
      Liu, Qibin  and
      Li, Xiaoxue  and
      Li, Zhuoyu  and
      Tao, Yangyu  and
      Gao, Fei  and
      Jiang, Cheng  and
      Wang, Bochao  and
      Liu, Kai  and
      Zhu, Jianchen  and
      Lam, Wai  and
      Zhou, Bo  and
      Wang, Di",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.506/",
    pages = "11046--11057",
    ISBN = "979-8-89176-390-6",
    abstract = "Recent progress in large language models (LLMs) is largely driven by scaling training compute through either pre-training with next-token prediction (NTP) or post-training with reinforcement learning (RL). The former contributes to learning broad knowledge and skills from general data, while struggling with data inefficiency and catastrophic forgetting in continual learning settings. The latter incentivizes reasoning capabilities with strong generalization, but is constrained by limited data availability due to its reliance on human annotation. To alleviate these issues, we propose Reinforcement Learning on Pre-Training data (RLPT), which combines the advantages of learning from general data and RL. In particular, RLPT derives reward signals directly from general text data through a next-segment reasoning objective, rewarding the policy for correctly predicting next text segments conditioned on the prefix text. Experiments across multiple benchmarks and models demonstrate the effectiveness of . For example, RLPT yields substantial improvements in continual pre-training ($+4.6\%$) and provides a strong foundation for post-training ($+3.4\%$) on Qwen3-8B-Base."
}