@inproceedings{chen-etal-2025-scale,
    title = "Scale Down to Speed Up: Dynamic Data Selection for Reinforcement Learning",
    author = "Chen, Zhuoyue  and
      Zhang, Jihai  and
      Liu, Ben  and
      Lin, Fangquan  and
      Yin, Wotao",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.412/",
    doi = "10.18653/v1/2025.findings-emnlp.412",
    pages = "7806--7817",
    ISBN = "979-8-89176-335-7",
    abstract = "Optimizing data utilization remains a central challenge in applying Reinforcement Learning (RL) to Large Language Models (LLMs), directly impacting sample efficiency, training stability, and final model performance.Current approaches often rely on massive static datasets, leading to computational inefficiency and redundant gradient updates.In this paper, we propose ScalingRL, a data-centric RL framework that dynamically selects the most informative training samples to optimize RL for mathematical reasoning.Specifically, ScalingRL introduces the Data Effectiveness Score (DES) that quantitatively ranks prompts according to three complementary factors: problem difficulty, Chain-of-Thought complexity, and reward adaptability.Then, ScalingRL employs an adaptive curriculum scheduler that progressively adjusts the overall scale and specific mix of training prompts{---}balancing exploration of new, challenging data with exploitation of previously learned concepts{---}thereby tailoring the data distribution to the model{'}s current learning trajectory and performance.Experimental results demonstrate that ScalingRL achieves comparable performance to full-data training methods while requiring only 1.5K samples instead of 220K, reducing training time from 13 days to just 4 hours on $8\times$A800 GPUs."
}