@inproceedings{zhi-etal-2026-spard,
    title = "{SPARD}: Self-Paced Curriculum for {RL} Alignment via Integrating Reward Dynamics and Data Utility",
    author = "Zhi, Xuyang  and
      Zhou, Peilun  and
      Lu, Chengqiang  and
      Lv, Hang  and
      Liang, Yiwei  and
      Zhang, Rongyang  and
      Gao, Yan  and
      Yiwu  and
      Hu, Yao  and
      Gu, Hongchao  and
      Lian, Defu  and
      Wang, Hao  and
      Chen, Enhong",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.2191/",
    pages = "47402--47422",
    ISBN = "979-8-89176-390-6",
    abstract = "The evolution of Large Language Models (LLMs) is shifting the focus from single, verifiable tasks toward complex, open-ended real-world scenarios, imposing significant challenges on the post-training phase. In these settings, the scale and complexity of reward systems have grown significantly, transitioning toward multi-objective formulations that encompass a comprehensive spectrum of model capabilities and application contexts. However, traditional methods typically rely on fixed reward weights, ignoring non-stationary learning dynamics and struggling with data heterogeneity across dimensions. To address these issues, we propose SPARD, a framework that establishes an automated, self-paced curriculum by perceiving learning progress to dynamically adjust multi-objective reward weights and data importance, thereby synchronizing learning intent with data utility for optimal performance. Extensive experiments across multiple benchmarks demonstrate that SPARD significantly enhances model capabilities across all domains. Our code is publicly available at https://github.com/USTC-StarTeam/SPARD."
}