@inproceedings{li-etal-2026-learnalign,
    title = "{L}earn{A}lign: Data Selection for {LLM} Reinforcement Learning with Improved Gradient Alignment",
    author = "Li, Shipeng  and
      Yang, Zhiqin  and
      Li, Shikun  and
      Xia, Xiaobo  and
      Liu, Hengyu  and
      Zhang, Xinghua  and
      Chen, Gaode  and
      Fang, Dong  and
      Tai, Ying  and
      Peng, Zhe",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.2009/",
    pages = "40417--40434",
    ISBN = "979-8-89176-395-1",
    abstract = "Reinforcement learning with verifiable rewards (RLVR) has become a key technique for enhancing LLMs' reasoning abilities, yet its data inefficiency remains a major bottleneck. To address this critical yet challenging issue, we present a novel gradient-alignment-based method, named LearnAlign, which intelligently selects the learnable and representative training reasoning data for RLVR post-training. To overcome the well-known response-length bias in gradient norms, we introduce the data learnability based on the success rate, which indicates the learning potential of each data point. Experiments across five reasoning benchmarks show that our method significantly reduces training data requirements while achieving minor performance degradation or even improving performance compared to full-data training. Specifically, it reduces data requirements by up to 1,000 data points with better performance (77.5{\%}) than that on the full dataset on the GSM8K benchmark (77.0{\%}). Furthermore, its efficiency is demonstrated on both mathematical and code benchmarks by using much less data from the DAPO-MATH-17K dataset."
}