@inproceedings{wu-etal-2026-templaterl,
    title = "{T}emplate{RL}: Structured Template-Guided Reinforcement Learning for {LLM} Reasoning",
    author = "Wu, Jinyang  and
      Liao, Chonghua  and
      Feng, Mingkuan  and
      Zhang, Shuai  and
      Wen, Zhengqi  and
      Luo, Haoran  and
      Yang, Ling  and
      Xu, Huazhe  and
      Tao, Jianhua",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.198/",
    pages = "4050--4080",
    ISBN = "979-8-89176-395-1",
    abstract = "Reinforcement learning (RL) has emerged as an effective paradigm for enhancing model reasoning. However, existing RL methods like GRPO often rely on unstructured self-sampling to fit scalar rewards, often producing inefficient rollouts that fail to capture transferable problem-solving strategies. To address these limitations, we propose **TemplateRL**, a structured template-guided RL framework that augments policy optimization with explicit template guidance. Our approach first constructs a problem-solving template library via MCTS on a small seed set, then seamlessly integrates this high-level structured guidance into RL training. By guiding rollout generation to align with proven template structures, TemplateRL significantly improves high-quality trajectory hit rates while reducing ineffective exploration. This structure-guided design steers the policy toward validated strategic patterns, stabilizing training dynamics, and enhancing RL sampling efficiency. Notably, the explicit template library is interpretable, editable, and supports online updates-enabling continuous updates during both training and inference. Extensive experiments demonstrate that TemplateRL outperforms GRPO by 99{\%} on AIME and 41{\%} on AMC, with superior stability on weak models and remarkable cross-domain generalization, highlighting its potential for broader tasks."
}