@inproceedings{qi-etal-2026-carl,
    title = "{CARL}: Constraint-Aware Reinforcement Learning for Planning with {LLM}s",
    author = "Qi, Qiuyi  and
      Zhang, Jinjian  and
      Bao, Mutian  and
      Liang, Tian  and
      Li, Guocong  and
      Liu, Dongnan  and
      Zhou, Wei  and
      Liu, Jie  and
      Kong, Ming  and
      Mo, Linjian  and
      Zhang, Feng  and
      Zhu, Qiang",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1069/",
    pages = "21257--21281",
    ISBN = "979-8-89176-395-1",
    abstract = "Despite their strong reasoning capabilities and extensive world knowledge, Large Language Models (LLMs) frequently generate plans that violate task constraints, undermining their reliability in real-world applications. This deficiency arises from a lack of systematic mechanisms to incorporate constraint information during the generation process. While existing approaches attempt to mitigate this by relying on external tools or task decomposition, they fail to enhance the model{'}s intrinsic constraint awareness. To address this, we propose Constraint-Aware Reinforcement Learning (CARL), a novel RL framework designed to strengthen LLMs' intrinsic focus on constraints. CARL introduces a constraint-aware reward by comparing the model{'}s output distributions under constrained and unconstrained inputs, encouraging constraint focus and penalizing neglect.Compatible with various RL frameworks and requiring no external solvers or top models, CARL enables scalable, end-to-end constraint-aware planning. Extensive experiments on BlocksWorld, TravelPlanner, and T-Eval demonstrate that CARL significantly outperforms standard Reinforcement Fine-Tuning (RFT) baselines and state-of-the-art reasoning models, exhibiting a markedly increased focus on constraints."
}