@inproceedings{dumitru-etal-2025-conciserl,
    title = "{C}oncise{RL}: Conciseness-Guided Reinforcement Learning for Efficient Reasoning Models",
    author = "Dumitru, Razvan-Gabriel  and
      Peteleaza, Darius  and
      Yadav, Vikas  and
      Pan, Liangming",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.927/",
    doi = "10.18653/v1/2025.findings-emnlp.927",
    pages = "17099--17123",
    ISBN = "979-8-89176-335-7",
    abstract = "Large language models excel at complex tasks by breaking down problems into structured reasoning steps. However, reasoning traces often extend beyond reaching a correct answer, causing wasted computation, reduced readability, and hallucinations. To address this, we introduce a novel hyperparameter-free conciseness score used as a reward signal within a reinforcement learning framework to guide models toward generating correct and concise reasoning traces. This score is evaluated by a large language model acting as a judge, enabling dynamic, context-aware feedback beyond simple token length. Our method achieves state-of-the-art efficiency{--}accuracy trade-offs on the MATH dataset, reducing token usage by up to 31x on simple problems while improving accuracy by 7{\%}, and on the hardest problems, it outperforms full reasoning by +7.5{\%} accuracy with up to 3.6x fewer tokens. On TheoremQA, our method improves accuracy by +2.2{\%} using 12.5x fewer tokens. We also conduct ablation studies on the judge model, reward composition, and problem difficulty, showing that our method dynamically adapts reasoning length based on problem difficulty and benefits significantly from stronger judges. The code, model weights, and datasets are open-sourced at https://github.com/RazvanDu/ConciseRL."
}