@inproceedings{zhao-etal-2025-mutis,
    title = "{M}u{TIS}: Enhancing Reasoning Efficiency through Multi Turn Intervention Sampling in Reinforcement Learning",
    author = "Zhao, Wenshuo  and
      Zhai, Haoxing  and
      Qiu, Xinyu  and
      Qi, Zhenting  and
      Li, Shuhe  and
      Zhu, Linchao",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.690/",
    pages = "13680--13692",
    ISBN = "979-8-89176-332-6",
    abstract = "Recently, large reasoning models (LRMs) have demonstrated state-of-the-art performance across a wide range of benchmarks. However, a common challenge for these models is the ``overthinking'' problem, which leads to excessive reasoning steps and significant computational overhead. Furthermore, the issues with long Chain-of-Thought (CoT) are especially pronounced in smaller models ($\leq$ 3B parameters). Aside from producing excessively verbose ``reflection words'', they often exhibit repetition and get trapped in unproductive generation loops. Existing solutions typically involve either using flexible reasoning chains as training data or leveraging the model{'}s latent space to bypass intermediate reasoning steps, but none of these methods have considered directly optimizing reasoning trajectories during the sampling phase of training. In our work, we introduce the Multi-Turn Intervention Sampling Framework (MuTIS). Our framework leverages multi-turn interventions to produce concise reasoning chains. It fine-tunes reasoning models through reinforcement learning, demonstrably breaking the accuracy-efficiency trade-off. It also demonstrates strong scalability, exhibiting excellent performance on 7B models. Code is available at https://github.com/Edric-Zhao/MuTIS/tree/main."
}