@inproceedings{cui-etal-2025-stepwise,
    title = "Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models",
    author = "Cui, Yingqian  and
      He, Pengfei  and
      Zeng, Jingying  and
      Liu, Hui  and
      Tang, Xianfeng  and
      Dai, Zhenwei  and
      Han, Yan  and
      Luo, Chen  and
      Huang, Jing  and
      Li, Zhen  and
      Wang, Suhang  and
      Xing, Yue  and
      Tang, Jiliang  and
      He, Qi",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/display_plenaries/2025.findings-acl.956/",
    pages = "18581--18597",
    ISBN = "979-8-89176-256-5",
    abstract = "Chain-of-Thought (CoT) reasoning, which breaks down complex tasks into intermediate reasoning steps, has significantly enhanced the performance of large language models (LLMs) on challenging tasks. However, the detailed reasoning process in CoT often incurs long generation times and high computational costs, partly due to the inclusion of unnecessary steps. To address this, we propose a method to identify critical reasoning steps using perplexity as a measure of their importance: a step is deemed critical if its removal causes a significant increase in perplexity. Our method enables models to focus solely on generating these critical steps. This can be achieved through two approaches: refining demonstration examples in few-shot CoT or fine-tuning the model using selected examples that include only critical steps. Comprehensive experiments validate the effectiveness of our method, which achieves a better balance between the reasoning accuracy and efficiency of CoT."
}