@inproceedings{li-etal-2026-cotjudger,
    title = "{C}o{TJ}udger: A Graph-Driven Framework for Automatic Evaluation of Chain-of-Thought Efficiency and Redundancy in {LRM}s",
    author = "Li, Siyi  and
      Shi, Jiajun  and
      Ni, Shiwen  and
      Zhang, Ge  and
      Li, Shuaimin  and
      Wang, Shijian  and
      Wen, Zhoufutu  and
      LI, Yizhi  and
      Alinejad-Rokny, Hamid  and
      Liu, Jiaheng  and
      Yang, Min  and
      Huang, Wenhao",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.2077/",
    pages = "41837--41863",
    ISBN = "979-8-89176-395-1",
    abstract = "Large Reasoning Models (LRMs) have demonstrated strong performance by producing extended Chain-of-Thought (CoT) traces before answering. However, this paradigm often induces over-reasoning: redundant calculations and circular self-verification that increase computational cost without improving outcomes. Existing evaluations largely emphasize final accuracy or coarse token counts, and lack automated tools to separate essential logic from structural redundancy. We introduce CoTJudger, a graph-driven framework that quantifies reasoning efficiency by converting free-form CoTs into directed dependency graphs and extracting the Shortest Effective Path (SEP) needed to reach a correct solution. This yields an interpretable efficiency signal {--} how much of a CoT is necessary versus structurally redundant {--} that is comparable across models and tasks. Evaluating 21 LRMs, CoTJudger reveals pervasive redundancy and surfaces recurring failure modes, including verification obsession and compensatory redundancy. These results provide a practical metric for disentangling reasoning ability from computational waste, enabling more targeted evaluation and diagnosis of LRM efficiency."
}