Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?

Samuel Lewis-Lim, Xingwei Tan, Zhixue Zhao, Nikolaos Aletras


Abstract
Recent work has demonstrated that using chain of thought (CoT), on soft-reasoning problems such as analytical and commonsense reasoning, often yields limited or even negative performance gains. CoT can also be unfaithful to the model’s actual reasoning. This paper investigates dynamics and unfaithfulness of CoT in soft-reasoning tasks across instruction-tuned, reasoning and reasoning-distilled models. Our findings show that distilled‐reasoning models rely heavily on CoT for these tasks, while instruction‐tuned and reasoning models often use it post‐hoc. Additionally, we find that CoT can steer model predictions without faithfully reflecting reasoning, indicating a disconnect between CoT influence and faithfulness.
Anthology ID:
2025.emnlp-main.1516
Volume:
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
29826–29841
Language:
URL:
https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1516/
DOI:
Bibkey:
Cite (ACL):
Samuel Lewis-Lim, Xingwei Tan, Zhixue Zhao, and Nikolaos Aletras. 2025. Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 29826–29841, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation? (Lewis-Lim et al., EMNLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1516.pdf
Checklist:
 2025.emnlp-main.1516.checklist.pdf