How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation

Hao Yang, Qinghua Zhao, Lei Li, Lingyi Meng, Mengda Yu


Abstract
Chain-of-Thought (CoT) prompting significantly enhances model reasoning, yet its internal mechanisms remain poorly understood. We analyze CoT’s operational principles by reversely tracing information flow across decoding, projection, and activation phases. Our quantitative analysis suggests that CoT may serve as a decoding space pruner, leveraging answer templates to guide output generation, with higher template adherence strongly correlating with improved performance. Furthermore, we surprisingly find that CoT modulates neuron engagement in a task-dependent manner: reducing neuron activation in open-domain tasks, yet increasing it in closed-domain scenarios. These findings offer a novel mechanistic interpretability framework and critical insights for the NLP community, enabling targeted CoT interventions to design more efficient and robust prompts. We released our code and data at https://github.com/How-Young-X/cot
Anthology ID:
2026.findings-acl.255
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5166–5199
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.255/
DOI:
Bibkey:
Cite (ACL):
Hao Yang, Qinghua Zhao, Lei Li, Lingyi Meng, and Mengda Yu. 2026. How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation. In Findings of the Association for Computational Linguistics: ACL 2026, pages 5166–5199, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation (Yang et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.255.pdf
Checklist:
 2026.findings-acl.255.checklist.pdf