Elamur Research⌕
☰
home / papers / 2026 / How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation

How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation

Paper · 2026 · Hao Yang, Qinghua Zhao, Lei Li et al.

Summary

Chain-of-Thought (CoT) prompting reliably improves LLM reasoning, but earlier explanations of why it works were mostly intuitive and not tied to model internals. This paper takes a mechanistic interpretability approach and traces information backward through three stages: the generated text (decoding), the output probability distributions (projection), and FFN neuron activity (activation). The authors compare vanilla 4-shot CoT with standard prompting on six pretrained models and nine arithmetic, commonsense and symbolic reasoning benchmarks.

Three findings stand out:

  • CoT seems to shrink the space of plausible outputs. Models copy the answer template shown in the exemplars, and closer adherence to that structure goes together with higher accuracy.
  • CoT makes next-token distributions more concentrated, which points to lower predictive uncertainty.
  • The effect on neurons depends on the task. Late-layer activation falls on open-domain tasks and rises on closed-domain ones.

The authors stress that this evidence is correlational rather than causal. Even so, it gives a framework for comparing prompting methods and for designing CoT interventions more deliberately.

Key points

  • Models: LLaMA3.1 (8B, 70B), LLaMA3.2-3B and Gemma2 (2B, 9B, 27B). Decoding was greedy with a 300-token cap, and accuracy was extracted with regular expressions.
  • Datasets:
    • Arithmetic: GSM8K, SVAMP, AQuA.
    • Commonsense: Bamboogle, StrategyQA, Date, Sports.
    • Symbolic: Coin Flip, Last Letters Concatenation.
  • Measuring imitation: "test points" are keywords in four categories (time, action, location & people, numbers). They were found via spaCy POS tagging plus manual checking. The analysis was complemented by cross-dataset prompt transfer experiments.
  • Neuron-level pattern: CoT acts like a "pruner" on open-domain tasks such as GSM8K and Bamboogle, where final-layer activation decreases. It acts like an "amplifier" on Coin Flip, AQuA, Sports and other closed-domain tasks, where final-layer activation increases.
  • Availability: code and data are released at github.com/How-Young-X/cot.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of PittsburghPittsburgh · United States59 works · 22,324 citations🎓Nanjing UniversityNanjing · China74 works · 7,814 citations🎓Beijing Institute of TechnologyBeijing · China34 works · 3,652 citations🎓Hefei UniversityHefei · China3 works · 1 citation

Also on the byline: The Ohio State University Wexner Medical Center.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Hao Yang, Qinghua Zhao, Lei Li, Lingyi Meng and Mengda Yu. 2026. How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.255

@inproceedings{yang2026how,
  title = {{How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation}},
  author = {Hao Yang, Qinghua Zhao, Lei Li, Lingyi Meng and Mengda Yu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.255},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.255}
}

References

44 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Meta AI. 2024. Llama 3.2: Connect 2024 — vision on the edge for mobile devices. Accessed: 2025-02-16. find
  2. BIG bench authors. 2023. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning Research.
  3. Zhen Bi, Ningyu Zhang, Yinuo Jiang, Shumin Deng, Guozhou Zheng, and Huajun Chen. 2024. When do program-of-thought works for reasoning? In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and… find
  4. Nicola Cancedda. 2024. Spectral filters, dark signals, and attention sinks. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4792– 4808, Bangkok, Thailand. Association for Computational Linguistics. find
  5. Ruizhe Chen, Tianxiang Hu, Yang Feng, and Zuozhu Liu. 2024a. Learnable privacy neurons localization in language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 256–264, Bangkok, Thailand. Association for Computational… find
  6. Sijia Chen, Baochun Li, and Di Niu. 2024b. Boosting of thoughts: Trial-and-error problem solving with large language models. In The Twelfth International Conference on Learning Representations. find
  7. Yew Ken Chia, Guizhen Chen, and 1 others. 2023. Contrastive chain-of-thought prompting. arXiv preprint arXiv:2311.09277.
  8. Karl Cobbe, Vineet Kosaraju, and 1 others. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
  9. Clément Dumas, Veniamin Veselovsky, Giovanni Monea, Robert West, and Chris Wendler. 2024. How do llamas process multilingual text? a latent exploration through activation patching. In ICML 2024 Workshop on Mechanistic Interpretability. find
  10. Subhabrata Dutta, Joykirat Singh, Soumen Chakrabarti, and Tanmoy Chakraborty. 2024. How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning. Preprint, arXiv:2402.18312. find
  11. Constanza Fierro, Negar Foroutan, Desmond Elliott, and Anders Søgaard. 2025. How do multilingual language models remember facts? Preprint, arXiv:2410.14387. find
  12. Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021a. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346– 361. find
  13. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021b. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495, Online and Punta Cana, Dominican Republic. Association for Computational… find
  14. Aaron Grattafiori and Abhimanyu Dubey etc. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
  15. Arnav Gudibande, Eric Wallace, and 1 others. 2024. The false promise of imitating proprietary language models. In The Twelfth International Conference on Learning Representations. find
  16. Wes Gurnee, Theo Horsley, and 1 others. 2024. Universal neurons in GPT2 language models. Transactions on Machine Learning Research. find
  17. Mingyu Jin, Qinkai Yu, and 1 others. 2024. The impact of reasoning step length on large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1830–1842, Bangkok, Thailand. Association for Computational Linguistics.
  18. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. Curran Associates Inc.
  19. Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K. Kummerfeld, and Rada Mihalcea. 2024. A mechanistic understanding of alignment algorithms: a case study on dpo and toxicity. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org. find
  20. Bo Li, Qinghua Zhao, and Lijie Wen. 2024. Rome: Memorization insights from text, logits and representation. arXiv preprint arXiv:2403.00510. — [arXiv:2403.00510](https://arxiv.org/abs/2403.00510) find
  21. Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Shishir G Patil, Matei Zaharia, Joseph E Gonzalez, and Ion Stoica. 2025. Llms can easily learn to reason from demonstrations structure, not content, is what matters! arXiv preprint arXiv:2502.07374. find
  22. Xiaonan Li and Xipeng Qiu. 2023. MoT: Memory-of-thought enables ChatGPT to self-improve. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6354– 6374, Singapore. Association for Computational Linguistics. find
  23. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 158–167, Vancouver… find
  24. Aman Madaan, Katherine Hermann, and Amir Yazdanbakhsh. 2023. What makes chain-of-thought prompting effective? a counterfactual study. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1448–1535, Singapore. Association for Computational Linguistics. find
  25. Aman Madaan and Amir Yazdanbakhsh. 2022. Text and patterns: For effective chain of thought, it takes two to tango. arXiv preprint arXiv:2209.07686. find
  26. Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. 2023. Progress measures for grokking via mechanistic interpretability. In The Eleventh International Conference on Learning Representations.
  27. Sania Nayab, Giulio Rossolini, and 1 others. 2025. Concise thoughts: Impact of output length on llm reasoning and cost. Preprint, arXiv:2407.19825. — [arXiv:2407.19825](https://arxiv.org/abs/2407.19825) find
  28. Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080–2094, Online. Associat…
  29. Akshara Prabhakar, Thomas L. Griffiths, and R. Thomas McCoy. 2024. Deciphering the factors influencing the efficacy of chain-of-thought: Probability, memorization, and noisy reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3710–3724, Miami, Florida, USA. Ass… find
  30. Ofir Press, Muru Zhang, and 1 others. 2023. Measuring and narrowing the compositionality gap in language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5687–5711, Singapore. Association for Computational Linguistics. find
  31. Daking Rai and Ziyu Yao. 2024. An investigation of neuron activation as a unified lens to explain chain-of-thought eliciting arithmetic reasoning of LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7174–7193, Bangkok, Tha… find
  32. Rylan Schaeffer, Kateryna Pistunova, and 1 others. 2023. Invalid logic, equivalent gains: The bizarreness of reasoning in language model prompting. In ICML 2023 Workshop on Knowledge and Logical Reasoning in the Era of Data-driven Learning. find
  33. Sree Harsha Tanneru, Dan Ley, Chirag Agarwal, and Himabindu Lakkaraju. 2024. On the hardness of faithful chain-of-thought reasoning in large language models. arXiv preprint arXiv:2406.10625. — [arXiv:2406.10625](https://arxiv.org/abs/2406.10625) find
  34. Gemma Team. 2024. Gemma 2: Improving open language models at a practical size. Preprint, arXiv:2408.00118. find
  35. Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. Curran Associates Inc. find
  36. Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. 2024. Do llamas work in English? on the latent language of multilingual transformers. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15366–15394, Bangkok, T…
  37. Skyler Wu, Eric Meng Shen, Charumathi Badrinath, Jiaqi Ma, and Himabindu Lakkaraju. 2023. Analyzing chain-of-thought prompting in large language models via gradient-based feature attributions. arXiv preprint arXiv:2307.13339.
  38. Xi Ye, Srinivasan Iyer, Asli Celikyilmaz, Veselin Stoyanov, Greg Durrett, and Ramakanth Pasunuru. 2023. Complementary explanations for effective in-context learning. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4469–4484, Toronto, Canada. Association for Computationa…
  39. Wei Jie Yeo, Ranjan Satapathy, and Erik Cambria. 2024. Towards faithful natural language explanations: A study using activation patching in large language models. Preprint, arXiv:2410.14155. find
  40. Biao Yi, Sishuo Chen, and 1 others. 2024. BadActs: A universal backdoor defense in the activation space. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5339–5352, Bangkok, Thailand. Association for Computational Linguistics. find
  41. Elena Voita, Javier Ferrando, and Christoforos Nalmpantis. 2024. Neurons in large language models: Dead, n-gram, positional. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1288–1301, Bangkok, Thailand. Association for Computational Linguistics. find
  42. Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. 2023. Towards understanding chain-of-thought prompting: An empirical study of what matters. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pag…
  43. Albert Webson and Ellie Pavlick. 2022. Do prompt-based models really understand the meaning of their prompts? In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2300–2344, Seattle, United States. As…
  44. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2026-how-chain-thought-works-tracing/