Elamur Research⌕
☰
home / papers / 2025 / Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?

Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?

Paper · 2025 · Samuel Lewis-Lim, Xingwei Tan, Zhixue Zhao et al.

Summary

Chain-of-Thought (CoT) prompting often gives little benefit on soft-reasoning tasks, and the written reasoning may not reflect what actually drives a model's answer. The authors ask whether, in these tasks, CoT actually shapes predictions or only justifies an answer the model has already chosen. They compare three model groups on multiple-choice benchmarks (CSQA, StrategyQA, MUSR, LSAT and GPQA):

  • instruction-tuned models (Qwen2.5-7B/32B-Instruct, Llama-8B-Instruct);
  • RL-trained reasoning models (Qwen3-32B, QwQ-32B);
  • distilled reasoning models (R1-Distill-Qwen-7B/32B, R1-Distill-Llama-8B).

Influence is measured with confidence trajectories: the probability of the eventual answer is recomputed after each reasoning step. Faithfulness is tested by inserting misleading hints into the prompt. GPT4.1 then judges whether a CoT that follows a hint admits to using it.

Distilled models depend heavily on their reasoning and often revise answers, while the other groups mostly keep their initial choice. Influence and faithfulness turn out to be separable. This weakens definitions of faithfulness that rely only on causal dependence.

Key points

  • Distilled models change their pre-CoT answer in about 65% of cases on average. The rates are 25% for instruction-tuned models and 24% for reasoning models. The distilled models often correct initial mistakes.
  • Instruction-tuned models mostly show flat confidence trajectories, which suggests post-hoc rationalisation, except on the harder GPQA. Distilled models show rising confidence, often with a sharp jump at the final step. Reasoning models are mixed, but when they do switch answers, they more often fix wrong ones.
  • Two cues are used: a sycophantic "Stanford Professor" suggestion and an XML metadata <answer> field. Verbalisation counts only when the CoT explicitly names the cue as its reason. Merely mentioning the cue does not count.
  • An unfaithful CoT can still steer the model's confidence. Conversely, a CoT can openly acknowledge a cue without that cue determining the final answer.
  • Published at EMNLP 2025 (University of Sheffield). Code is at github.com/samlewislim/cot-dynamics.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of SheffieldSheffield · United Kingdom113 works · 21,542 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Samuel Lewis-Lim, Xingwei Tan, Zhixue Zhao and Nikolaos Aletras. 2025. Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.1516

@inproceedings{lewislim2025analysing,
  title = {{Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?}},
  author = {Samuel Lewis-Lim, Xingwei Tan, Zhixue Zhao and Nikolaos Aletras},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.1516},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.1516}
}

References

35 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AI@Meta. 2024. Llama 3 model card. find
  2. Iván Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda, and Arthur Conmy. 2025. Chain-of-thought reasoning in the wild is not always faithful. Preprint, arXiv:2503.08679. find
  3. Guangsheng Bao, Hongbo Zhang, Cunxiang Wang, Linyi Yang, and Yue Zhang. 2025. How likely do LLMs with CoT mimic human reasoning? In Proceedings of the 31st International Conference on Computational Linguistics, pages 7831–7850, Abu Dhabi, UAE. Association for Computational Linguistics.
  4. Oliver Bentham, Nathan Stringham, and Ana Marasovic. 2024. Chain-of-thought unfaithfulness as disguised accuracy. Transactions on Machine Learning Research. Reproducibility Certification. find
  5. Jason Chan, Robert J. Gaizauskas, and Zhixue Zhao. 2025. RULEBREAKERS: Challenging LLMs at the crossroads between formal logic and human-like reasoning. In Forty-second International Conference on Machine Learning. find
  6. Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, John Schulman, Arushi Somani, Carson Denison, Peter Hase, Misha Wagner, Fabien Roger, and Vlad Mikuli. 2025. Reasoning models don’t always say what they think. find
  7. James Chua and Owain Evans. 2025. Are deepseek r1 and other reasoning models more faithful? Preprint, arXiv:2501.08156. — [arXiv:2501.08156](https://arxiv.org/abs/2501.08156) find
  8. DeepSeek-AI. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. Preprint, arXiv:2501.12948.
  9. Pedro Ferreira, Wilker Aziz, and Ivan Titov. 2025. Truthful or fabricated? using causal attribution to mitigate reward hacking in explanations. Preprint, arXiv:2504.05294. — [arXiv:2504.05294](https://arxiv.org/abs/2504.05294) find
  10. Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346– 361. find
  11. Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Mudit Verma, Kaya Stechly, Siddhant Bhambri, Lucas Paul Saldyt, and Anil B Murthy. 2024. Position: LLMs can’t plan, but can help planning in LLM-modulo frameworks. In Forty-first International Conference on Machine Learning. find
  12. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  13. Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, et al. 2023. Measuring faithfulness in chain-of-thought reasoning. Preprint, arXiv:2307.13702.
  14. Zayne Rea Sprague, Xi Ye, Kaj Bostrom, Swarat Chaudhuri, and Greg Durrett. 2024. MuSR: Testing the limits of chain-of-thought with multistep soft reasoning. In The Twelfth International Conference on Learning Representations. find
  15. Ryan Liu, Jiayi Geng, Addison J. Wu, Ilia Sucholutsky, Tania Lombrozo, and Thomas L. Griffiths. 2024. Mind your step (by step): Chain-of-thought can reduce performance on tasks where thinking makes humans worse. Preprint, arXiv:2410.21333. — [arXiv:2410.21333](https://arxiv.org/abs/2410.21333) find
  16. Zayne Rea Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Durrett. 2025. To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning. In The Thirteenth International Conference on Learning Re…
  17. OpenAI. 2024. Introducing openai o1-preview. find
  18. OpenAI. 2025a. Introducing gpt-4.1 in the api \| openai. find
  19. OpenAI. 2025b. Introducing openai o3 and o4-mini. find
  20. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Tra… — Training language models to follow instructions with human feedback
  21. Debjit Paul, Robert West, Antoine Bosselut, and Boi Faltings. 2024. Making reasoning matter: Measuring and improving faithfulness of chain-of-thought reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15012– 15032, Miami, Florida, USA. Association for Computat…
  22. Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, et al. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  23. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2024. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling.
  24. Laura Ruis, Maximilian Mozes, Juhan Bae, Siddhartha Rao Kamalakara, Dwaraknath Gnaneshwar, Acyr Locatelli, Robert Kirk, Tim Rocktäschel, Edward Grefenstette, and Max Bartolo. 2025. Procedural knowledge in pretraining drives reasoning in large language models. In The Thirteenth International Conferen… find
  25. Noah Siegel, Oana-Maria Camburu, Nicolas Heess, and Maria Perez-Ortiz. 2024. The probabilities also matter: A more faithful metric for faithfulness of free-text explanations in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2… find
  26. Alon Talmor, Jonathan Herzig, and Jonathan Berant. 2019. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long an…
  27. Qwen Team. 2025. QwQ-32B: Embracing the power of reinforcement learning. find
  28. Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. 2023. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. In Thirty-seventh Conference on Neural Information Processing Systems.
  29. Martin Tutek, Fateme Hashemi Chaleshtori, Ana Marasović, and Yonatan Belinkov. 2025. Measuring faithfulness of chains of thought by unlearning reasoning steps. Preprint, arXiv:2502.14829. — [arXiv:2502.14829](https://arxiv.org/abs/2502.14829) find
  30. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. 2024. MMLU-pro: A more robust and challenging multi-task language understanding benchmark…
  31. Zezhong Wang, Xingshan Zeng, Weiwen Liu, Yufei Wang, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu, and Kam-Fai Wong. 2025. Chain-of-probe: Examining the necessity and accuracy of CoT step-by-step. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 2586–260… find
  32. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, R…
  33. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lho… — Transformers: State-of-the-Art Natural Language Processing
  34. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  35. John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. Swe-agent: Agent-computer interfaces enable automated software engineering. Preprint, arXiv:2405.15793. *Figure 5: Influence Distribution for all instruction-tuned models* Wanjun Zhon…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lim-2025-analysing-chain-thought-dynamics-active/