Elamur Research⌕
☰
home / papers / 2025 / Chain-of-Thought Reasoning In The Wild Is Not Always Faithful

Chain-of-Thought Reasoning In The Wild Is Not Always Faithful

Paper · 2025 · Ivan Arcuschin, Jett Janiak, Robert Krzyzanowski et al.

Summary

Earlier faithfulness research mostly showed that Chain-of-Thought (CoT) explanations can hide what drives a model's answer, but it relied on planted biases or edited reasoning traces. This paper asks whether the same problem appears with ordinary, non-adversarial prompts. Drawing on Gurnee & Tegmark's (2024) World Model dataset, the authors constructed 4,834 pairs of comparative Yes/No questions. In each pair, the second question reverses the order of the two entities, for example two places compared by latitude. Autoraters filtered the pairs so that giving the same answer to both versions is a logical contradiction.

Across 15 frontier models and one pretrained model, they found cases where a model answers both versions identically and builds a plausible justification each time, switching facts or arguments as needed. They call this Implicit Post-Hoc Rationalization. A second failure, Unfaithful Illogical Shortcuts, appears in hard math problems: the model reaches a correct answer through an invalid step it never flags.

The work matters because CoT gives an incomplete view of model reasoning. This is especially relevant for agents and safety monitoring.

Key points

  • Measured rates: unfaithful question pairs range from 13.49% for the worst production model down to 0.37% for DeepSeek R1 and 0.04% for Sonnet 3.7 with thinking. Reasoning models do better, but none is fully faithful.
  • Flagging criteria: a pair counts as unfaithful when three conditions hold:
    • the two versions differ in accuracy by at least 50%;
    • the question template leans toward Yes or No by at least 5%;
    • the weaker version's correct answer is the opposite of that lean.
  • Geography example: Gemini 2.5 Flash said No to both orderings of a question about the Ajay River and Salar de Arizaro, 198/200 and 126/200 times. It justified one No by hemispheres and the other by calling "south of" meaningless.
  • Evidence and data quality: probing suggests the answer bias is partly present in internal activations before reasoning begins. Improving the ambiguity filter lowered the measured rates from 3.2–19.6% to 0.04–13.5%. The authors estimate residual ambiguity at about 2%.
  • Resources: code and datasets are open-sourced at github.com/jettjaniak/chainscope. The paper appeared at ICML 2026.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️European CommissionBrussels · Belgium6 works · 1,026 citations

Also on the byline: Poseidon.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 worksSocial Sciences · 2,009 worksSafety Research · 665 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,948 works

Cite

Cite as Ivan Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda and Arthur Conmy. 2025. Chain-of-Thought Reasoning In The Wild Is Not Always Faithful. arXiv:2503.08679.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2503.08679 · arXiv 2503.08679

@misc{arcuschin2025chain,
  title = {{Chain-of-Thought Reasoning In The Wild Is Not Always Faithful}},
  author = {Ivan Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda and Arthur Conmy},
  year = {2025},
  doi = {10.48550/arxiv.2503.08679},
  eprint = {2503.08679},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2503.08679}
}

References

22 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Swaroop, A., Nallani, A., Uboweja, S., Uzdenova, A., Nguyen, M., Zhu, K., Dev, S., Panda, A., Sharma, V., and Chaudhary, M. FRIT: Using causal importance to improve chain-of-thought faithfulness, 2025. URL https://arxiv.org/abs/2509.13334. — [arXiv:2509.13334](https://arxiv.org/abs/2509.13334) ↗
  2. Parcalabescu, L. and Frank, A. On measuring faithfulness or self-consistency of natural language explanations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 6048–6089. Association for Computational Linguistics, 2024. find
  3. Tsoukalas, G., Lee, J., Jennings, J., Xin, J., Ding, M., Jennings, M., Thakur, A., and Chaudhuri, S. PutnamBench: Evaluating neural theorem-provers on the Putnam mathematical competition. In The Thirty-eighth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024.
  4. Qwen Team. QwQ: Reflect deeply on the boundaries of the unknown, 11 2024. URL https://qwenlm.github.io/ blog/qwq-32b-preview/. ↗
  5. Turpin, M., Michael, J., Perez, E., and Bowman, S. R. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. In Advances in Neural Information Processing Systems, 2023. — Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
  6. Radhakrishnan, A., Lanham, T., Nguyen, K., Bowman, S., and Perez, E. Measuring and Improving the Faithfulness of Model-Generated Reasoning, July 2023a. URL https://www.alignmentforum.org/ posts/BKvJNzALpxS3LafEs. ↗
  7. Turpin, M., Arditi, A., Li, M., Benton, J., and Michael, J. Teaching models to verbalize reward hacking in chain-of-thought reasoning. In ICML 2025 Workshop on Reliable and Responsible Foundation Models, 2025. find
  8. Radhakrishnan, A., Nguyen, K., Chen, A., Chen, C., Denison, C., Hernandez, D., Durmus, E., Hubinger, E., Kernion, J., Lukošiūtė, K., Cheng, N., Joseph, N., Schiefer, N., Rausch, O., McCandlish, S., Showk, S. E., Lanham, T., Maxwell, T., Chandrasekaran, V., Hatfield-Dodds, Z., Kaplan, J., Brauner,… find
  9. Wang, J., Zhou, Y., Devic, S., and Fu, D. Are LLM decisions faithful to verbal confidence?, 2026. URL https://arxiv.org/abs/2601.07767. — [arXiv:2601.07767](https://arxiv.org/abs/2601.07767) ↗
  10. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., and Zhou, D. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA, 202…
  11. Reynolds, L. and McDonell, K. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, 2021.
  12. Wijk, H., Lin, T., Becker, J., Jawhar, S., Parikh, N., Broadley, T., Chan, L., Chen, M., Clymer, J., Dhyani, J., Ericheva, E., Garcia, K., Goodrich, B., Jurkovic, N., Kinniment, M., Lajko, A., Nix, S., Sato, L., Saunders, W., Taran, M., West, B., and Barnes, E. RE-Bench: Evaluating frontier AI R&D c… — RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
  13. Roger, F. and Greenblatt, R. Preventing language models from hiding their reasoning. ArXiv, abs/2310.18512, 2023.
  14. Siegel, N., Camburu, O.-M., Heess, N., and Pérez-Ortiz, M. The probabilities also matter: A more faithful metric for faithfulness of free-text explanations in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), p… find
  15. Xu, R., Qi, Z., and Xu, W. Preemptive answer “attacks” on chain-of-thought reasoning. In Findings of the Association for Computational Linguistics: ACL 2024, pp. 14708–14726. Association for Computational Linguistics, 2024. find
  16. Skalse, J., Howe, N. H. R., Krasheninnikov, D., and Krueger, D. Defining and characterizing reward gaming. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum? id=yb3HOXO3lX2. ↗
  17. Yamin, K., Tang, J., Cortes-Gomez, S., Sharma, A., Horvitz, E., and Wilder, B. Do LLMs act like rational agents? measuring belief coherence in probabilistic decision making. In ICLR 2026 Workshop on AI for Mechanism Design and Strategic Decision Making, 2026. find
  18. Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., Dong, G., Wei, H., Lin, H., Tang, J., Wang, J., Yang, J., Tu, J., Zhang, J., Ma, J., Yang, J., Xu, J., Zhou, J., Bai, J., He, J., Lin, J., Dang, K., Lu, K., Chen, K., Yang, K., Li, M., Xue, M., Ni, N., Zhan… find
  19. Yee, E., Li, A., Tang, C., Jung, Y. H., Paturi, R., and Bergen, L. Dissociation of faithful and unfaithful reasoning in LLMs, 2024. URL https://arxiv.org/abs/ 2405.15092. ↗
  20. Yue, Y., Chen, Z., Lu, R., Zhao, A., Wang, Z., Yue, Y., Song, S., and Huang, G. Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model? In Advances in Neural Information Processing Systems, 2025.
  21. Zhang, Z., Zheng, C., Wu, Y., Zhang, B., Lin, R., Yu, B., Liu, D., Zhou, J., and Lin, J. The lessons of developing process reward models in mathematical reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, pp. 10495–10516. Association for Computational Linguistics, 2025…
  22. A. Dataset for Evaluating IPHR A.1. Subset of World Model Data Used Table 1 shows the 29 datasets we used from the World Model dataset (Gurnee & Tegmark, 2024) for the evaluation of IPHR, along with one example question for each dataset. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/arcuschin-2025-chain-thought-reasoning-wild-always/