Elamur Research⌕
☰
home / papers / 2026 / Reasoning Fails Where Step Flow Breaks

Reasoning Fails Where Step Flow Breaks

Paper · 2026 · Xiaoyu Xu, Yulan Pan, Xiaosong Yuan et al.

Summary

Large reasoning models (LRMs) produce long question–thinking–summary traces, but their mistakes are hard to trace back to specific parts of the reasoning. Token-level attention and saliency maps are too dense and noisy for this. The authors propose Step-Saliency, which multiplies attention by its gradient, row-normalizes the result, and averages it within reasoning steps. This yields compact step→step influence maps that can be compared across layers.

The maps show two failure patterns in incorrect traces:

  • Shallow Lock-in: early layers focus narrowly on the current step and ignore earlier context.
  • Deep Decay: late layers lose influence from the thinking segment, so the summary mostly attends to itself.

To counter these, they introduce StepFlow, a test-time fix that needs no retraining. Odds-Equal Bridge keeps attention mass on the question and earlier steps in shallow layers. Step Momentum Injection carries a small residual from the previous step across step boundaries in deep layers.

StepFlow raises accuracy on math, science and coding benchmarks for several LRMs. This suggests that some reasoning failures come from broken information flow rather than missing knowledge.

Key points

  • Models and benchmarks: DeepSeek-R1-Distill (7B/14B/32B), GPT-OSS-20B and QwQ-32B-Preview, tested on AIME24, AIME25, AMC23, MATH-500, GPQA-Diamond and LiveCodeBench.
  • Ablation on GPT-OSS-20B (medium): each component helps on its own, and the combination works best.
    • AIME25: 62.0 → 69.2
    • GPQA-D: 65.2 → 70.3
    • LiveCodeBench: 70.0 → 79.5
  • LiveCodeBench by difficulty: the largest gains are on medium problems (70.2 → 84.0) and hard problems (37.8 → 52.0).
  • Compute-normalized comparison (R1-Distill-32B, AIME 24/25 averaged):
    • At about 1.35× compute, StepFlow gains 5.7× more than simply generating longer outputs.
    • Self-consistency needs about k=8 (8× compute) to match StepFlow.
    • StepFlow plus SC(k=2) beats SC(k=4).
  • Error types and layer coverage: StepFlow mainly fixes information-propagation errors (72%) rather than conceptual ones (10–14%). Gains are best when it is applied to the bottom and top quarter of layers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Shanghai Jiao Tong UniversityShanghai · China149 works · 16,161 citations🎓Jilin UniversityChangchun · China25 works · 5,252 citations🎓Fuzhou UniversityFuzhou · China3 works · 17 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Xiaoyu Xu, Yulan Pan, Xiaosong Yuan, Zhihong Shen, Minghao Su, Yuanhao Su and Xiaofeng Zhang. 2026. Reasoning Fails Where Step Flow Breaks. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1212

@inproceedings{xu2026reasoning,
  title = {{Reasoning Fails Where Step Flow Breaks}},
  author = {Xiaoyu Xu, Yulan Pan, Xiaosong Yuan, Zhihong Shen, Minghao Su, Yuanhao Su and Xiaofeng Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1212},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1212}
}

References

52 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shuoyang Ding and Philipp Koehn. 2021. Evaluating saliency methods for neural language models. arXiv preprint arXiv:2104.05824. find
  2. Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, and 1 others. 2025. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925.
  3. Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. 2024. Large language models for mathematical reasoning: Progresses and challenges. arXiv preprint arXiv:2402.00157.
  4. Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. 2017. Towards better understanding of gradient-based attribution methods for deep neural networks. arXiv preprint arXiv:1711.06104. — [arXiv:1711.06104](https://arxiv.org/abs/1711.06104) find
  5. Arindam Banerjee, Srujana Merugu, Inderjit S Dhillon, and Joydeep Ghosh. 2005. Clustering with bregman divergences. Journal of machine learning research, 6(Oct):1705–1749. find
  6. Fazl Barez, Tung-Yu Wu, Iván Arcuschin, Michael Lan, Vincent Wang, Noah Siegel, Nicolas Collignon, Clement Neo, Isabelle Lee, Alasdair Paren, and 1 others. 2025. Chain-of-thought is not explainability. Preprint, alphaXiv, page v1. find
  7. Angie Boggust, Harini Suresh, Hendrik Strobelt, John Guttag, and Arvind Satyanarayan. 2023. Saliency cards: a framework to characterize and compare saliency methods. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 285–296. find
  8. Johan Boye and Birger Moell. 2025. Large language models and mathematical reasoning failures. arXiv preprint arXiv:2502.11574. find
  9. Erik Cambria, Lorenzo Malandri, Fabio Mercorio, Navid Nobani, and Andrea Seveso. 2024. Xai meets llms: A survey of the relation between explainable ai and large language models. arXiv preprint arXiv:2407.15248. — [arXiv:2407.15248](https://arxiv.org/abs/2407.15248) find
  10. Chuxue Cao, Mengze Li, Juntao Dai, Jinluan Yang, Zijian Zhao, Shengyu Zhang, Weijie Shi, Chengzhong Liu, Sirui Han, and Yike Guo. 2025. Towards advanced mathematical reasoning for llms via first-order logic theorem proving. arXiv preprint arXiv:2506.17104. find
  11. Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. 2025a. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567.
  12. Zican Dong, Tianyi Tang, Lunyi Li, and Wayne Xin Zhao. 2023. A survey on long text modeling with transformers. arXiv preprint arXiv:2302.14502. — [arXiv:2302.14502](https://arxiv.org/abs/2302.14502) find
  13. Subhabrata Dutta, Joykirat Singh, Soumen Chakrabarti, and Tanmoy Chakraborty. 2024. How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning. arXiv preprint arXiv:2402.18312. find
  14. Sinan Fan, Liang Xie, Chen Shen, Ge Teng, Xiaosong Yuan, Xiaofeng Zhang, Chenxi Huang, Wenxiao Wang, Xiaofei He, and Jieping Ye. 2025. Improving complex reasoning with dynamic prompt corruption: A soft prompt optimization approach. arXiv preprint arXiv:2503.13208. — [arXiv:2503.13208](https://arxiv.org/abs/2503.13208) find
  15. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495. find
  16. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  17. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874.
  18. Tobias Huber, Benedikt Limmer, and Elisabeth André. 2022. Benchmarking perturbation-based saliency maps for explaining atari agents. Frontiers in Artificial Intelligence, 5:903875. find
  19. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974.
  20. Sarthak Jain and Byron C Wallace. 2019. Attention is not explanation. arXiv preprint arXiv:1902.10186.
  21. Mingyu Jin, Weidi Luo, Sitao Cheng, Xinyi Wang, Wenyue Hua, Ruixiang Tang, William Yang Wang, and Yongfeng Zhang. 2025. Disentangling memory and reasoning ability in large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pape…
  22. Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. 2020. Attention is not only a weight: Analyzing transformers with vector norms. arXiv preprint arXiv:2004.10102.
  23. Debjit Paul, Robert West, Antoine Bosselut, and Boi Faltings. 2024. Making reasoning matter: Measuring and improving faithfulness of chain-of-thought reasoning. arXiv preprint arXiv:2402.13950.
  24. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199– 22213.
  25. Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, and Niki Van Stein. 2025. Multi-step reasoning with large language models, a survey. ACM Computing Surveys.
  26. Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, and 1 others. 2023. Measuring faithfulness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702.
  27. Jinu Lee and Julia Hockenmaier. 2025. Evaluating step-by-step reasoning traces: A survey. arXiv preprint arXiv:2502.12289. find
  28. Alan Li, Yixin Liu, Arpan Sarkar, Doug Downey, and Arman Cohan. 2025a. Demystifying scientific problem-solving in llms by probing knowledge and reasoning. arXiv preprint arXiv:2508.19202. — [arXiv:2508.19202](https://arxiv.org/abs/2508.19202) find
  29. Chengpeng Li, Mingfeng Xue, Zhenru Zhang, Jiaxi Yang, Beichen Zhang, Bowen Yu, Binyuan Hui, Junyang Lin, Xiang Wang, and Dayiheng Liu. 2025b. Start: Self-taught reasoner with tools. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 13523–13564.
  30. Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. 2023. Making language models better reasoners with step-aware verifier. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5315–5333.
  31. Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, and 1 others. 2025c. From system 1 to system 2: A survey of reasoning large language models. arXiv preprint arXiv:2502.17419.
  32. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. In The Twelfth International Conference on Learning Representations.
  33. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling.
  34. Baturay Saglam, Paul Kassianik, Blaine Nelson, Sajana Weerawardhena, Yaron Singer, and Amin Karbasi. 2025. Large language models encode semantics in low-dimensional linear subspaces. arXiv preprint arXiv:2507.09709. — [arXiv:2507.09709](https://arxiv.org/abs/2507.09709) find
  35. Hassan Sajjad, Nadir Durrani, Fahim Dalvi, Firoj Alam, Abdul Khan, and Jia Xu. 2022. Analyzing encoded concepts in transformer language models. In Proceedings of the 2022 Conference of the North American chapter of the Association for Computational Linguistics: Human language technologies, pages 308… find
  36. Adi Simhi, Itay Itzhak, Fazl Barez, Gabriel Stanovsky, and Yonatan Belinkov. Trust me, i’m wrong: Llms hallucinate with certainty despite knowing the answer. find
  37. Peiyang Song, Pengrui Han, and Noah Goodman. 2025. A survey on large language model reasoning failures. In 2nd AI for Math Workshop@ ICML 2025. find
  38. Qwen Team. 2025. Qwq-32b: Embracing the power of reinforcement learning. find
  39. Betty Van Aken, Benjamin Winter, Alexander Löser, and Felix A Gers. 2019. How does bert answer questions? a layer-wise analysis of transformer representations. In Proceedings of the 28th ACM international conference on information and knowledge management, pages 1823–1832. find
  40. Shikhar Vashishth, Shyam Upadhyay, Gaurav Singh Tomar, and Manaal Faruqui. 2019. Attention interpretability across nlp tasks. arXiv preprint arXiv:1909.11218. — [arXiv:1909.11218](https://arxiv.org/abs/1909.11218) find
  41. Haoyan Luo and Lucia Specia. 2024. From understanding to utilization: A survey on explainability for large language models. arXiv preprint arXiv:2401.12874. — [arXiv:2401.12874](https://arxiv.org/abs/2401.12874) find
  42. Jesse Vig and Yonatan Belinkov. 2019. Analyzing the structure of attention in a transformer language model. arXiv preprint arXiv:1906.04284.
  43. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori B Hashimoto. 2025. s1: Simple test-time Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, Goran Glavaš, and Anna Korhonen. 2020. Probing pretrai… — [arXiv:2010.05731](https://arxiv.org/abs/2010.05731) find
  44. Junlin Wang, Jens Tuyls, Eric Wallace, and Sameer Singh. 2020a. Gradient-based analysis of nlp models is manipulable. arXiv preprint arXiv:2010.05419. find
  45. Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. 2023a. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. arXiv preprint arXiv:2305.04091. find
  46. Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Yuwei Fang, Siqi Sun, Yu Cheng, and Jingjing Liu. 2020b. Cluster-former: Clustering-based sparse transformer for long-range dependency encoding. arXiv preprint arXiv:2009.06097. — [arXiv:2009.06097](https://arxiv.org/abs/2009.06097) find
  47. Xuezhi Wang, Y Liu, and Z Hu. 2023b. Self-consistency improves chain-of-thought reasoning in large language models. arXiv preprint arXiv:2304.00006. — [arXiv:2304.00006](https://arxiv.org/abs/2304.00006) find
  48. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  49. Juncheng Wu, Sheng Liu, Haoqin Tu, Hang Yu, Xiaoke Huang, James Zou, Cihang Xie, and Yuyin Zhou. 2025. Knowledge or reasoning? a close look at how llms think across domains. arXiv preprint arXiv:2506.02126. find
  50. Xiaosong Yuan, Chen Shen, Shaotian Yan, Xiaofeng Zhang, Liang Xie, Wenxiao Wang, Renchu Guan, Ying Wang, and Jieping Ye. 2024. Instance-adaptive zero-shot chain-of-thought prompting. Advances in Neural Information Processing Systems, 37:125469– 125486. find
  51. Qingru Zhang, Chandan Singh, Liyuan Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, and Tuo Zhao. 2023. Tell your model where to attend: Post-hoc attention steering for llms. arXiv preprint arXiv:2311.02262. — [arXiv:2311.02262](https://arxiv.org/abs/2311.02262) find
  52. Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 15(2):1–38.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xu-2026-reasoning-fails-where-step-flow/