Elamur Research⌕
☰
home / papers / 2025 / Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLMs

Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLMs

Paper · 2025 · Zhen Xiong, Yujun Cai, Zhecheng Li et al.

Summary

Large Reasoning Models (LRMs) such as DeepSeek-R1 produce long Chain-of-Thought traces, yet they often behave erratically. They may overthink, hallucinate premises, or perform worse when given few-shot examples. Earlier analyses mostly tracked a single variable, the number of reasoning tokens. Xiong et al. instead propose a framework that turns a raw reasoning trace into a directed graph. First, the output is split at "\n\n" delimiters into small units. An LLM, guided by a dedicated prompt, then merges these units into larger, semantically coherent reasoning steps, which become the nodes. Edges encode dependencies between steps in a ternary adjacency matrix with values −1, 0 and 1. From these graphs the authors compute topological metrics, including exploration density, branching ratio, convergence ratio and linearity. They compare these metrics across model sizes and prompting styles on math and coding tasks. Richer, less linear graphs go together with higher accuracy, while prompts that are more constrained or verbose push models toward linear reasoning. The work offers a way to judge reasoning quality beyond final-answer accuracy and has practical implications for prompt engineering.

Key points

  • The pipeline has three stages: segmentation at "\n\n", LLM-based context-aware clustering of units into steps, and extraction of dependency edges, which yields a graph G = (V, A) with A ∈ {−1, 0, 1}^{n×n}.
  • Three models were evaluated: DeepSeek-R1-distilled Llama-8B and Qwen-14B, plus Qwen3-32B. Each was tested under zero-shot, "concise" and "explanatory" prompting.
  • Zero-shot prompting gave the best results in every case. For example, Qwen3-32B reached 56.2% accuracy with zero-shot prompting and 50.1% with explanatory prompts. Over the same change, its linearity rose from 0.444 to 0.931.
  • Higher accuracy tracks with three graph metrics, each significant at the 0.05 level: a correlation of r = 0.68 with exploration density, r = 0.67 with the branching ratio, and r = 0.68 with the convergence ratio.
  • Few-shot demonstrations, especially verbose ones, lead models to imitate the structure of the examples, producing shorter and more stereotyped reasoning chains. Even a single demonstration triggers this shift.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Southern CaliforniaLos Angeles · United States162 works · 35,836 citations🎓University of California San DiegoSan Diego · United States73 works · 51,948 citations🎓The University of QueenslandBrisbane · Australia14 works · 4,603 citations🎓University of California, MercedMerced · United States11 works · 425 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 works

Cite

Cite as Zhen Xiong, Yujun Cai, Zhecheng Li and Yiwei Wang. 2025. Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLMs. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.896

@inproceedings{xiong2025mapping,
  title = {{Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLMs}},
  author = {Zhen Xiong, Yujun Cai, Zhecheng Li and Yiwei Wang},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.896},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.896}
}

References

24 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2025. Claude’s extended thinking. https://www.anthropic.com/news/ visible-extended-thinking. Accessed: 2025-05-18. ↗
  2. Marthe Ballon, Andres Algaba, and Vincent Ginis. 2025. The relationship between reasoning and performance in large language models – o3 (mini) thinks harder, not longer. ArXiv. find
  3. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  4. Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. 2025. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. https://arxiv.org/abs/2503.09567. ArXiv preprint arXiv:2503.09567. ↗
  5. Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. 2024. Do not think that much for 2+3=? on the overthinking of o1-like llms. https://arxiv.org/abs/2412.21187. ArXiv preprint a… ↗
  6. Sergio Da Silva. 2023. System 1 vs. system 2 thinking. Psych, 5(4):1057–1076. find
  7. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z.F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 2 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms…
  8. DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, and 179 others. 2024. Deepseek-v3 technical report. https://arxiv.org… ↗
  9. Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2024. Swe-bench: Can language models resolve real-world github issues? In Proceedings of the Twelfth International Conference on Learning Representations (ICLR).
  10. Koray Kavukcuoglu. 2025. Gemini 2.5: Our most intelligent ai model. https://blog.google/technology/google-deepmind/ gemini-model-thinking-updates-march-2025/ \#gemini-2-5-pro. Accessed: 2025-05-18. ↗
  11. Amit Arnold Levy and Mor Geva. 2024. Language models encode numbers using digit representations in base 10. arXiv preprint arXiv:2410.11781. find
  12. Zhecheng Li, Yiwei Wang, Bryan Hooi, Yujun Cai, Zhen Xiong, Nanyun Peng, and Kai-Wei Chang. 2024. Vulnerability of llms to vertically aligned text manipulations. arXiv preprint arXiv:2410.20016.
  13. Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, Yingying Zhang, Fei Yin, Jiahua Dong, Zhiwei Li, Bao-Long Bi, Ling-Rui Mei, Junfeng Fang, Zhijiang Guo, Le Song, and Cheng-Lin Liu. 2025. From system 1 to system 2…
  14. OpenAI. 2025. Openai o3 and o4-mini system card. https://openai.com/index/ o3-o4-mini-system-card/. Accessed: 2025-05-18. ↗
  15. OpenAI, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720.
  16. Bhrij Patel, Souradip Chakraborty, Wesley A. Suttle, Mengdi Wang, Amrit Singh Bedi, and Dinesh Manocha. 2024. Aime: Ai system optimization via multiple llm evaluators. https://arxiv.org/abs/ 2410.03131. ArXiv preprint arXiv:2410.03131. — [arXiv:2410.03131](https://arxiv.org/abs/2410.03131) ↗
  17. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2023. Gpqa: A graduate-level google-proof q&a benchmark. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP).
  18. Yisheng Song, Ting Wang, Puyu Cai, Subrota K. Mondal, and Jyoti Prakash Sahoo. 2023. A comprehensive survey of few-shot learning: Evolution, applications, challenges, and opportunities. ACM Comput. Surv., 55(13s). find
  19. Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, and Xia Hu. 2025. Stop overthinking: A survey on efficient reasoning for large language models. https://arxiv.org/abs/2503.16419. ArXiv preprint arXiv:2503.16419. ↗
  20. Yiwei Wang, Yujun Cai, Muhao Chen, Yuxuan Liang, and Bryan Hooi. 2023. Primacy effect of chatgpt. arXiv preprint arXiv:2310.13206. find
  21. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems.
  22. Zhen Xiong, Yujun Cai, Bryan Hooi, Nanyun Peng, Zhecheng Li, and Yiwei Wang. 2025. Enhancing llm character-level manipulation via divide and conquer. arXiv preprint arXiv:2502.08180. — [arXiv:2502.08180](https://arxiv.org/abs/2502.08180) find
  23. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025a. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  24. Wenkai Yang, Shuming Ma, Yankai Lin, and Furu Wei. 2025b. Towards thinking-optimal scaling of test-time compute for llm reasoning. https://arxiv.org/abs/2502.18080. ArXiv preprint arXiv:2502.18080. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xiong-2025-mapping-minds-llms-graph-based/