Elamur Research⌕
☰
home / papers / 2025 / Understanding the Thinking Process of Reasoning Models: A Perspective from Schoenfeld's Episode Theory

Understanding the Thinking Process of Reasoning Models: A Perspective from Schoenfeld's Episode Theory

Paper · 2025 · Ming Li, Nan Zhang, Chenrui Fan et al.

Summary

Large Reasoning Models produce long chains of thought, but there has been no theory-based way to describe how those traces are organized. The authors borrow Schoenfeld's Episode Theory, a framework from mathematics education built from think-aloud recordings of students, and use it to label DeepSeek-R1 reasoning on SAT math problems. They collected R1 outputs for 1,385 SAT Suite Question Bank items. They then annotated a subset hierarchically: each paragraph received a coarse label and each sentence received one of seven episode labels. They release the annotated corpus and detailed guidebooks.

Transition analysis shows recurring patterns resembling human problem solving, such as planning typically leading into implementation. The authors also test automatic labeling. GPT-4.1, prompted with the guidebook and examples, performs best. Fine-tuned BERT is the strongest trained classifier. This matters because it offers a reusable, cognitively grounded protocol for interpreting and eventually steering machine reasoning. The current scope is limited to moderately difficult SAT math, and automatic labeling accuracy is still modest.

Key points

  • Annotated set: 38 R1 responses (two for each of 19 SAT skill categories), totaling 915 paragraphs and 3,087 sentences.
  • Label scheme:
    • Paragraph level: General, Verify, Explore.
    • Sentence level: Read, Analyze, Plan, Implement, Explore, Verify, Monitor. Monitor is an addition to Schoenfeld's original six episodes.
  • Transition patterns: The strongest off-diagonal transitions are Read→Analyze, Plan→Implement, and Explore→Analyze.
  • Automatic labeling results (30% test split, sentence level):
    • GPT-4.1 (examples + guidebook): accuracy 0.805, Cohen's κ 0.764.
    • BERT: 0.732 accuracy, κ 0.671.
    • RoBERTa: 0.730 accuracy, κ 0.670.
    • Gemini-embedding SVM, MLP, and KNN classifiers scored lower.
    • Adding the guidebook to prompts clearly boosted LLM accuracy.
  • Main error sources: Most confusions occur between Verify and either Analyze or Implement. Resources are available at github.com/MingLiiii/Schoenfeld_Reasoning.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Maryland, College ParkCollege Park · United States132 works · 25,380 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Ming Li, Nan Zhang, Chenrui Fan, Hong Jiao, Yanbin Fu, Sydney Peters, Qingshu Xu, Robert Lissitz and Tianyi Zhou. 2025. Understanding the Thinking Process of Reasoning Models: A Perspective from Schoenfeld's Episode Theory. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.emnlp-main.922

@inproceedings{li2025understanding,
  title = {{Understanding the Thinking Process of Reasoning Models: A Perspective from Schoenfeld's Episode Theory}},
  author = {Ming Li, Nan Zhang, Chenrui Fan, Hong Jiao, Yanbin Fu, Sydney Peters, Qingshu Xu, Robert Lissitz and Tianyi Zhou},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.922},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.922}
}

References

44 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hyung Won Chung, Le Hou, S. Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Dasha Valter, Sharan Narang, Gaurav Mishra, and 13 others. 2022. Scaling i… find
  2. Ellie Darlington. 2013. The use of bloom’s taxonomy in advanced mathematics questions. In Proceedings of the British Society for Research into Learning Mathematics, volume 33, pages 7–12. find
  3. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, and etc. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. Preprint, arXiv:2501.12948.
  4. Carole Greenes. 1995. Mathematics learning and knowing: A cognitive process. Journal of Education, 177(1):85–106. find
  5. E. Harskamp and C. Suhre. 2007. Schoenfeld’s problem solving theory in a student controlled learning environment. Computers & Education, 49(3):822–839. find
  6. Lulu Huang, Da Li, Hao Liu, and Lingxiao Cheng. 2025. Beyond accuracy: The role of calibration in self-improving large language models. arXiv preprint arXiv:2504.02902. — [arXiv:2504.02902](https://arxiv.org/abs/2504.02902) find
  7. David R. Krathwohl. 2002. A revision of bloom’s taxonomy: An overview. Theory into Practice, 41(4):212– 218. find
  8. Ana Kuzle. 2013. Patterns of metacognitive behavior during mathematics problem-solving in a dynamic geometry environment. International Electronic Journal of Mathematics Education, 8(1):20–40. find
  9. Ming Li, Yanhong Li, and Tianyi Zhou. 2024a. What happened in llms layers when trained for fast vs. slow thinking: A gradient perspective. arXiv preprint arXiv:2410.23743.
  10. Ming Li, Zhengyuan Yang, Xiyao Wang, Dianqi Li, Kevin Lin, Tianyi Zhou, and Lijuan Wang. 2025. What makes reasoning models different? follow the reasoning leader for efficient decoding. arXiv preprint arXiv:2506.06998. — [arXiv:2506.06998](https://arxiv.org/abs/2506.06998) find
  11. Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou. 2024b. Superfiltering: Weak-to-strong data filtering for fast instruction-tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pag…
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  13. Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. 2024c. From quantity to quality: Boosting LLM performance with self-guided data selection for instruction tuning. In Proceedings of the 2024 Conference of the North American Chapter of t…
  14. Chenrui Fan, Ming Li, Lichao Sun, and Tianyi Zhou. 2025. Missing premise exacerbates overthinking: Are reasoning models losing critical thinking skill? arXiv preprint arXiv:2504.06514. — [arXiv:2504.06514](https://arxiv.org/abs/2504.06514) find
  15. Changshu Liu, Shizhuo Dylan Zhang, Ali Reza Ibrahimzada, and Reyhaneh Jabbarvand. 2024. Codemind: A framework to challenge large language models for code reasoning. Preprint, arXiv:2402.09664. — [arXiv:2402.09664](https://arxiv.org/abs/2402.09664) find
  16. Kanishk Gandhi, Abhishek Chakravarthy, Ansh Singh, Nik Lile, and Noah D. Goodman. 2025. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. Preprint, arXiv:2503.01307.
  17. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692.
  18. Sara Vera Marjanović, Arkil Patel, Vaibhav Adlakha, Milad Aghajohari, Parishad BehnamGhader, Mehar Bhatia, Aditi Khandelwal, Austin Kraft, Benno Krojer, Xing Han Lù, and 1 others. 2025. Deepseek-r1 thoughtology: Let’s think about llm reasoning. arXiv preprint arXiv:2504.07128. find
  19. John Mason, Leone Burton, and Kaye Stacey. 2010. Thinking Mathematically, second edition. Pearson Education, Harlow, England. find
  20. Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2021. Cross-task generalization via natural language crowdsourcing instructions. arXiv preprint arXiv:2104.08773. find
  21. Melanie Mitchell. 2025. Artificial intelligence learns to reason. Science, 387(6740):eadw5211. find
  22. Lianlei Shan, Shixian Luo, Zezhou Zhu, Yu Yuan, and Yong Wu. 2025. Cognitive memory in large language models. Preprint, arXiv:2504.02441. — [arXiv:2504.02441](https://arxiv.org/abs/2504.02441) find
  23. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Preprint, arXiv:2402.03300.
  24. Mark Steyvers and Megan A. K. Peters. 2025. Metacognition and uncertainty communication in humans and large language models. arXiv preprint arXiv:2504.14045. — [arXiv:2504.14045](https://arxiv.org/abs/2504.14045) find
  25. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. Preprint, arXiv:2501.19393.
  26. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. Preprint…
  27. Sam Musker, Alex Duchnowski, Raphaël Millière, and Ellie Pavlick. 2025. Llms as models for analogical reasoning. Preprint, arXiv:2406.13803. find
  28. Yuqing Wang and Yun Zhao. 2023. Gemini in reasoning: Unveiling commonsense in multimodal large language models. Preprint, arXiv:2312.17661. — [arXiv:2312.17661](https://arxiv.org/abs/2312.17661) find
  29. OpenAI. 2023. Gpt-4 technical report. arXiv:2303.08774.
  30. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022. Finetuned language models are zero-shot learners. In International Conference on Learning Representations.
  31. Preprint, OpenAI. 2024a. OpenAI o1 System Card.
  32. OpenAI. 2024b. Openai o1 system card. https://openai.com/index/openai-o1-system-card/. Accessed: 2025-08-12. ↗
  33. George Pólya. 1945. How to Solve It. Princeton University Press, Princeton. find
  34. Farzad Radmehr and Michael Drake. 2018. Revised bloom’s taxonomy and major theories and frameworks that influence the teaching, learning, and assessment of mathematics: a comparison. International Journal of Mathematical Education in Science and Technology, 49(6):917–929. find
  35. Teven Le Scao, Angela Fan, Christopher Akiki, Elizabeth-Jane Pavlick, Suzana Ili’c, Daniel Hesslow, Roman Castagn’e, Alexandra Sasha Luccioni, Franccois Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Rose Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot,… find
  36. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elicits reasoning in large language models. Preprint, arXiv:2201.11903.
  37. Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, and Pengfei Liu. 2025. Evaluating mathematical reasoning beyond accuracy. Preprint, arXiv:2404.05692.
  38. Wei Xiong, Hanning Zhang, Chenlu Ye, Lichang Chen, Nan Jiang, and Tong Zhang. 2025. Self-rewarding correction for mathematical reasoning. Preprint, arXiv:2502.19613. — [arXiv:2502.19613](https://arxiv.org/abs/2502.19613) find
  39. Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. 2024. A survey on knowledge distillation of large language models. Preprint, arXiv:2402.13116. — [arXiv:2402.13116](https://arxiv.org/abs/2402.13116) find
  40. Mathematical Problem Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. 2025. Limo: Less is more for reasoning. Preprint, arXiv:2502.03387.
  41. Alan H. Schoenfeld. 2016. Learning to think mathematically: Problem solving, metacognition, and sense making in mathematics. Journal of Education, 196(2):1–38. find
  42. Joseph B. W. Yeo and Ban Har Yeap. 2010. Characterising the cognitive processes in mathematical investigation. International Journal for Mathematics Teaching and Learning, pages 1–10. find
  43. Alan H. Schoenfeld. 1985. Solving. Academic Press. find
  44. Weixiang Zhao, Xingyu Sui, Jiahe Guo, Yulin Hu, Yang Deng, Yanyan Zhao, Bing Qin, Wanxiang Che, Tat-Seng Chua, and Ting Liu. 2025. Trade-offs in large reasoning models: An empirical analysis of deliberative and adaptive reasoning over foundational capabilities. arXiv preprint arXiv:2503.17979. — [arXiv:2503.17979](https://arxiv.org/abs/2503.17979) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2025-understanding-thinking-process-reasoning-models/