Elamur Research⌕
☰
home / papers / 2026 / Tracing Mathematical Proficiency Through Problem-Solving Processes

Tracing Mathematical Proficiency Through Problem-Solving Processes

Paper · 2026 · Jungyang Park, Suho Kang, Jaewoo Park et al.

Summary

Knowledge Tracing (KT) models usually forecast whether a learner will solve the next problem from past right/wrong outcomes and metadata such as concepts or timing. These signals reveal little about how the student actually reasons. The authors reframe the task as KT-PSP, in which every interaction also includes the student's written problem-solving process (PSP). To support this task they release KT-PSP-25, a set of math sessions from a tablet-based learning platform. Handwritten work was converted to text with a GPT-5 OCR pipeline and anonymized.

Their framework, StatusKT, chains three LLM stages:

  • a teacher model drafts problem-specific Mathematical Proficiency (MP) indicators,
  • a student model answers them using the learner's written work,
  • a teacher model judges which indicators were mastered.

The resulting MP signals are added as auxiliary inputs to existing DLKT backbones. This improved prediction across architectures and helped in cold-start conditions. It also produced interpretable proficiency explanations. The study suggests that reasoning traces, not only final outcomes, carry valuable signal for intelligent tutoring.

Key points

  • KT-PSP-25 contains 22,289 interactions from 1,343 students on 2,696 problems covering 490 knowledge components. The data were collected from November 2024 to July 2025 and are released under CC BY-NC 4.0.
  • Student PSPs average 24.76 words, compared with 73.54 words for reference solutions. The average correct ratio per student is 0.72.
  • In a cold-start test with student-level holdout and only the first five interactions, adding StatusKT raised ACC for every backbone. AUC was comparable or better in most cases.
  • Case analysis identified a failure mode in which MP indicators are over-enriched or misaligned with the item. The authors propose separating core from enriched indicators as future work.
  • All three stages use GPT-5, which requires two LLM calls per student-problem interaction at evaluation time. Distilling the pipeline into smaller models is planned.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Seoul National UniversitySeoul · South Korea41 works · 45,671 citations🎓Yonsei UniversitySeoul · South Korea17 works · 501 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksInformation Systems · 1,437 worksComputer Science Applications · 102 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Jungyang Park, Suho Kang, Jaewoo Park, Jae Hong Kim, Jaewoo Shin, Seonjoon Park and Youngjae Yu. 2026. Tracing Mathematical Proficiency Through Problem-Solving Processes. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.961

@inproceedings{park2026tracing,
  title = {{Tracing Mathematical Proficiency Through Problem-Solving Processes}},
  author = {Jungyang Park, Suho Kang, Jaewoo Park, Jae Hong Kim, Jaewoo Shin, Seonjoon Park and Youngjae Yu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.961},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.961}
}

References

47 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ghodai Abdelrahman, Sherif Abdelfattah, Qing Wang, and Yu Lin. 2022. Dbe-kt22: A knowledge tracing dataset based on online student evaluation. arXiv preprint arXiv:2208.12651. — [arXiv:2208.12651](https://arxiv.org/abs/2208.12651) find
  2. Ghodai Abdelrahman and Qing Wang. 2019. Knowledge tracing with sequential key-value memory networks. In Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pages 175–184. 19259 9 find
  3. Kingma DP Ba J Adam and 1 others. 2014. A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 1412(6).
  4. Youheng Bai, Xueyi Li, Zitao Liu, Yaying Huang, Teng Guo, Mingliang Hou, Feng Xia, and Weiqi Luo. 2025. cskt: Addressing cold-start problem in knowledge tracing via kernel bias and cone attention. Expert Systems with Applications, 266:125988. find
  5. Indronil Bhattacharjee and Christabel Wayllace. 2025. Cold start problem: An experimental study of knowledge tracing models with new students. Preprint, arXiv:2505.21517. — [arXiv:2505.21517](https://arxiv.org/abs/2505.21517) find
  6. Haw-Shiuan Chang, Hwai-Jung Hsu, Kuan-Ta Chen, and 1 others. 2015. Modeling exercise relationships in e-learning: A unified approach. In EDM, pages 532–535. find
  7. Jiahao Chen, Zitao Liu, Shuyan Huang, Qiongqiong Liu, and Weiqi Luo. 2023. Improving interpretability of deep sequential knowledge tracing models with question-centric cognitive representations. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 14196–14204. find
  8. Penghe Chen, Yu Lu, Yang Pian, Yan Li, and Yunbo Cao. 2022. Introducing response time into guessing and slipping for cognitive diagnosis. In International Conference on Artificial Intelligence in Education, pages 320–324. Springer. find
  9. Barbara Chiu, Christopher Randles, and Stefan Irby. 2022. Analyzing student problem-solving with match. In Frontiers in Education, volume 6, page 769042. Frontiers Media SA. find
  10. Youngduck Choi, Youngnam Lee, Junghyun Cho, Jineon Baek, Byungsoo Kim, Yeongmin Cha, Dongmin Shin, Chan Bae, and Jaewe Heo. 2020a. Towards an appropriate query, key, and value computation for knowledge tracing. In Proceedings of the seventh ACM conference on learning@ scale, pages 341–344. find
  11. Youngduck Choi, Youngnam Lee, Dongmin Shin, Junghyun Cho, Seoyon Park, Seewoo Lee, Jineon Baek, Chan Bae, Byungsoo Kim, and Jaewe Heo. 2020b. Ednet: A large-scale hierarchical dataset in education. In International conference on artificial intelligence in education, pages 69–73. Springer. find
  12. Albert T Corbett and John R Anderson. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction, 4(4):253–278. find
  13. Ibrahim AH El-Shara, Ahmad AS Tabieh, and Sahar YA Abu Helu. 2025. The effect of using matgpt on mathematical proficiency among undergraduate students. International Journal of Information and Education Technology, 15(4). find
  14. Mingyu Feng, Neil Heffernan, and Kenneth Koedinger. 2009. Addressing the assessment challenge with an online system that tutors as it assesses. User modeling and user-adapted interaction, 19(3):243– 266. find
  15. Bradford Findell, Jane Swafford, and Jeremy Kilpatrick. 2001. Adding it up: Helping children learn mathematics. National Academies Press. find
  16. Aritra Ghosh, Neil Heffernan, and Andrew S Lan. 2020. Context-aware attentive knowledge tracing. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2330–2339. find
  17. Bert F Green Jr. 1951. A general solution for the latent class model of latent structure analysis. Psychometrika, 16(2):151–166. find
  18. Teng Guo, Yu Qin, Yubin Xia, Mingliang Hou, Zitao Liu, Feng Xia, and Weiqi Luo. 2025. Enhancing knowledge tracing through decoupling cognitive pattern from error-prone data. In Proceedings of the ACM on Web Conference 2025, pages 5108–5116. find
  19. Xiaopeng Guo, Zhijie Huang, Jie Gao, Mingyu Shang, Maojing Shu, and Jun Sun. 2021. Enhancing knowledge tracing via adversarial training. In Proceedings of the 29th ACM international conference on multimedia, pages 367–375. find
  20. Yuxiang Guo, Shuanghong Shen, Qi Liu, Zhenya Huang, Linbo Zhu, Yu Su, and Enhong Chen. 2024. Mitigating cold-start problems in knowledge tracing with large language models: An attribute-aware approach. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, C… find
  21. Tao Huang, Shengze Hu, Huali Yang, Jing Geng, Zhifei Li, Zhuoran Xu, and Xinjia Ou. 2024. Response speed enhanced fine-grained knowledge tracing: A multi-task learning perspective. Expert Systems with Applications, 238:122107. find
  22. KDD 2010. 2010. Kdd2010. https://kdd.org/kdd-cup/view/ kdd-cup-2010-student-performance-evaluation. Accessed: 2010. ↗
  23. Dohee Kim, Unggi Lee, Sookbun Lee, Jiyeong Bae, Taekyung Ahn, Jaekwon Park, Gunho Lee, and Hyeoncheol Kim. 2025. Es-kt-24: A multimodal knowledge tracing benchmark dataset with educational game playing video and synthetic text generation. In International Conference on Intelligent Tutoring Systems,… find
  24. Erica Kleinman, Murtuza Shergadwala, Zhaoqing Teng, Jennifer Villareale, Andy Bryant, Jichen Zhu, and Magy Seif El-Nasr. 2022. Analyzing students’ problem-solving sequences: A human-in-the-loop approach. Journal of learning analytics, 9(2):138– 160. 19260 10 find
  25. Xueyi Li, Youheng Bai, Teng Guo, Zitao Liu, Yaying Huang, Xiangyu Zhao, Feng Xia, Weiqi Luo, and Jian Weng. 2024. Enhancing length generalization for attention based knowledge tracing models with linear biases. In Proceedings of the thirty-third international joint conference on artificial intellige… find
  26. Guimei Liu, Huijing Zhan, and Jung-jae Kim. 2024. Question difficulty consistent knowledge tracing. In Proceedings of the ACM Web Conference 2024, pages 4239–4248. find
  27. Zitao Liu, Teng Guo, Qianru Liang, Mingliang Hou, Bojun Zhan, Jiliang Tang, Weiqi Luo, and Jian Weng. 2025. Deep learning based knowledge tracing: A review, a tool and empirical studies. IEEE Transactions on Knowledge and Data Engineering. find
  28. Zitao Liu, Qiongqiong Liu, Jiahao Chen, Shuyan Huang, and Weiqi Luo. 2023a. simplekt: a simple but tough-to-beat baseline for knowledge tracing. arXiv preprint arXiv:2302.06881. — [arXiv:2302.06881](https://arxiv.org/abs/2302.06881) find
  29. Zitao Liu, Qiongqiong Liu, Jiahao Chen, Shuyan Huang, Jiliang Tang, and Weiqi Luo. 2022. pykt: A python library to benchmark deep learning based knowledge tracing models. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track. find
  30. Zitao Liu, Qiongqiong Liu, Teng Guo, Jiahao Chen, Shuyan Huang, Xiangyu Zhao, Jiliang Tang, Weiqi Luo, and Jian Weng. 2023b. Xes3g5m: A knowledge tracing benchmark dataset with auxiliary information. Advances in Neural Information Processing Systems, 36:32958–32970. find
  31. Koki Nagatani, Qian Zhang, Masahiro Sato, Yan-Ying Chen, Francine Chen, and Tomoko Ohkuma. 2019. Augmenting knowledge tracing by considering forgetting behavior. In The world wide web conference, pages 3101–3107. find
  32. Hiromi Nakagawa, Yusuke Iwasawa, and Yutaka Matsuo. 2019. Graph-based knowledge tracing: modeling student proficiency using graph neural network. In IEEE/WIC/aCM international conference on web intelligence, pages 156–163. find
  33. OpenAI. 2025. Gpt-5 system card. https://cdn.openai.com/gpt-5-system-card.pdf. Accessed: 2025-09-24. ↗
  34. Shalini Pandey and George Karypis. 2019. A self-attentive model for knowledge tracing. arXiv preprint arXiv:1907.06837. find
  35. Zachary A Pardos, Ryan SJD Baker, Maria OCZ San Pedro, Sujith M Gowda, and Supreeth M Gowda. 2014. Affective states and state tests: Investigating how affect and engagement during the school year predict end-of-year learning outcomes. Journal of learning Analytics, 1(1):107–128. find
  36. Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J Guibas, and Jascha Sohl-Dickstein. 2015. Deep knowledge tracing. Advances in neural information processing systems, 28. find
  37. Stefan Slater and Ryan S. Baker. 2018. Degree of error in Bayesian knowledge tracing estimates from differences in sample sizes. Behaviormetrika, 45(2):475– 493. find
  38. Yu Su, Zeyu Cheng, Pengfei Luo, Jinze Wu, Lei Zhang, Qi Liu, and Shijin Wang. 2021. Time-and-concept enhanced deep multidimensional item response theory for interpretable knowledge tracing. Knowledge-Based Systems, 218:106819. find
  39. Peter Sullivan. 2011. Using the proficiencies from the australian mathematics curriculum to enrich mathematics teaching and assessment. Australian Education Review, (59). find
  40. Paul Tschisgale, Marcus Kubsch, Peter Wulff, Stefan Petersen, and Knut Neumann. 2025. Exploring the sequential structure of students’ physics problem-solving approaches using process mining and sequence analysis. Physical Review Physics Education Research, 21(1):010111. find
  41. Fidele Ukobizaba, Gabriel Nizeyimana, and Angel Mukuka. 2021. Assessment strategies for enhancing students’ mathematical problem-solving skills: A review of literature. Eurasia Journal of Mathematics, Science and Technology Education, 17(3). find
  42. Yutao Wang and Neil T Heffernan. 2012. Leveraging first response time into the knowledge tracing model. International Educational Data Mining Society. find
  43. Yang Yang, Jian Shen, Yanru Qu, Yunfei Liu, Kerong Wang, Yaoming Zhu, Weinan Zhang, and Yong Yu. 2020. Gikt: A graph-based interaction model for knowledge tracing. Preprint, arXiv:2009.05991. — [arXiv:2009.05991](https://arxiv.org/abs/2009.05991) find
  44. Chun-Kit Yeung. 2019. Deep-irt: Make deep learning based knowledge tracing explainable using item response theory. arXiv preprint arXiv:1904.11738. — [arXiv:1904.11738](https://arxiv.org/abs/1904.11738) find
  45. Chun-Kit Yeung and Dit-Yan Yeung. 2018. Addressing two problems in deep knowledge tracing via prediction-consistent regularization. In Proceedings of the fifth annual ACM conference on learning at scale, pages 1–10. find
  46. Jiani Zhang, Xingjian Shi, Irwin King, and Dit-Yan Yeung. 2017. Dynamic key-value memory networks for knowledge tracing. In Proceedings of the 26th international conference on World Wide Web, pages 765–774. find
  47. Yiyun Zhou, Wenkang Han, and Jingyuan Chen. 2025. Revisiting applicable and comprehensive knowledge tracing in large-scale data. arXiv preprint arXiv:2501.14256. 19261 11 — [arXiv:2501.14256](https://arxiv.org/abs/2501.14256) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/park-2026-tracing-mathematical-proficiency-through-problem/