Elamur Research⌕
☰
home / papers / 2026 / Towards Pedagogically Aligned LLM Tutors for Math Mistake Remediation

Towards Pedagogically Aligned LLM Tutors for Math Mistake Remediation

Paper · 2026 · Kseniia Petukhova, Tien Dat Nguyen and Ekaterina Kochmar

Summary

LLM tutors often fail at good teaching practice. A common failure is handing students the final answer instead of helping them find and fix their own errors. This paper aligns open models for math mistake remediation with a two-stage pipeline. First, supervised fine-tuning (SFT) on tutoring dialogs. Second, Direct Preference Optimization (DPO) on synthetic preference pairs. To build the pairs, the authors treat LLM-written solutions from MR-GSM8K and PRM800K as student work. GPT-5 then writes good tutor replies, and deliberately weakened versions are created along pedagogical aspects such as scaffolding and factuality. Four input settings (V1–V4) test whether giving the model a correctness flag and/or the gold solution helps. Extra information, especially both together, clearly raises factual accuracy. The authors take this as support for modular tutoring systems, where solving, checking and feedback are handled by separate components. The best model, Qwen3-8B trained with DPO in V4, is competitive with GPT-5 in human judgments and was even preferred over it. The study also shows that automatic tutoring evaluators often disagree with each other. Code, data and models are openly released.

Key points

  • SFT data: MathDial (2,861 conversations) and SocraTeach (~35K), split into tutor turns, plus preferred synthetic responses. DPO pairs: built from MR-GSM8K and PRM800K, contrasting a GPT-5 reply with degraded variants.
  • Input configurations:
    • V1: dialog context only
    • V2: context plus correctness flag
    • V3: context plus gold solution
    • V4: context plus both
  • Evaluation:
    • Automatic: GPT-5 as judge and the LoMTL model from AITutor-EvalKit, run on 1,000 test samples.
    • Human: 105 response pairs, four annotators.
  • GPT-4.1-nano results: The DPO variant scored best on pedagogical dimensions according to LoMTL. However, GPT-5 judged its factual accuracy lower than the SFT variant's.
  • Limitations: Learning outcomes with real students were not measured. The synthetic data comes from a single model family (GPT-5/GPT-4.1). Resources are available at github.com/Kpetyxova/towards-aligned-math-tutor.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Mohamed bin Zayed University of Artificial IntelligenceAbu Dhabi · United Arab Emirates145 works · 6,810 citations🎓University of VictoriaVictoria · Canada4 works · 1,006 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksSocial Sciences · 2,009 worksInformation Systems · 1,888 worksDemography · 11 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 4,145 works

Cite

Cite as Kseniia Petukhova, Tien Dat Nguyen and Ekaterina Kochmar. 2026. Towards Pedagogically Aligned LLM Tutors for Math Mistake Remediation. Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.bea-1.10 · arXiv 2606.21502

@inproceedings{petukhova2026pedagogically,
  title = {{Towards Pedagogically Aligned LLM Tutors for Math Mistake Remediation}},
  author = {Kseniia Petukhova, Tien Dat Nguyen and Ekaterina Kochmar},
  booktitle = {Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.bea-1.10},
  eprint = {2606.21502},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2026.bea-1.10}
}

References

36 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot, Daniel Guo, Daniele Calandriello, Michal Valko, and Rémi Munos. 2023. A General Theoretical Paradigm to Understand Learning from Human Preferences. Preprint, arXiv:2310.12036.
  3. Benjamin S Bloom. 1984. The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational researcher, 13(6):4–16. find
  4. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874.
  5. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. LoRA: Low-Rank Adaptation of Large Language Models. Preprint, arXiv:2106.09685.
  6. Irina Jurenka, Markus Kunesch, Kevin R McKee, Daniel Gillick, Shaojian Zhu, Sara Wiltberger, Shubham Milind Phal, Katherine Hermann, Daniel Kasenberg, Avishkar Bhoopchand, and 1 others. 2024. Towards responsible development of generative AI for education: An evaluation-driven approach. arXiv preprin…
  7. Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. 2024. Prometheus 2: An open source language model specialized in evaluating other language models. In Proceedings of the 2024 Conference on Empirical Metho…
  8. Timothy A Carey and Richard J Mullan. 2004. What is Socratic questioning? Psychotherapy: theory, research, practice, training, 41(3):217. find
  9. Ekaterina Kochmar, Kaushal Kumar Maurya, Kseniia Petukhova, KV Aditya Srivatsa, Anaïs Tack, and Justin Vasselli. 2025. Findings of the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors. Preprint, arXiv:2507.10579.
  10. David Dinucu-Jianu, Jakub Macina, Nico Daheim, Ido Hakimi, Iryna Gurevych, and Mrinmaya Sachan. 2025. From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning. Preprint, arXiv:2505.15607.
  11. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199– 22213.
  12. Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. KTO: Model Alignment as Prospect Theoretic Optimization. Preprint, arXiv:2402.01306. find
  13. Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty, and Jimmy Xiangji Huang. 2023. A systematic study and comprehensive evaluation of ChatGPT on benchmark datasets. arXiv preprint arXiv:2305.18486.
  14. Scott Freeman, Sarah L Eddy, Miles McDonough, Michelle K Smith, Nnadozie Okoroafor, Hannah Jordt, and Mary Pat Wenderoth. 2014. Active learning increases student performance in science, engineering, and mathematics. Proceedings of the national academy of sciences, 111(23):8410–8415. find
  15. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. In The Twelfth International Conference on Learning Representations.
  16. Wensheng Gan, Zhenlian Qi, Jiayang Wu, and Jerry Chun-Wei Lin. 2023. Large language models in education: Vision and opportunities. In 2023 IEEE international conference on big data (BigData), pages 4776–4785. IEEE. find
  17. Jiayu Liu, Zhenya Huang, Tong Xiao, Jing Sha, Jinze Wu, Qi Liu, Shijin Wang, and Enhong Chen. 2024a. SocraticLM: Exploring socratic personalized teaching with large language models. Advances in Neural Information Processing Systems, 37:85693–85721. find
  18. Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, and 1 others. 2024. CChat-GLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. arXiv preprint arXiv:2406.12793.
  19. Rongxin Liu, Carter Zenke, Charlie Liu, Andrew Holmes, Patrick Thornton, and David J Malan. 2024b. Teaching CS50 with AI: leveraging generative artificial intelligence in computer science education. In Proceedings of the 55th ACM technical symposium on computer science education V. 1, pages 750–756. find
  20. Jakub Macina, Nico Daheim, Sankalan Pal Chowdhury, Tanmay Sinha, Manu Kapur, Iryna Gurevych, and Mrinmaya Sachan. 2023. MathDial: A dialogue tutoring dataset with rich pedagogical properties grounded in math reasoning problems. arXiv preprint arXiv:2305.14536.
  21. Kaushal Kumar Maurya, KV Aditya Srivatsa, Kseniia Petukhova, and Ekaterina Kochmar. 2025. Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutors. Preprint, arXiv:2412.09416.
  22. Hunter McNichols, Wanyong Feng, Jaewook Lee, Alexander Scarlatos, Digory Smith, Simon Woodhead, and Andrew Lan. 2023. Automated distractor and feedback generation for math multiple-choice questions via in-context learning. arXiv preprint arXiv:2308.03234.
  23. Ethan Mollick and Lilach Mollick. 2024. Instructors as innovators: A future-focused approach to new AI learning opportunities, with prompts. arXiv preprint arXiv:2407.05181. — [arXiv:2407.05181](https://arxiv.org/abs/2407.05181) find
  24. Rose E Wang, Qingyang Zhang, Carly Robinson, Susanna Loeb, and Dorottya Demszky. 2023. Bridging the novice-expert gap via models of decision-making: A case study on remediating math mistakes. arXiv preprint arXiv:2310.10648. find
  25. Shen Wang, Tianlong Xu, Hang Li, Chaoli Zhang, Joleen Liang, Jiliang Tang, Philip S Yu, and Qingsong Wen. 2024. Large language models for education: A survey and outlook. arXiv preprint arXiv:2403.18105.
  26. Sebastian Wollny, Jan Schneider, Daniele Di Mitri, Joshua Weidlich, Marc Rittberger, and Hendrik Drachsler. 2021. Are we there yet? – A systematic literature review on chatbots in education. Frontiers in artificial intelligence, 4:654924. find
  27. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  28. Numaan Naeem, Kaushal Kumar Maurya, Kseniia Petukhova, and Ekaterina Kochmar. 2025. AITutor-EvalKit: Exploring the Capabilities of AI Tutors. arXiv preprint arXiv:2512.03688.
  29. Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Shaochen Zhong, Bing Yin, and Xia Hu. 2024. Harnessing the power of llms in practice: A survey on chatgpt and beyond. ACM Transactions on Knowledge Discovery from Data, 18(6):1–32.
  30. Benjamin D Nye, Arthur C Graesser, and Xiangen Hu. 2014. AutoTutor and family: A review of 17 years of natural language tutoring. International Journal of Artificial Intelligence in Education, 24:427–469. find
  31. Zhongshen Zeng, Pengguang Chen, Shu Liu, Haiyun Jiang, and Jiaya Jia. 2023. MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation. arXiv preprint arXiv:2312.17080.
  32. Sankalan Pal Chowdhury, Vilém Zouhar, and Mrinmaya Sachan. 2024. AutoTutor meets large language models: A language model tutor with rich pedagogy and guardrails. In Proceedings of the Eleventh ACM Conference on Learning@ Scale, pages 5–15.
  33. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  34. Jiaming Shen, Ran Xu, Yennie Jun, Zhen Qin, Tianqi Liu, Carl Yang, Yi Liang, Simon Baumgartner, and Michael Bendersky. 2024. Boosting reward model with preference-conditional multi-aspect synthetic data generation. arXiv preprint arXiv:2407.16008.
  35. Shashank Sonkar, Kangqi Ni, Sapana Chaudhary, and Richard G. Baraniuk. 2024. Pedagogical Alignment of Large Language Models. Preprint, arXiv:2402.05000.
  36. LearnLM Team, Abhinit Modi, Aditya Srikanth Veerubhotla, Aliya Rysbek, Andrea Huber, Brett Wiltshire, Brian Veprek, Daniel Gillick, Daniel Kasenberg, Derek Ahmed, and 1 others. 2024. LearnLM: Improving Gemini for Learning. arXiv preprint arXiv:2412.16429.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/petukhova-2026-pedagogically-aligned-llm-tutors-math/