Elamur Research⌕
☰
home / papers / 2026 / MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring

MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring

Paper · 2026 · Tengchao Yang, Sichen Guo, Mengzhao Jia et al.

Summary

MMTutorBench addresses a gap in multimodal evaluation. Existing math benchmarks test whether models can read handwriting or solve problems. They do not test whether a model can act as a tutor: working out where a student is stuck, naming the relevant idea, and suggesting only the next step.

The authors built 770 tutoring scenarios from frames of 292 instructional YouTube videos. Gemini-2.5-Pro first located pedagogically critical "key-steps", then human annotators checked them and reconstructed the surrounding context. Each item pairs a student's handwritten work with a help-seeking question. Responses are split into three tasks inspired by Pólya: Insight Discovery, Operation Formulation, and Operation Execution. A rubric-based LLM-as-a-Judge scores each response on six dimensions using rubrics written for that specific problem.

Across 12 MLLMs, proprietary models outperform open-source ones, and all fall well short of human tutors. Feeding OCR text instead of images lowers quality, while few-shot prompting helps little. The benchmark shows that strong problem solving does not guarantee good tutoring.

Key points

  • The dataset has 770 problems and 1,414 images, drawn from 292 videos on 14 math YouTube channels. It spans junior high school to university level.
  • Responses are judged on six dimensions: Insight and Coherence, plus how well the operation is formulated (Operation Formulation) and carried out (Operation Execution), whether the answer keeps to an appropriate scope (Solution Scope Control), and how concise it is (Brevity).
  • Gemini-2.5-Pro ranks first (4.77 total), followed by GPT-5 (4.41). Open models such as InternVL3.5-8B (3.19) and MiMo-VL-7B-RL (2.75) trail well behind.
  • The rubric judge agrees with human experts more closely than any baseline. It reaches Pearson r = 0.725, versus 0.625 for a standard LLM judge and 0.386 for ROUGE-L. Its scores also track closely across two judges, GPT-o4-mini and Qwen3-30B-A3B-Instruct-2507, with correlation above 0.98.
  • Gemini-2.5-Pro's most common failure is scope control: 31.04% of its answers score zero there. Its outputs are almost always coherent, with only 2.73% scoring zero on Coherence, so answers tend to read well while being substantively flawed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Fudan UniversityShanghai · China202 works · 22,509 citations🎓University of Notre DameNotre Dame · United States59 works · 37,127 citations🎓Tongji UniversityShanghai · China33 works · 2,993 citations🎓Nanjing University of Posts and TelecommunicationsNanjing · China4 works · 22 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Vision and Pattern Recognition · 3,486 worksInformation Systems · 1,730 works

🧭 Topics 2Research subjects and languages the work is about.Evaluation & benchmarks · 1,341 worksVision & language · 811 works

Cite

Cite as Tengchao Yang, Sichen Guo, Mengzhao Jia, Jiaming Su, Yuanyang Liu, Zhihan Zhang and Meng Jiang. 2026. MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1068

@inproceedings{yang2026mmtutorbench,
  title = {{MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring}},
  author = {Tengchao Yang, Sichen Guo, Mengzhao Jia, Jiaming Su, Yuanyang Liu, Zhihan Zhang and Meng Jiang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1068},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1068}
}

References

36 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mark H Ashcraft. 2002. Math anxiety: Personal, educational, and cognitive consequences. Current directions in psychological science, 11(5):181–185. find
  2. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Ming-Hsuan Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025. Qwen2.5-vl technical report. CoRR, abs/2502.13923…
  3. Connie Barroso, Colleen M Ganley, Amanda L McGraw, Elyssa A Geer, Sara A Hart, and Mia C Daucourt. 2021. A meta-analysis of the relation between math anxiety and math achievement. Psychological bulletin, 147(2):134. find
  4. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit S. Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 81 othe… — Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
  5. Ling Fu, Zhebin Kuang, Jiajun Song, Mingxin Huang, Biao Yang, Yuzhe Li, Linghao Zhu, Qidi Luo, Xinyu Wang, Hao Lu, and 1 others. 2024. Ocrbench v2: An improved benchmark for evaluating large multimodal models on visual text localization and reasoning. arXiv preprint arXiv:2501.00321. — [arXiv:2501.00321](https://arxiv.org/abs/2501.00321) find
  6. Philippe Gervais, Anastasiia Fadeeva, and Andrii Maksai. 2025. Mathwriting: A dataset for handwritten mathematical expression recognition. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 5459–5469. find
  7. Adit Gupta, Jennifer Reddig, Tommaso Calo, Daniel Weitekamp, and Christopher J. MacLellan. 2025. Beyond final answers: Evaluating large language models for math tutoring. Preprint, arXiv:2503.16460. — [arXiv:2503.16460](https://arxiv.org/abs/2503.16460) find
  8. Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang, Yan Wang, Yean Cheng, Zehai He, Zhe Su, Zhen Yang, Ziyang Pan, Aohan Zeng, and 58 others. 2025. Glm-4.1v-thinking: Towards versatile multimodal reasoning wit…
  9. Fareya Ikram, Alexander Scarlatos, and Andrew Lan. 2025. Exploring llms for predicting tutor strategy and student outcomes in dialogues. Preprint, arXiv:2507.06910. find
  10. Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean-Bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, Gaël Liu,… find
  11. Alexander Scarlatos, Naiming Liu, Jaewook Lee, Richard Baraniuk, and Andrew Lan. 2025. Training LLM-Based Tutors to Improve Student Learning Outcomes in Dialogues, page 251–266. Springer Nature Switzerland. find
  12. Debanjana Kar, Leopold Böss, Dacia Braca, Sebastian Maximilian Dennerlein, Nina Christine Hubig, Philipp Wintersberger, and Yufang Hou. 2025. Math-buddy: A multimodal system for affective math tutoring. arXiv preprint arXiv:2508.19993. find
  13. Alan H. Schoenfeld. 1987. Pólya, problem solving, and education. Mathematics Magazine, 60(5):283–291. find
  14. Seungone Kim, Juyoung Suk, Ji Yong Cho, Shayne Longpre, Chaeeun Kim, Dongkeun Yoon, Guijin Son, Yejin Cho, Sheikh Shafayat, Jinheon Baek, Sue Hyun Park, Hyeonbin Hwang, Jinkyung Jo, Hyowon Cho, Haebin Shin, Seongyun Lee, Hanseok Oh, Noah Lee, Namgyu Ho, and 13 others. 2025. The biggen bench: A princ… find
  15. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2024. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations (ICLR).
  16. Jakub Macina, Nico Daheim, Ido Hakimi, Manu Kapur, Iryna Gurevych, and Mrinmaya Sachan. 2025. Mathtutorbench: A benchmark for measuring open-ended pedagogical capabilities of LLM tutors. CoRR, abs/2502.18940. find
  17. Dajung Diana Oh, Michael M Barger, and Eva M Pomerantz. 2022. Parents’ math anxiety and their controlling and autonomy-supportive involvement in children’s math learning: Implications for children’s math achievement. Developmental Psychology, 58(11):2158. find
  18. OpenAI. 2024. Hello gpt-4o. News announcement by OpenAI. find
  19. OpenAI. 2025a. Chatgpt study mode. https://openai.com/zh-Hans-CN/index/ chatgpt-study-mode/. Accessed: 2025-10-05. ↗
  20. OpenAI. 2025b. Gpt-5 is here. Technical report, OpenAI. News announcement by OpenAI. find
  21. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  22. Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, and 1 others. 2024. We-math: Does your large multimodal model achieve human-like mathematical reasoning? arXiv preprint arXiv:2407.01284.
  23. Kai Sun, Yushi Bai, Ji Qi, Lei Hou, and Juanzi Li. 2024. Mm-math: Advancing multimodal math evaluation with process evaluation and fine-grained classification. arXiv preprint arXiv:2404.05091. find
  24. MiniCPM Team. 2025. MiniCPM4: Ultra-efficient llms on end devices.
  25. Danielle R. Thomas, Conrad Borchers, Jionghao Lin, Sanjit Kakarla, Shambhavi Bhushan, Erin Gatz, Shivang Gupta, Ralph Abboud, and Kenneth R. Koedinger. 2025. Leveraging llms to assess tutor moves in real-life dialogues: A feasibility study. Preprint, arXiv:2506.17410. — [arXiv:2506.17410](https://arxiv.org/abs/2506.17410) find
  26. Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. 2024a. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems, 37:95095–95169.
  27. Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. 2024b. Measuring multimodal mathematical reasoning with math-vision dataset. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
  28. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, and 56 others. 2025. Internvl3.5: Advancing open-source multimodal…
  29. Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simon-celli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612. find
  30. Allan Wigfield and Judith L Meece. 1988. Math anxiety in elementary and secondary school students. Journal of educational Psychology, 80(2):210. find
  31. Bingquan Xia, Bowen Shen, Cici, Dawei Zhu, Di Zhang, Gang Wang, Hailin Zhang, Huaqiu Liu, Jiebao Xiao, Jinhao Dong, Liang Zhao, Peidian Li, Peng Wang, Shihua Yu, Shimao Chen, Weikun Wang, Wenhan Ma, Xiangwei Deng, Yi Huang, and 44 others. 2025. Mimo: Unlocking the reasoning potential of language mod… find
  32. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  33. Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Chen, Nitesh V Chawla, and Xiangliang Zhang. 2024. Justice or prejudice? quantifying biases in llm-as-a-judge. Preprint, arXiv:2410.02736.
  34. Ye Yuan, Xiao Liu, Wondimu Dikubab, Hui Liu, Zhilong Ji, Zhongqin Wu, and Xiang Bai. 2022. Syntax-aware network for handwritten mathematical expression recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4553–4562. find
  35. Liang Zhang and Edith Aurora Graf. 2025. Mathematical computation and reasoning errors by large language models. Preprint, arXiv:2508.09932. find
  36. Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, and 1 others. 2024. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? arXiv preprint arXiv:2403.14624.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2026-mmtutorbench-first-multimodal-benchmark-ai/