Elamur Research⌕
☰
home / papers / 2026 / Tailoring Diagnostic Modeling to Individual Learners: Personalized Distractor Generation via MCTS-Guided Reasoning Reconstruction

Tailoring Diagnostic Modeling to Individual Learners: Personalized Distractor Generation via MCTS-Guided Reasoning Reconstruction

Paper · 2026 · Tao Wu, Jingyuan Chen, Wang Lin et al.

Summary

Multiple-choice distractors are usually designed for the whole student population, so they miss the specific reasoning errors of individual learners. The authors define a new task, Personalized Distractor Generation: build wrong options for one student from that student's past answers. The task is hard because each learner has few QA records and those records show only the chosen option, with no reasoning. The proposed framework needs no training and works in two stages. First, Monte Carlo Tree Search explores step-by-step faulty solution paths that could explain each past error. A self-evaluation signal favors paths that look like real student reasoning. The best paths and extracted concepts form a per-student misconception prototype. Second, for a new question, the system retrieves the relevant misconceptions, simulates the student's solution and outputs the resulting distractor. On the new Student_1361 dataset, the method beats existing generators and also handles group-level settings. A user study suggests personalized options expose misconceptions that standard distractors leave hidden.

Key points

  • Dataset Student_1361: 1,361 students across 6 subjects, from elementary to undergraduate level (e.g., mathematics, computer science), with about 80K QA records.
    • Each student has on average 49 history records and 9 test records.
    • Test items are only questions the student got wrong, and the student's actual chosen option is the ground truth.
    • Data come from public Eedi data (the Eedi_700 subset) plus an online QA platform.
    • Records are limited to sequences completed within one week, on the assumption that a student's cognitive state stays stable over that period.
  • Comparisons:
    • Baselines include group-level generators such as D-GEN, DiVERT and LOOKALIKE.
    • Human raters scored the method's distractors above those from CoT prompting across all tested models, consistent with the LLM-based evaluation.
  • User study (CEval questions):
    • Groups: a control group saw official distractors, while a treatment group saw one personalized distractor (generated with Claude-4-Sonnet) plus two frequent ones.
    • Design: every question also offered a "None of the above" option.
    • Metrics: Error Rate, Abstain Rate and PD Rate.
    • Result: the treatment group showed more errors, fewer abstentions and a substantial share of picks of the personalized option.
  • Limitations: personalization weakens when a student has very sparse history, in which case the system falls back to concept- or group-level distractors. The reconstructed reasoning paths are plausible approximations, not the students' true thought processes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hong Kong University of Science and TechnologyHong Kong · China170 works · 26,341 citations🎓Zhejiang UniversityHangzhou · China224 works · 13,587 citations🎓East China Normal UniversityShanghai · China69 works · 2,490 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksInformation Systems · 1,730 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,685 works

Cite

Cite as Tao Wu, Jingyuan Chen, Wang Lin, Jian Zhan, Mengze Li, Fangzhou Jin, Min Zhang, Kun Kuang and Fei Wu. 2026. Tailoring Diagnostic Modeling to Individual Learners: Personalized Distractor Generation via MCTS-Guided Reasoning Reconstruction. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.983

@inproceedings{wu2026tailoring,
  title = {{Tailoring Diagnostic Modeling to Individual Learners: Personalized Distractor Generation via MCTS-Guided Reasoning Reconstruction}},
  author = {Tao Wu, Jingyuan Chen, Wang Lin, Jian Zhan, Mengze Li, Fangzhou Jin, Min Zhang, Kun Kuang and Fei Wu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.983},
  url = {https://doi.org/10.18653/v1/2026.acl-long.983}
}

References

58 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. 2024. Large language models for mathematical reasoning: Progresses and challenges. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024: Student Research Works…
  2. Elaf Alhazmi, Quan Sheng, Wei Emma Zhang, Munazza Zaib, and Ahoud Alhazmi. 2024. Distractor generation in multiple-choice tasks: A survey of methods, datasets, and evaluation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, Novem… find
  3. Suyeong An, Junghoon Kim, Minsam Kim, and Juneyoung Park. 2022. No task left behind: Multi-task learning of knowledge tracing and option tracing for better student assessment. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications o… find
  4. Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. find
  5. Luca Benedetto, Giovanni Aradelli, Antonia Donvito, Alberto Lucchetti, Andrea Cappelli, and Paula Buttery. 2024. Using llms to simulate students’ responses to exam questions. In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pages 11… find
  6. Giorgio Biancini, Alessio Ferrato, and Carla Limongelli. 2024. Multiple-choice question generation using large language models: Methodology and educator insights. In Adjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization, UMAP Adjunct 2024, Cagliari, Italy, J… find
  7. Semere Kiros Bitew, Johannes Deleu, Chris Develder, and Thomas Demeester. 2023. Distractor generation for multiple-choice questions with predictive prompting and large language models. In Machine Learning and Principles and Practice of Knowledge Discovery in Databases - International Workshops of EC… find
  8. Grace Byun and Jinho D. Choi. 2025. D-GEN: automatic distractor generation and evaluation for reliable assessment of generative models. In Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 - August 1, 2025, pages 3316–3349. Association for Computational Li… find
  9. Devrim Çavusoglu, Seçil Sen, and Ulas Sert. 2024. Dis-gem: Distractor generation for multiple choice questions with span masking. In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pages 9714–9732. Association for Computational Lingui… find
  10. Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. 2024a. Alphamath almost zero: Process supervision without process. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024…
  11. Jingyuan Chen, Tao Wu, Wei Ji, and Fei Wu. 2024b. Wisdombot: Tuning large language models with artificial intelligence knowledge. Frontiers of Digital Education, 1(2):159–170. find
  12. DeepSeek-AI. 2024. Deepseek-v3 technical report. CoRR, abs/2412.19437. find
  13. Zhiang Dong, Jingyuan Chen, and Fei Wu. 2025. Knowledge is power: Harnessing large language models for enhanced cognitive diagnosis. In AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, February 25 - March 4, 2025, Philadelphia, PA, USA, pages 164–172. AAAI Press. find
  14. Wanyong Feng, Jaewook Lee, Hunter McNichols, Alexander Scarlatos, Digory Smith, Simon Woodhead, Nancy Otero Ornelas, and Andrew S. Lan. 2024. Exploring automated distractor generation for math multiple-choice questions via large language models. In Findings of the Association for Computational Lingu… find
  15. Nigel Fernandez, Alexander Scarlatos, Wanyong Feng, Simon Woodhead, and Andrew S. Lan. 2024. Divert: Distractor generation with variational errors represented as text for math multiple-choice questions. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP… find
  16. Aritra Ghosh, Jay Raspat, and Andrew S. Lan. 2021. Option tracing: Beyond correctness analysis in knowledge tracing. In Artificial Intelligence in Education - 22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14-18, 2021, Proceedings, Part I, Lecture Notes in Computer Science,… find
  17. Ze Han, Yu Su, Qi Mo, and Xinyu Nie. 2025. Neural network student cognitive diagnosis model based on multi-relational graph. In Proceedings of the 2025 2nd International Conference on Generative Artificial Intelligence and Information Security, GAIIS 2025, Hangzhou, China, February 21-23, 2025, page… find
  18. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
  19. Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Jiayi Lei, Yao Fu, Maosong Sun, and Junxian He. 2023. C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models. In Advances in Neural Information Proce…
  20. Zihan Huang, Tao Wu, Wang Lin, Shengyu Zhang, Jingyuan Chen, and Fei Wu. 2025. Autogeo: Automating geometric image dataset creation for enhanced geometry understanding. IEEE Trans. Multim., 27:3105–3116. find
  21. Hugging Face. 2025. Math-Verify: A robust mathematical expression evaluation system. https://github.com/huggingface/Math-Verify. Accessed: 2025-05-15. ↗
  22. Paul Ngee Kiong Lau, Sie-Hoe Lau, Kian Sam Hong, and Hasbee Usop. 2011. Guessing, partial knowledge, and misconceptions in multiple-choice tests. J. Educ. Technol. Soc., 14(4):99–110. find
  23. Yooseop Lee, Suin Kim, and Yohan Jo. 2025. Generating plausible distractors for multiple-choice questions via student choice prediction. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July 27 - August 1,… find
  24. Ang Li, Yiquan Wu, Yinghao Hu, Lizhi Qing, Shihang Wang, Chengyuan Liu, Tao Wu, Adam Jatowt, Ming Cai, Fei Wu, and Kun Kuang. 2025. Coevo: Co-evolution of LLM and retrieval model for domain-specific information retrieval. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language… find
  25. Mengze Li, Kairong Han, Jiahe Xu, Yueying Li, Tao Wu, Zhou Zhao, Jiaxu Miao, Shengyu Zhang, and Jingyuan Chen. 2024. Cross-modal observation hypothesis inference. In Proceedings of the 32nd ACM International Conference on Multimedia, MM 2024, Melbourne, VIC, Australia, 28 October 2024 - 1 November 2… find
  26. Chen Liang, Xiao Yang, Neisarg Dave, Drew Wham, Bart Pursel, and C. Lee Giles. 2018. Distractor generation for multiple choice questions using learning to rank. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications@NAACL-HLT 2018, New Orleans, LA, U… find
  27. Zhenghao Lin, Zihao Tang, Xiao Liu, Yeyun Gong, Yi Cheng, Qi Chen, Hang Li, Ying Xin, Ziyue Yang, Kailai Yang, Yu Yan, Xiao Liang, Shuai Lu, Yiming Huang, Zheheng Luo, Lei Qu, Xuan Feng, Yaoxiang Wang, Yuqing Xia, Feiyang Chen, Yuting Jiang, Yasen Hu, Hao Ni, Binyang Li, Guoshuai Zhao, Jui-Hao Chian… — [arXiv:2501.13629](https://arxiv.org/abs/2501.13629) find
  28. Ali Malik, Mike Wu, Vrinda Vasavada, Jinpeng Song, Madison Coots, John Mitchell, Noah D. Goodman, and Chris Piech. 2021. Generative grading: Near human-level accuracy for automated feedback on richly structured problems. In Proceedings of the 14th International Conference on Educational Data Mining,… find
  29. Sérgio Silva Mucciaccia, Thiago Meireles Paixão, Filipe Wall Mutz, Claudine Santos Badue, Alberto Ferreira de Souza, and Thiago Oliveira-Santos. 2025. Automatic multiple-choice question generation and evaluation systems based on LLM: A study case with university resolutions. In Proceedings of the 31… find
  30. Bang Nguyen, Tingting Du, Mengxia Yu, Lawrence An-grave, and Meng Jiang. 2025. QG-SMS: enhancing test item analysis via student modeling and simulation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July… find
  31. OpenAI. 2023. GPT-4 technical report. abs/2303.08774.
  32. CoRR, Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe.… — Training language models to follow instructions with human feedback
  33. Nisarg Parikh, Nigel Fernandez, Alexander Scarlatos, Simon Woodhead, and Andrew S. Lan. 2025. Lookalike: Consistent distractor generation in math mcqs. CoRR, abs/2505.01903. find
  34. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Sy…
  35. Alex Reinhart, Ciaran Evans, Amanda Luby, Josue Orellana, Mikaela Meyer, Jerzy Wieczorek, Peter Elliott, Philipp Burckhardt, and Rebecca Nugent. 2022. Think-aloud interviews: A tool for exploring student statistical reasoning. Journal of Statistics and Data Science Education, 30(2):100–113. find
  36. Alexander Scarlatos, Wanyong Feng, Andrew S. Lan, Simon Woodhead, and Digory Smith. 2024. Improving automated distractor generation for math multiple-choice questions with overgenerate-and-rank. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications, BEA 2… find
  37. Shashank Sonkar, Xinghe Chen, Naiming Liu, Richard G. Baraniuk, and Mrinmaya Sachan. 2024. Llm-based cognitive models of students with misconceptions. CoRR, abs/2410.12294. — [arXiv:2410.12294](https://arxiv.org/abs/2410.12294) find
  38. KV Aditya Srivatsa, Kaushal Kumar Maurya, and Ekaterina Kochmar. 2025. Can llms reliably simulate real students’ abilities in mathematics and reading comprehension? CoRR, abs/2507.08232. find
  39. Xinjie Sun, Qi Liu, Kai Zhang, Shuanghong Shen, Fei Wang, Yan Zhuang, Zheng Zhang, Weiyin Gong, Shijin Wang, Lina Yang, and Xingying Huo. 2025. HCD: A hierarchy constraint-aware neural cognitive diagnosis framework. Neural Networks, 190:107668. find
  40. Zihao Tang, Zheqi Lv, Shengyu Zhang, Fei Wu, and Kun Kuang. 2024. Modelgpt: Unleashing llm’s capabilities for tailored model generation. CoRR, abs/2402.12408. — [arXiv:2402.12408](https://arxiv.org/abs/2402.12408) find
  41. Zihao Tang, Xin Yu, Ziyu Xiao, Zengxuan Wen, Zelin Li, Jiaxi Zhou, Hualei Wang, Haohua Wang, Haizhen Huang, Weiwei Deng, Feng Sun, and Qi Zhang. 2026. Mnemis: Dual-route retrieval on hierarchical graphs for long-term LLM memory. CoRR, abs/2602.15313.
  42. Llama Team. 2024. The llama 3 herd of models. CoRR, abs/2407.21783.
  43. Fei Wang, Weibo Gao, Qi Liu, Jiatong Li, Guanhao Zhao, Zheng Zhang, Zhenya Huang, Mengxiao Zhu, Shijin Wang, Wei Tong, and Enhong Chen. 2024a. A survey of models for cognitive diagnosis: New developments and future directions. CoRR, abs/2407.05458. — [arXiv:2407.05458](https://arxiv.org/abs/2407.05458) find
  44. Qingsong Wang, Tao Wu, Wang Lin, Yueying Feng, Gongsheng Yuan, Chang Yao, and Jingyuan Chen. 2025a. Cognitive-level adaptive generation via capability-aware retrieval and style adaptation. In Findings of the Association for Computational Linguistics: EMNLP 2025, Suzhou, China, November 4-9, 2025, pa… find
  45. Rose E. Wang, Qingyang Zhang, Carly Robinson, Susanna Loeb, and Dorottya Demszky. 2024b. Bridging the novice-expert gap via models of decision-making: A case study on remediating math mistakes. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational L… find
  46. Shulei Wang, Wang Lin, Hai Huang, Hanting Wang, Sihang Cai, WenKang Han, Tao Jin, Jingyuan Chen, Jiacheng Sun, Jieming Zhu, and Zhou Zhao. 2025b. Towards transformer-based aligned generation with self-coherence guidance. CoRR, abs/2503.17675. — [arXiv:2503.17675](https://arxiv.org/abs/2503.17675) find
  47. Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2021. Denoising implicit feedback for recommendation. In WSDM ’21, The Fourteenth ACM International Conference on Web Search and Data Mining, Virtual Event, Israel, March 8-12, 2021, pages 373–381. ACM. find
  48. Zichao Wang, Angus Lamb, Evgeny Saveliev, Pashmina Cameron, Yordan Zaykov, José Miguel Hernández-Lobato, Richard E. Turner, Richard G. Baraniuk, Craig Barton, Simon Peyton Jones, Simon Woodhead, and Cheng Zhang. 2020. Results and insights from diagnostic questions: The neurips 2020 education challen… find
  49. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processi…
  50. Siyu Wu, Yang Cao, Jiajun Cui, Runze Li, Hong Qian, Bo Jiang, and Wei Zhang. 2024a. A comprehensive exploration of personalized learning in smart education: From student modeling to personalized recommendations. CoRR, abs/2402.01666. — [arXiv:2402.01666](https://arxiv.org/abs/2402.01666) find
  51. Tao Wu, Jingyuan Chen, Wang Lin, Mengze Li, Yumeng Zhu, Ang Li, Kun Kuang, and Fei Wu. 2025. Embracing imperfection: Simulating students with diverse cognitive levels using llm-based agents. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa…
  52. Tao Wu, Mengze Li, Jingyuan Chen, Wei Ji, Wang Lin, Jinyang Gao, Kun Kuang, Zhou Zhao, and Fei Wu. 2024b. Semantic alignment for multimodal large language models. In Proceedings of the 32nd ACM International Conference on Multimedia, MM 2024, Melbourne, VIC, Australia, 28 October 2024 - 1 November 2… find
  53. Weicai Yan, Wang Lin, Zirun Guo, Ye Wang, Fangming Feng, Xiaoda Yang, Zehan Wang, and Tao Jin. 2025. Diff-prompt: Diffusion-driven prompt generator with mask supervision. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. find
  54. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023,… find
  55. Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. 2024a. Rest-mcts*: LLM self-training via process reward guided tree search. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver,…
  56. Di Zhang, Xiaoshui Huang, Dongzhan Zhou, Yuqiang Li, and Wanli Ouyang. 2024b. Accessing GPT-4 level mathematical olympiad solutions via monte carlo tree self-refine with llama-3 8b. CoRR, abs/2406.07394. find
  57. Yumeng Zhu, Caifeng Zhu, Tao Wu, Shulei Wang, Yiyun Zhou, Jingyuan Chen, Fei Wu, and Yan Li. 2025. Impact of assignment completion assisted by large language model-based chatbot on middle school students’ learning. Educ. Inf. Technol., 30(2):2429– 2461. find
  58. In this appendix, we present the following content: A Detailed Comparison Between Group-level and Personalized Distractor Generation 14 B Dataset Details B.1 Dataset Statistics and Sample Illustrations . . . . . . . . . . . . . . B.2 Invalid Data Criteria . . . . . . . C Method Details C.1 Pseudocod… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2026-tailoring-diagnostic-modeling-individual-learners/