Elamur Research⌕
☰
home / papers / 2026 / One LLM Does Not Simulate All Students: Ability-Aware Student Simulation via Cognitive Diagnosis Guided LLM Assignment

One LLM Does Not Simulate All Students: Ability-Aware Student Simulation via Cognitive Diagnosis Guided LLM Assignment

Paper · 2026 · Huixing Que, Qi Liu, Weibo Gao et al.

Summary

LLM-based student simulators usually use one strong model to stand in for every learner. The authors show this creates an ability-dependent distortion. Powerful models tend to answer correctly where weak students would fail. Small models lack the knowledge to imitate strong students. So simulations inflate low performers and deflate high performers.

To fix this, the paper treats simulation as a model-assignment problem driven by cognitive alignment rather than task success. Neural Cognitive Diagnosis (NeuralCD) is trained on response logs from both real students and a heterogeneous pool of LLMs, which are handled as artificial learners. This places both groups in one concept-level mastery space. Each student is then paired with the LLM whose proficiency vector is most similar, and that model simulates the student's future answers.

Experiments, including an extra test on XES3G5M, show reduced ability bias and better balanced performance across proficiency levels than single-model baselines. The added overhead is small. The work matters for educational AI because it frames cognitive diagnosis as a principled tool for choosing which model should represent which learner.

Key points

  • The bias is measured with Acc+ and Acc− (accuracy on correct vs. incorrect student responses) and summarized by BAA. Stronger models raise Acc+ but lower Acc−.
  • The method has three stages:
    • collect responses from students and LLMs;
    • build NeuralCD profiles, which rely on a Q-matrix (MIRT is suggested when no Q-matrix exists);
    • assign each student the LLM that maximizes profile similarity.
  • On XES3G5M, a four-model pool was used, including Qwen2.5-7B-Instruct, Qwen2.5-14B-Instruct and DeepSeek-V3.2. The method still obtained the best BAA there, though not the best overall accuracy in the main results.
  • Cost stays low:
    • NeuralCD has only about 0.3M parameters;
    • LLM profiling is a one-time offline step;
    • API calls match those of single-model baselines, and routing weaker students to cheaper models can reduce spending.
  • Stated limitations:
    • no support for cold-start students without history;
    • a non-exhaustive LLM pool;
    • evaluation limited to answer correctness;
    • no comparison with fine-tuning.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Science and Technology of ChinaHefei · China108 works · 112,698 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksInformation Systems · 1,437 worksComputer Science Applications · 102 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Huixing Que, Qi Liu, Weibo Gao and Zhenya Huang. 2026. One LLM Does Not Simulate All Students: Ability-Aware Student Simulation via Cognitive Diagnosis Guided LLM Assignment. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.302

@inproceedings{que2026one,
  title = {{One LLM Does Not Simulate All Students: Ability-Aware Student Simulation via Cognitive Diagnosis Guided LLM Assignment}},
  author = {Huixing Que, Qi Liu, Weibo Gao and Zhenya Huang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.302},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.302}
}

References

53 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zahra Abbasiantaeb, Yifei Yuan, Evangelos Kanoulas, and Mohammad Aliannejadi. 2024. Let the llms talk: Simulating human-to-human conversational qa via zero-shot llm-to-llm interactions. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 8–17. find
  2. Ghodai Abdelrahman, Sherif Abdelfattah, Qing Wang, and Yu Lin. 2022. Dbe-kt22: A knowledge tracing dataset based on online student evaluation. arXiv preprint arXiv:2208.12651. — [arXiv:2208.12651](https://arxiv.org/abs/2208.12651) find
  3. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  4. Jasper Meynard Arana, Kristine Ann M Carandang, Ethan Robert Casin, Christian Alis, Daniel Stanley Tan, Erika Fille Legara, and Christopher Monterola. 2025. Foundations of peers: Assessing llm role performance in educational simulations. In Proceedings of the 63rd Annual Meeting of the Association f… find
  5. Shir Ashury Tahan, Ariel Gera, Benjamin Sznajder, Leshem Choshen, Liat Ein-Dor, and Eyal Shnarch. 2024. Label-efficient model selection for text generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8384–8402, Bangkok, T… find
  6. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609.
  7. Frank B Baker. 2001. The basics of item response theory. ERIC. find
  8. Matthew L Bernacki, Meghan J Greene, and Nikki G Lobczowski. 2021. A systematic review of research on personalized learning: Personalized by whom, to what, how, and for what purpose (s)? Educational Psychology Review, 33(4):1675–1715. find
  9. Saptarshi Bhowmik, Luke West, Alex Barrett, Nuodi Zhang, Chih-Pu Dai, Zlatko Sokolikj, Sherry Southerland, Xin Yuan, and Fengfeng Ke. 2024. Evaluation of an llm-powered student agent for teacher training. In European conference on technology enhanced learning, pages 68–74. Springer. find
  10. Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M Buhmann. 2010. The balanced accuracy and its posterior distribution. In 2010 20th international conference on pattern recognition, pages 3121–3124. IEEE. find
  11. R Philip Chalmers. 2012. mirt: A multidimensional item response theory package for the r environment. Journal of statistical Software, 48:1–29. find
  12. Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jinheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S Yu, and 1 others. 2025. Llm agents for education: Advances and applications. arXiv preprint arXiv:2503.11733. find
  13. Xiangxiang Dai, Jin Li, Xutong Liu, Anqi Yu, and John Lui. 2024. Cost-effective online multi-llm selection with versatile reward models. arXiv preprint arXiv:2405.16587. — [arXiv:2405.16587](https://arxiv.org/abs/2405.16587) find
  14. Jimmy De La Torre. 2009. Dina model and parameter estimation: A didactic. Journal of educational and behavioral statistics, 34(1):115–130. find
  15. Christine DeMars. 2010. Item response theory. Oxford University Press. find
  16. Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruhle, Laks VS Lakshmanan, and Ahmed Hassan Awadallah. 2024. Hybrid llm: Cost-efficient and quality-aware query routing. arXiv preprint arXiv:2404.14618. find
  17. Tao Feng, Yanzhen Shen, and Jiaxuan You. 2024. Graphrouter: A graph-based router for llm selections. arXiv preprint arXiv:2410.03834. — [arXiv:2410.03834](https://arxiv.org/abs/2410.03834) find
  18. Weibo Gao, Qi Liu, Zhenya Huang, Yu Yin, Haoyang Bi, Mu-Chun Wang, Jianhui Ma, Shijin Wang, and Yu Su. 2021. Rcd: Relation map driven cognitive diagnosis for intelligent education systems. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrie… find
  19. Weibo Gao, Qi Liu, Linan Yue, Fangzhou Yao, Rui Lv, Zheng Zhang, Hao Wang, and Zhenya Huang. 2025. Agent4edu: Generating learner response data by generative agents for intelligent education systems. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 23923–23932. find
  20. Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. 2024. Routerbench: A benchmark for multi-llm routing system. arXiv preprint arXiv:2403.12031. — [arXiv:2403.12031](https://arxiv.org/abs/2403.12031) find
  21. Maher Joe Khan Omar Jian. 2023. Personalized learning through ai. Advances in Engineering Innovation, 5:16–19. find
  22. Wittawat Jitkrittum, Harikrishna Narasimhan, Ankit Singh Rawat, Jeevesh Juneja, Congchao Wang, Zifeng Wang, Alec Go, Chen-Yu Lee, Pradeep Shenoy, Rina Panigrahy, and 1 others. 2025. Universal model routing for efficient llm inference. arXiv preprint arXiv:2502.08773. — [arXiv:2502.08773](https://arxiv.org/abs/2502.08773) find
  23. Jacqueline Leighton and Mark Gierl. 2007. Cognitive diagnostic assessment for education: Theory and applications. Cambridge University Press. find
  24. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. find
  25. Yingjie Liu, Tiancheng Zhang, Xuecen Wang, Ge Yu, and Tao Li. 2023a. New development of cognitive diagnosis models. Frontiers of Computer Science, 17(1):171604. find
  26. Zitao Liu, Qiongqiong Liu, Teng Guo, Jiahao Chen, Shuyan Huang, Xiangyu Zhao, Jiliang Tang, Weiqi Luo, and Jian Weng. 2023b. Xes3g5m: A knowledge tracing benchmark dataset with auxiliary information. Advances in Neural Information Processing Systems, 36:32958–32970. find
  27. Frederic M Lord. 2012. Applications of item response theory to practical testing problems. Routledge. find
  28. Xinyi Lu and Xu Wang. 2024. Generative students: Using llm-simulated student profiles to support question item evaluation. In Proceedings of the Eleventh ACM Conference on Learning@ Scale, pages 16–27. find
  29. Rui Lv, Qi Liu, Weibo Gao, Jiatong Li, Kai Zhang, and Shiwei Tong. 2025. Real: How can llms simulate the real teacher? retrieval-enhanced agent for adaptive learning. Thinking, 1:2. find
  30. Fanhang Man, Huandong Wang, Jianjie Fang, Zhaoyi Deng, Baining Zhao, Xinlei Chen, and Yong Li. 2025. Context-aware sentiment forecasting via LLM-based multi-perspective role-playing agents. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap… find
  31. Amogh Mannekote, Adam Davies, Jina Kang, and Kristy Elizabeth Boyer. 2025. Can llms reliably simulate human learner actions? a simulation authoring framework for open-ended learning environments. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 29044–29052. find
  32. Kai Mei, Wujiang Xu, Minghao Guo, Shuhang Lin, and Yongfeng Zhang. 2025. Omnirouter: Budget and performance controllable multi-llm routing. ACM SIGKDD Explorations Newsletter, 27(2):107–116. find
  33. Alexander Tobias Neumann, Yue Yin, Sulayman Sowe, Stefan Decker, and Matthias Jarke. 2024. An llm-driven chatbot in higher education for databases and information systems. IEEE Transactions on Education. find
  34. Pranoy Panda, Raghav Magazine, Chaitanya Devaguptapu, Sho Takemori, and Vishal Sharma. 2025. Adaptive llm routing under budget constraints. arXiv preprint arXiv:2508.21141. find
  35. Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22. find
  36. Mark D Reckase. 2009. Historical background for multidimensional item response theory (mirt). In Multidimensional item response theory, pages 57–77. Springer. find
  37. Atikah Shemshack and Jonathan Michael Spector. 2020. A systematic literature review of personalized learning terms. Smart learning environments, 7(1):33. find
  38. Wei Song, Zhenya Huang, Cheng Cheng, Weibo Gao, Bihan Xu, GuanHao Zhao, Fei Wang, and Runze Wu. 2025. Irt-router: Effective and interpretable multi-llm routing via item response theory. arXiv preprint arXiv:2506.01048. find
  39. Kikumi K Tatsuoka. 1983. Rule space: An approach for dealing with misconceptions based on item response theory. Journal of educational measurement, pages 345–354. find
  40. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
  41. Jonathan L Templin and Robert A Henson. 2006. Measurement of psychological disorders using cognitive diagnosis models. Psychological methods, 11(3):287. find
  42. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  43. Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang, Yu Yin, Shijin Wang, and Yu Su. 2022. Neuralcd: a general framework for cognitive diagnosis. IEEE Transactions on Knowledge and Data Engineering, 35(8):8312–8327. find
  44. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  45. Qingsong Wen, Jing Liang, Carles Sierra, Rose Luckin, Richard Tong, Zitao Liu, Peng Cui, and Jiliang Tang. 2024. Ai for education (ai4edu): Advancing personalized education with llm and adaptive learning. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages… find
  46. Tao Wu, Jingyuan Chen, Wang Lin, Mengze Li, Yumeng Zhu, Ang Li, Kun Kuang, and Fei Wu. 2025. Embracing imperfection: Simulating students with diverse cognitive levels using llm-based agents. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa…
  47. Yu Xia, Fang Kong, Tong Yu, Liya Guo, Ryan A Rossi, Sungchul Kim, and Shuai Li. 2024. Which llm to play? convergence-aware online model selection with time-increasing bandits. In Proceedings of the ACM Web Conference 2024, pages 4059–4070. find
  48. Songlin Xu, Hao-Ning Wen, Hongyi Pan, Dallas Dominguez, Dongyin Hu, and Xinyu Zhang. 2025. Classroom simulacra: Building contextual student generative agents in online education for learning behavioral simulation. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages… find
  49. Songlin Xu, Xinyu Zhang, and Lianhui Qin. 2024. Eduagent: Generative student agents in learning. arXiv preprint arXiv:2404.07963. — [arXiv:2404.07963](https://arxiv.org/abs/2404.07963) find
  50. Yizhe Yang, Palakorn Achananuparp, Heyan Huang, Jing Jiang, Nicholas Gabriel Lim, Cameron Tan Shi Ern, Phey Ling Kit, Jenny Giam Xiuhui, John Pinto, and Ee-Peng Lim. 2025. Consistent client simulation for motivational interviewing-based counseling. In Proceedings of the 63rd Annual Meeting of the As… find
  51. Zheyuan Zhang, Daniel Zhang-Li, Jifan Yu, Linlu Gong, Jinchang Zhou, Zhanxin Hao, Jianxiao Jiang, Jie Cao, Huiqin Liu, Zhiyuan Liu, and 1 others. 2025. Simulating classroom education with llm-empowered agents. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Associ…
  52. Zesen Zhao, Shuowei Jin, and Z Morley Mao. 2024. Eagle: Efficient training-free router for multi-llm inference. arXiv preprint arXiv:2409.15518. — [arXiv:2409.15518](https://arxiv.org/abs/2409.15518) find
  53. Longwei Zheng, Fei Jiang, Xiaoqing Gu, Yuanyuan Li, Gong Wang, and Haomin Zhang. 2025. Teaching via llm-enhanced simulations: Authenticity and barriers to suspension of disbelief. The Internet and Higher Education, 65:100990. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/que-2026-one-llm-does-not-simulate/