Elamur Research⌕
☰
home / papers / 2026 / MorphoBench: A Benchmark with Difficulty Adaptive to Model Reasoning

MorphoBench: A Benchmark with Difficulty Adaptive to Model Reasoning

Paper · 2026 · Xukai Wang, Xuanbo Liu, Mingrui Chen et al.

Summary

Static reasoning benchmarks quickly become too easy for frontier models. Many also reward recall of rare facts rather than logical inference. MorphoBench addresses this with a multidisciplinary benchmark whose difficulty can be tuned to the models being tested.

The authors pooled about 1,300 reasoning questions from three sources:

  • existing benchmarks (HLE, MME-Reasoning, HistBench),
  • Olympiad problems in mathematics, physics and chemistry,
  • template-generated tasks such as circuit black boxes, with answers checked by simulation.

They model problem solving as finding a path through a proof graph. Difficulty is then adjusted by altering key intermediate statements that models produce while reasoning. Supportive or misleading hints change the reasoning chain, perturbations of crucial cues make conditions harder to recognize, and parameters control the generated tasks. A reviewer-judge LLM agent scores each output on correctness, reasoning quality and hint-following.

Tests on models including GPT-5.1, Gemini-3-Pro and o3 show that accuracy shifts in the intended direction under these adjustments. Reasoning quality does not always track accuracy. The benchmark therefore supports fairer, more diagnostic evaluation as models improve.

Key points

  • The initial set, MORPHO-R(v0), contains 1,307 questions. Mathematics is the largest discipline (560, ~36%), followed by Engineering (439), Natural Sciences (250), Other (212) and Social Sciences (86).
  • On the hint-based splits, GPT-5.1 leads on R(Lite) and R(v0) with 55.28% and 50.77%. Gemini-3-Pro is best on the misleading-hint split R(Complex) with 42.96%.
  • Cue perturbation lowers accuracy for every model. Gemini-3-Pro tops the clean P(v0) split (51.47%), but o3 is most robust on P(Perturbed) (40.55%).
  • The generated circuit tasks in MORPHO-G have 10 difficulty levels. Gemini-2.5-Pro falls from 75.9% to 0–13% as levels rise, while o3 stays between 30% and 58% with no clear trend.
  • Code and data are available at github.com/OpenDCAI/MorphoBench.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China244 works · 39,359 citations🏛️Chinese Academy of SciencesBeijing · China194 works · 23,906 citations🔬Institute of AutomationBeijing · China70 works · 14,251 citations🎓Beihang UniversityBeijing · China65 works · 8,452 citations🎓University of Chinese Academy of SciencesBeijing · China147 works · 8,480 citations🎓University College of Applied ScienceGaza · Palestine3 works · 2 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksEvaluation & benchmarks · 1,220 works

Cite

Cite as Xukai Wang, Xuanbo Liu, Mingrui Chen, Haitian Zhong, Xuanlin Yang, Bohan Zeng, Jinbo Hu, Hao Liang, Junbo Niu, Xuchen Li, Ruitao Wu, Ruichuan An, Yang Shi, Liu Liu, Qiang Liu, Zhouchen Lin, Xu-Yao Zhang, Wentao Zhang and Bin Dong. 2026. MorphoBench: A Benchmark with Difficulty Adaptive to Model Reasoning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1171

@inproceedings{wang2026morphobench,
  title = {{MorphoBench: A Benchmark with Difficulty Adaptive to Model Reasoning}},
  author = {Xukai Wang, Xuanbo Liu, Mingrui Chen, Haitian Zhong, Xuanlin Yang, Bohan Zeng, Jinbo Hu, Hao Liang, Junbo Niu, Xuchen Li, Ruitao Wu, Ruichuan An, Yang Shi, Liu Liu, Qiang Liu, Zhouchen Lin, Xu-Yao Zhang, Wentao Zhang and Bin Dong},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1171},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1171}
}

References

40 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609.
  3. Tianyi Bai, Zengjie Hu, Fupeng Sun, Jiantao Qiu, Yizhen Jiang, Guangxin He, Bohan Zeng, Conghui He, Binhang Yuan, and Wentao Zhang. 2025. Multi-step visual reasoning with visual tokens scaling and verification. arXiv preprint arXiv:2506.07235. — [arXiv:2506.07235](https://arxiv.org/abs/2506.07235) find
  4. Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. 2025. Sft or rl? an early investigation into training r1-like reasoning large vision-language models. arXiv preprint arXiv:2504.11468.
  5. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
  7. Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, and 1 others. 2025. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. In Proceedings of the Computer Vision and Pattern Reco…
  8. Runquan Gui, Zhihai Wang, Jie Wang, Chi Ma, Huiling Zhen, Mingxuan Yuan, Jianye Hao, Defu Lian, Enhong Chen, and Feng Wu. 2025. Hypertree planning: Enhancing llm reasoning via hierarchical thinking. Preprint, arXiv:2505.02322. — [arXiv:2505.02322](https://arxiv.org/abs/2505.02322) find
  9. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025a. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  10. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, and 1 others. 2024a. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXiv preprint arXiv:2401.14196. find
  11. Meng-Hao Guo, Jiajun Xu, Yi Zhang, Jiaxi Song, Haoyang Peng, Yi-Xuan Deng, Xinzhi Dong, Kiyohiro Nakayama, Zhengyang Geng, Chen Wang, and 1 others. 2025b. R-bench: Graduate-level multidisciplinary benchmarks for llm & mllm complex reasoning evaluation. arXiv preprint arXiv:2505.02018. — [arXiv:2505.02018](https://arxiv.org/abs/2505.02018) find
  12. Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. 2024b. Large language model based multi-agents: A survey of progress and challenges. arXiv preprint arXiv:2402.01680.
  13. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300.
  14. Kairui Hu, Penghao Wu, Fanyi Pu, Wang Xiao, Yuanhan Zhang, Xiang Yue, Bo Li, and Ziwei Liu. 2025. Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos. arXiv preprint arXiv:2501.13826. — [arXiv:2501.13826](https://arxiv.org/abs/2501.13826) find
  15. Yichen Huang and Lin F. Yang. 2025. Gemini 2.5 pro capable of winning gold at imo 2025. Preprint, arXiv:2507.15855. — [arXiv:2507.15855](https://arxiv.org/abs/2507.15855) find
  16. Zekun Li, Xianjun Yang, Kyuri Choi, Wanrong Zhu, Ryan Hsieh, HyeonJung Kim, Jin Hyuk Lim, Sungyoung Ji, Byungju Lee, Xifeng Yan, and 1 others. 2024. Mmsci: A multimodal multi-discipline dataset for phd-level scientific comprehension. In AI for Accelerated Materials Design-Vienna 2024. find
  17. Hao Liang, Ruitao Wu, Bohan Zeng, Junbo Niu, Wentao Zhang, and Bin Dong. 2025. Multimodal reasoning for science: Technical report and 1st place solution to the icml 2025 seephys challenge. arXiv preprint arXiv:2509.06079. — [arXiv:2509.06079](https://arxiv.org/abs/2509.06079) find
  18. Hanmeng Liu, Ruoxi Ning, Zhiyang Teng, Jian Liu, Qiji Zhou, and Yue Zhang. 2023. Evaluating the logical reasoning ability of chatgpt and gpt-4. arXiv preprint arXiv:2304.03439.
  19. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255.
  20. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393.
  21. OpenAI. 2025b. o3: Advanced reasoning model. find
  22. Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, and 1 others. 2025. Humanity’s last exam. arXiv preprint arXiv:2501.14249.
  23. Jiahao Qiu, Jingzhe Shi, Xinzhe Juan, Zelin Zhao, Jiayi Geng, Shilong Liu, Hongru Wang, Sanfeng Wu, and Mengdi Wang. 2025a. Physics supernova: Ai agent matches elite gold medalists at ipho 2025. Preprint, arXiv:2509.01659. — [arXiv:2509.01659](https://arxiv.org/abs/2509.01659) find
  24. Jiahao Qiu, Fulian Xiao, Yimin Wang, Yuchen Mao, Yijia Chen, Xinzhe Juan, Shu Zhang, Siran Wang, Xuan Qi, Tongcheng Zhang, and 1 others. 2025b. On path to multimodal historical reasoning: Histbench and histagent. arXiv preprint arXiv:2505.20246. — [arXiv:2505.20246](https://arxiv.org/abs/2505.20246) find
  25. Mrinal Rawat, Ambuje Gupta, Rushil Goomer, Alessandro Di Bari, Neha Gupta, and Roberto Pieraccini. 2025. Pre-act: Multi-step planning and reasoning improves acting in llm agents. Preprint, arXiv:2505.09970. — [arXiv:2505.09970](https://arxiv.org/abs/2505.09970) find
  26. Jiacheng Ruan, Dan Jiang, Xian Gao, Ting Liu, Yuzhuo Fu, and Yangyang Kang. 2025. Mme-sci: A comprehensive and challenging science benchmark for multimodal large language models. arXiv preprint arXiv:2508.13938. — [arXiv:2508.13938](https://arxiv.org/abs/2508.13938) find
  27. Hui Shen, Taiqiang Wu, Qi Han, Yunta Hsieh, Jizhou Wang, Yuyue Zhang, Yuxin Cheng, Zijian Hao, Yuansheng Ni, Xin Wang, and 1 others. 2025. Phyx: Does your model have the" wits" for physical reasoning? arXiv preprint arXiv:2505.15929. — [arXiv:2505.15929](https://arxiv.org/abs/2505.15929) find
  28. Yang Shi, Huanqian Wang, Wulin Xie, Huanyao Zhang, Lijie Zhao, Yi-Fan Zhang, Xinfeng Li, Chaoyou Fu, Zhuoer Wen, Wenting Liu, and 1 others. 2025. Mme-videoocr: Evaluating ocr-based capabilities of multimodal llms in video scenarios. arXiv preprint arXiv:2505.21333. — [arXiv:2505.21333](https://arxiv.org/abs/2505.21333) find
  29. Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, and 1 others. 2025. Openthinkimg: Learning to think with images via visual tool reinforcement learning. arXiv preprint arXiv:2505.08617. — [arXiv:2505.08617](https://arxiv.org/abs/2505.08617) find
  30. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, and 1 others. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266–95290…
  31. Kun Xiang, Heng Li, Terry Jingchen Zhang, Yinya Huang, Zirong Liu, Peixin Qu, Jixi He, Jiaqi Chen, Yu-Jie Yuan, Jianhua Han, and 1 others. 2025. Seephys: Does seeing help thinking?–benchmarking vision-based physics reasoning. arXiv preprint arXiv:2505.19099. find
  32. Ran Xu, Yuchen Zhuang, Yishan Zhong, Yue Yu, Xiangru Tang, Hang Wu, May D Wang, Peifeng Ruan, Donghan Yang, Tao Wang, and 1 others. 2025. Medagentgym: Training llm agents for code-based medical reasoning at scale. arXiv preprint arXiv:2506.04405. — [arXiv:2506.04405](https://arxiv.org/abs/2506.04405) find
  33. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Preprint, arXiv:2305.10601. find
  34. Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. 2023. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490.
  35. Jiakang Yuan, Tianshuo Peng, Yilei Jiang, Yiting Lu, Renrui Zhang, Kaituo Feng, Chaoyou Fu, Tao Chen, Lei Bai, Bo Zhang, and 1 others. 2025. Mme-reasoning: A comprehensive benchmark for logical reasoning in mllms. arXiv preprint arXiv:2505.21327. — [arXiv:2505.21327](https://arxiv.org/abs/2505.21327) find
  36. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, and 1 others. 2024. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vi…
  37. Jianrui Zhang, Mu Cai, and Yong Jae Lee. 2024. Vinoground: Scrutinizing lmms over dense temporal reasoning with short videos. arXiv preprint arXiv:2410.02763. — [arXiv:2410.02763](https://arxiv.org/abs/2410.02763) find
  38. Zihan Zheng, Zerui Cheng, Zeyu Shen, Shang Zhou, Kaiyuan Liu, Hansen He, Dongruixuan Li, Stanley Wei, Hangyi Hao, Jianzhu Yao, and 1 others. 2025. Livecodebench pro: How do olympiad medalists judge llms in competitive programming? arXiv preprint arXiv:2506.11928. — [arXiv:2506.11928](https://arxiv.org/abs/2506.11928) find
  39. Kun Zhou, Beichen Zhang, Zhipeng Chen, Xin Zhao, Jing Sha, Zhichao Sheng, Shijin Wang, Ji-Rong Wen, and 1 others. 2024. Jiuzhang3. 0: Efficiently improving mathematical reasoning by training small data synthesis models. Advances in Neural Information Processing Systems, 37:1854–1889. find
  40. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elicits reasoning in large language models. Preprint, arXiv:2201.11903.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-morphobench-benchmark-difficulty-adaptive-model/