Elamur Research⌕
☰
home / papers / 2025 / SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?

SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?

Paper · 2025 · Jingyi Chai, Shuo Tang, Rui Ye et al.

Summary

The paper asks whether openly available models can reach top results on Humanity's Last Exam (HLE), a very hard expert-written benchmark that serves as a proxy for readiness to support scientific research. Previous leaders on HLE were closed products from OpenAI and Google. The authors do not train a new model. Instead, they build X-Master, an agent based on an open reasoning model such as DeepSeek-R1. While it reasons, the agent writes Python snippets to call tools. These snippets run in a sandbox with scientific libraries and custom web search and page-parsing utilities, and the outputs are fed back into the model's context. Because the base model is not naturally agentic, they insert first-person guiding text at the start of its thinking. On top of this, X-Masters runs several agent roles: first broadly in parallel, then by combining and refining their outputs. This open, inference-time setup scores 32.1% on HLE, ahead of the earlier closed systems. It shows that leading benchmark results are reachable without proprietary models or retraining.

Key points

  • HLE result: X-Masters reaches 32.1% on HLE and is the first system reported above 30%. This beats OpenAI Deep Research (26.6%) by 5.5 points and Google's Deep Research (26.9%) by 5.2 points.
  • Code as interaction language:
    • Inside the model's reasoning, tool calls are written as code between <code> tags.
    • The code is executed, and the results come back between <execution_results> tags.
    • This gives the agent access to libraries like NumPy/SciPy, custom web search and parsing tools, and even tools it writes on the fly.
  • Initial Reasoning Guidance: first-person guiding statements are placed right after the <think> token. This steers non-agentic reasoners like DeepSeek-R1 toward tool use, where ordinary prompting proved unreliable.
  • Scattered-and-stacked workflow:
    • In the "scattered" phase, parallel Solver agents propose answers and Critic agents fix flaws.
    • In the "stacked" phase, Rewriter agents merge the outputs and a Selector agent picks the final answer.
    • In the ablation, removing scattering drops the score to 25.5% and removing stacking drops it to 25.0%.
  • Openness and series context: the code is released at github.com/sjtu-sai-agents/X-Master (Shanghai Jiao Tong University and DP Technology). This work is Part I of the SciMaster series, which plans specialized scientific tools and end-to-end trained agents.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksDecision Sciences · 306 worksInformation Systems and Management · 96 works

Cite

Cite as Jingyi Chai, Shuo Tang, Rui Ye, Yuwen Du, Xinyu Zhu, Mengcheng Zhou, Yanfeng Wang, Weinan E, Yuzhi Zhang, Linfeng Zhang and Siheng Chen. 2025. SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?. arXiv:2507.05241.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2507.05241 · arXiv 2507.05241

@misc{chai2025scimaster,
  title = {{SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?}},
  author = {Jingyi Chai, Shuo Tang, Rui Ye, Yuwen Du, Xinyu Zhu, Mengcheng Zhou, Yanfeng Wang, Weinan E, Yuzhi Zhang, Linfeng Zhang and Siheng Chen},
  year = {2025},
  doi = {10.48550/arxiv.2507.05241},
  eprint = {2507.05241},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2507.05241}
}

References

36 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. Introducing claude 4. https://www.anthropic.com/news/claude-4, 2025. Accessed: 2025-07-03. ↗
  2. Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine learning, 47:235–256, 2002. find
  3. Google DeepMind. Gemini 2.5: Our most intelligent ai model. https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/\#gemini-2-5-thinking, 2025. Accessed: 2025-06-24. ↗
  4. DeepSeek-AI. Deepseek-r1-0528. https://huggingface.co/deepseek-ai/DeepSeek-R1-0528, 2025. Accessed: 2025-06-27. ↗
  5. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
  6. Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. Retool: Reinforcement learning for strategic tool use in llms. arXiv preprint arXiv:2504.11536, 2025.
  7. Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Anil Palepu, Petar Sirkovic, Artiom Myaskovsky, Felix Weissenberger, Keran Rong, Ryutaro Tanno, et al. Towards an ai co-scientist. arXiv preprint arXiv:2502.18864, 2025.
  8. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025.
  9. Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al. Metagpt: Meta programming for a multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations, 2024.
  10. Kexin Huang, Serena Zhang, Hanchen Wang, Yuanhao Qu, Yingzhou Lu, Yusuf Roohani, Ryan Li, Lin Qiu, Junze Zhang, Yin Di, et al. Biomni: A general-purpose biomedical ai agent. bioRxiv, pages 2025–05, 2025. find
  11. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516, 2025a.
  12. Ruofan Jin, Zaixi Zhang, Mengdi Wang, and Le Cong. Stella: Self-evolving llm agent for biomedical research. arXiv preprint arXiv:2507.02004, 2025b.
  13. Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-o1: Agentic search-enhanced large reasoning models. arXiv preprint arXiv:2501.05366, 2025a.
  14. Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji-Rong Wen, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability. arXiv preprint arXiv:2504.21776, 2025b.
  15. Xuefeng Li, Haoyang Zou, and Pengfei Liu. Torl: Scaling tool-integrated rl. arXiv preprint arXiv:2503.23383, 2025c.
  16. Moonshot-AI. Kimi-researcher: End-to-end rl training for emerging agentic capabilities. https://moonshotai.github.io/Kimi-Researcher/, 2025. Accessed: 2025-06-26. ↗
  17. OpenAI. Introducing chatgpt. https://openai.com/index/chatgpt/, 2022. Accessed: 2025-06-24. ↗
  18. OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
  19. OpenAI. Introducing openai o3 and o4-mini. introducing-o3-and-o4-mini/, 2025a. Accessed: 2025-06-24. find
  20. OpenAI. Openai o3-mini. https://openai.com/index/openai-o3-mini/, 2025b. Accessed: 2025-06-26. ↗
  21. OpenAI. Introducing deep research. https://openai.com/index/introducing-deep-research/, 2025c. Accessed: 2025-06-26. ↗
  22. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. NIPS, 35:27730–27744, 2022.
  23. Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al. Humanity’s last exam. arXiv preprint arXiv:2501.14249, 2025.
  24. Mihir Prabhudesai, Lili Chen, Alex Ippoliti, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak. Maximizing confidence alone improves reasoning. arXiv preprint arXiv:2505.22660, 2025. — [arXiv:2505.22660](https://arxiv.org/abs/2505.22660) find
  25. Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. Chatdev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1…
  26. Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-searcher: Incentivizing the search capability in llms via reinforcement learning. arXiv preprint arXiv:2503.05592, 2025.
  27. Richard S Sutton, Andrew G Barto, et al. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998. find
  28. Gerald Tesauro et al. Temporal difference learning and td-gammon. Communications of the ACM, 38(3):58–68, 1995. find
  29. Jialong Wu, Baixuan Li, Runnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingchu Zhang, Zekun Xi, Yong Jiang, Pengjun Xie, et al. Webdancer: Towards autonomous information seeking agency. arXiv preprint arXiv:2505.22648, 2025a.
  30. Junde Wu, Jiayuan Zhu, and Yuyuan Liu. Agentic reasoning: Reasoning llms with tools for the deep research. arXiv preprint arXiv:2502.04644, 2025b. — [arXiv:2502.04644](https://arxiv.org/abs/2502.04644) find
  31. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei… find
  32. Rui Ye, Shuo Tang, Rui Ge, Yaxin Du, Zhenfei Yin, Siheng Chen, and Jing Shao. Mas-gpt: Training llms to build llm-based multi-agent systems. arXiv preprint arXiv:2503.03686, 2025. — [arXiv:2503.03686](https://arxiv.org/abs/2503.03686) find
  33. Zhongyue Zhang, Zijie Qiu, Yingcheng Wu, Shuya Li, Dingyan Wang, Zhuomin Zhou, Duo An, Yuhan Chen, Yu Li, Yongbo Wang, et al. Origene: A self-evolving virtual disease biologist automating therapeutic target discovery. bioRxiv, pages 2025–06, 2025. https://openai.com/index/ ↗
  34. Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng, and Gao Huang. Absolute zero: Reinforced self-play reasoning with zero data. arXiv preprint arXiv:2505.03335, 2025a.
  35. Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, and Dawn Song. Learning to reason without external rewards. arXiv preprint arXiv:2505.19590, 2025b.
  36. Cases Query Fourteen circles of radius one are tightly packed into another circle. What is the radius of this circle up to 4 significant digits? Response Recognize that “tightly packed” means finding the smallest enclosing circle for 14 unit-radius circles; decide that known optimal values must exis… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chai-2025-scimaster-towards-general-purpose-scientific/