Elamur Research⌕
☰
home / papers / 2025 / DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressively

DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressively

Paper · 2025 · Yixuan Weng, Minjun Zhu, Qiujie Xie et al.

Summary

DeepScientist addresses a weakness of earlier "AI Scientist" agents. Those systems produce novel but often aimless results that rarely solve problems humans actually care about. The authors frame research as goal-directed Bayesian Optimization: the agent searches the space of candidate methods for the one that maximizes an expensive, black-box measure of scientific value, here a task performance metric. An LLM agent cycles through three stages of rising cost. It proposes hypotheses, implements and tests them in code, then analyzes and writes up the successful ones. A growing Findings Memory of human and self-generated results helps it decide when to refine promising leads and when to explore new directions. Starting from recent human SOTA methods on three AI tasks, the system ran for about a month on 16 H800 GPUs and beat the baseline on every task. The work offers large-scale evidence that autonomous agents can move past human SOTA. It also shows that nearly all generated ideas fail, which makes filtering and validation the new bottleneck.

Key points

  • Three tasks were used, each starting from a human SOTA method (ICML 2025 Spotlight, ACL 2025 Outstanding, ICLR 2024). The gains were:
    • Agent Failure Attribution: +183.7% accuracy (method A2P)
    • LLM Inference Acceleration: +1.9% tokens/second (method ACRA)
    • AI Text Detection: +7.9% AUROC on RAID (methods T-Detect, TDT, PA-Detect)
  • The run consumed about 20,000 GPU hours and roughly $100,000 in total. It produced around 5,000 ideas, about 1,100 of which were tested experimentally and only 21 of which led to real innovations.
  • On RAID, the agent matched roughly three years of human progress in about two weeks. All its zero-shot detectors use Falcon-7B and run faster than the previous SOTA, Binoculars.
  • The authors report a near-linear relationship between allocated compute and the number of valuable discoveries.
  • Code is available at github.com/ResearAI/DeepScientist. The generated papers were posted on arXiv and reviewed by a small program committee.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Westlake UniversityHangzhou · China60 works · 8,829 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksDecision Sciences · 306 worksInformation Systems and Management · 96 worksMaterials Science · 58 worksMaterials Chemistry · 58 works

Cite

Cite as Yixuan Weng, Minjun Zhu, Qiujie Xie, Qiyao Sun, Zhen Lin, Sifan Liu and Yue Zhang. 2025. DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressively. arXiv:2509.26603.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2509.26603 · arXiv 2509.26603

@misc{weng2025deepscientist,
  title = {{DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressively}},
  author = {Yixuan Weng, Minjun Zhu, Qiujie Xie, Qiyao Sun, Zhen Lin, Sifan Liu and Yue Zhang},
  year = {2025},
  doi = {10.48550/arxiv.2509.26603},
  eprint = {2509.26603},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2509.26603}
}

References

55 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, et al. The falcon series of open language models. arXiv preprint arXiv:2311.16867, 2023.
  2. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021.
  3. Eser Aygün, Anastasiya Belyaeva, Gheorghe Comanici, Marc Coram, Hao Cui, Jake Garrison, Renee Johnston Anton Kast, Cory Y. McLean, Peter Norgaard, Zahra Shamsi, David Smalling, James Thompson, Subhashini Venugopalan, Brian P. Williams, Chujun He, Sarah Martinson, Martyna Plomecka, Lai Wei, Yuchen Zh… — An AI system to help scientists write expert-level empirical software
  4. Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. Researchagent: Iterative research idea generation over scientific literature with large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics…
  5. Guangsheng Bao, Yanbin Zhao, Juncai He, and Yue Zhang. Glimpse: Enabling white-box methods to use proprietary models for zero-shot llm-generated text detection. In The Thirteenth International Conference on Learning Representations, a. Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Z… — Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature
  6. Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, et al. Mle-bench: Evaluating machine learning agents on machine learning engineering. arXiv preprint arXiv:2410.07095, 2024.
  7. Cristina Cornelio, Sanjeeb Dash, Vernon Austel, Tyler R. Josephson, Joao Goncalves, Kenneth L. Clarkson, Nimrod Megiddo, Bachir El Khadir, and Lior Horesh. Combining data and theory for derivable scientific discovery with AI-Descartes. Nature Communications, 14(1):1777, April 2023. ISSN 2041-1723. d… — [doi:10.1038/s41467-023-37236-y](https://doi.org/10.1038/s41467-023-37236-y) find
  8. Cristina Cornelio, Takuya Ito, Ryan Cory-Wright, Sanjeeb Dash, and Lior Horesh. The need for verification in ai-driven scientific discovery, 2025. URL https://arxiv.org/abs/2509.01398. — [arXiv:2509.01398](https://arxiv.org/abs/2509.01398) ↗
  9. Ryan Cory-Wright, Cristina Cornelio, Sanjeeb Dash, Bachir El Khadir, and Lior Horesh. Evolving scientific discovery by unifying data and background knowledge with ai hilbert. Nature Communications, 15:5922, July 2024. doi: 10.1038/s41467-024-50074-w. URL https://doi.org/10.1038/s41467-024-50074-w. — [doi:10.1038/s41467-024-50074-w](https://doi.org/10.1038/s41467-024-50074-w) ↗
  10. Liam Dugan, Alyssa Hwang, Filip Trhlík, Andrew Zhu, Josh Magnus Ludan, Hainiu Xu, Daphne Ippolito, and Chris Callison-Burch. RAID: A shared benchmark for robust evaluation of machine-generated text detectors. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics…
  11. Alhussein Fawzi, Matej Balog, Aja Huang, Thomas Hubert, Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Francisco J R. Ruiz, Julian Schrittwieser, Grzegorz Swirszcz, et al. Discovering faster matrix multiplication algorithms with reinforcement learning. Nature, 610(7930):47–53… find
  12. Alexander Fleming. Penicillin. British medical journal, 2(4210):386, 1941. find
  13. Peter I Frazier. A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811, 2018. — [arXiv:1807.02811](https://arxiv.org/abs/1807.02811) find
  14. Roman Garnett. Bayesian optimization. Cambridge University Press, 2023. find
  15. Soumya Suvra Ghosal, Souradip Chakraborty, Jonas Geiping, Furong Huang, Dinesh Manocha, and Amrit Bedi. A survey on the possibilities & impossibilities of AI-generated text detection. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=AXtFeYjboj. Su… ↗
  16. Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Anil Palepu, Petar Sirkovic, Artiom Myaskovsky, Felix Weissenberger, Keran Rong, Ryutaro Tanno, et al. Towards an ai co-scientist. arXiv preprint arXiv:2502.18864, 2025.
  17. Martin A Green. Silicon solar cells: evolution, high-efficiency design and efficiency enhancements. Semiconductor science and technology, 8(1):1, 1993. find
  18. Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Spotting llms with binoculars: Zero-shot detection of machine-generated text. In International Conference on Machine Learning, pp. 17519–17537. PMLR, 2024.
  19. Xinyi Hou, Yanjie Zhao, Shenao Wang, and Haoyu Wang. Model context protocol (mcp): Landscape, security threats, and future research directions. arXiv preprint arXiv:2503.23278, 2025. — [arXiv:2503.23278](https://arxiv.org/abs/2503.23278) find
  20. Xiaomeng Hu, Pin-Yu Chen, and Tsung-Yi Ho. Radar: Robust ai-text detection via adversarial learning. Advances in neural information processing systems, 36:15077–15095, 2023.
  21. Intology. Zochi technical report. arXiv, 2025. find
  22. Tang Jiabin, Xia Lianghao, Li Zhonghang, and Huang Chao. Ai-researcher: Autonomous scientific innovation, 2025. URL https://arxiv.org/abs/2505.18705. ↗
  23. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, pp. 611–626, 2023.
  24. Robert Tjarko Lange, Yuki Imajuku, and Edoardo Cetin. Shinkaevolve: Towards open-ended and sample-efficient program evolution. arXiv preprint arXiv:2509.19349, 2025.
  25. Bin Li and Yixuan Weng. Prompt-based system for personality and interpersonal reactivity prediction. Software Impacts, 12:100296, 2022. find
  26. Bin Li, Yixuan Weng, Qiya Song, and Hanjun Deng. Artificial text detection with multiple training strategies. arXiv preprint arXiv:2212.05194, 2022. — [arXiv:2212.05194](https://arxiv.org/abs/2212.05194) find
  27. Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292v3, 2024. URL https://www.arxiv.org/abs/2408.06292v3. ↗
  28. Jiacheng Miao, Joe R. Davis, Jonathan K. Pritchard, and James Zou. Paper2agent: Reimagining research papers as interactive and reliable ai agents, 2025. URL https://arxiv.org/abs/2509.06917. — [arXiv:2509.06917](https://arxiv.org/abs/2509.06917) ↗
  29. Gordon Moore. Moore’s law. Electronics Magazine, 38(8):114, 1965. find
  30. Alexander Novikov, Ngân Vu, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al. Alphaevolve: A coding agent for scientific and algorithmic discovery. Technical report, Technical report, Google DeepMin…
  31. José R Penadés, Juraj Gottweis, Lingchen He, Jonasz B Patkowski, Alexander Shurick, Wei-Hung Weng, Tao Tu, Anil Palepu, Artiom Myaskovsky, Annalisa Pawlosky, et al. Ai mirrors experimental science to uncover a novel mechanism of gene transfer crucial to bacterial evolution. bioRxiv, pp. 2025–02, 202… find
  32. Samuel Schmidgall and Michael Moor. Agentrxiv: Towards collaborative autonomous research. arXiv preprint arXiv:2503.18102, 2025.
  33. Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. Agent laboratory: Using llm agents as research assistants. arXiv preprint arXiv:2501.04227v1, 2025a. URL https://www.arxiv.org/abs/2501.04227v1. ↗
  34. Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. Agent laboratory: Using llm agents as research assistants. arXiv preprint arXiv:2501.04227, 2025b.
  35. Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Jun Shern Chan, Leon Maksin, Rachel Dias, Evan Mays, Benjamin Kinsella, Wyatt Thompson, et al. Paperbench: Evaluating ai’s ability to replicate ai research. arXiv preprint arXiv:2504.01848, 2025.
  36. Jinyan Su, Terry Yue Zhuo, Di Wang, and Preslav Nakov. Detectllm: Leveraging log rank information for zero-shot detection of machine-generated text. In The 2023 Conference on Empirical Methods in Natural Language Processing.
  37. Kyle Swanson, Wesley Wu, Nash L Bulaong, John E Pak, and James Zou. The virtual lab of ai agents designs new sars-cov-2 nanobodies. Nature, pp. 1–3, 2025. find
  38. Jue Wang, Ke Chen, Gang Chen, Lidan Shou, and Julian McAuley. Skipbert: Efficient inference with shallow layer skipping. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 7287–7301, 2022. find
  39. Yifan Wei, Xiaoyan Yu, Yixuan Weng, Huanhuan Ma, Yuanzhe Zhang, Jun Zhao, and Kang Liu. Does knowledge localization hold true? surprising differences between entity and relation perspectives in language models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Mana… find
  40. Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Kang Liu, and Jun Zhao. Mastering symbolic operations: Augmenting language models with compiled neural networks. In The Twelfth International Conference on Learning Representations. Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Li… — Large Language Models are Better Reasoners with Self-Verification
  41. Yixuan Weng, Shizhu He, Kang Liu, Shengping Liu, and Jun Zhao. Controllm: Crafting diverse personalities for language models. arXiv preprint arXiv:2402.10151, 2024. — [arXiv:2402.10151](https://arxiv.org/abs/2402.10151) find
  42. Yixuan Weng, Minjun Zhu, Guangsheng Bao, Hongbo Zhang, Jindong Wang, Yue Zhang, and Linyi Yang. Cycleresearcher: Improving automated research via automated review. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=bjcsVLoHYs. ↗
  43. Christopher Wolters, Xiaoxuan Yang, Ulf Schlichtmann, and Toyotaro Suzumura. Memory is all you need: An overview of compute-in-memory architectures for accelerating large language model inference, 2024. URL https://arxiv.org/abs/2406.08413. ↗
  44. Heming Xia, Zhe Yang, Qingxiu Dong, Peiyi Wang, Yongqi Li, Tao Ge, Tianyu Liu, Wenjie Li, and Zhifang Sui. Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for…
  45. Qiujie Xie, Qingqiu Li, Zhuohao Yu, Yuejie Zhang, Yue Zhang, and Linyi Yang. An empirical analysis of uncertainty in large language model evaluations. In The Thirteenth International Conference on Learning Representations, 2025a. URL https://openreview.net/forum?id=J4xLuCt2kg. ↗
  46. Qiujie Xie, Yixuan Weng, Minjun Zhu, Fuchen Shen, Shulin Huang, Zhen Lin, Jiahui Zhou, Zilan Mao, Zijie Yang, Linyi Yang, et al. How far are ai scientists from changing the world? arXiv preprint arXiv:2507.23276, 2025b. find
  47. Yutaro Yamada, Robert Tjarko Lange, Cong Lu, Shengran Hu, Chris Lu, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist-v2: Workshop-level automated scientific discovery via agentic tree search. arXiv preprint arXiv:2504.08066, 2025.
  48. Zijie Yang, Yukai Wang, and Lijing Zhang. Ai becomes a masterbrain scientist. bioRxiv, pp. 2023– 04, 2023. find
  49. Paul Zarchan. Progress in astronautics and aeronautics: fundamentals of Kalman filtering: a practical approach, volume 208. Aiaa, 2005. find
  50. Guibin Zhang, Junhao Wang, Junjie Chen, Wangchunshu Zhou, Kun Wang, and Shuicheng Yan. Agentracer: Who is inducing failure in the llm agentic systems?, 2025a. URL https://arxiv.org/abs/2509.03312. — [arXiv:2509.03312](https://arxiv.org/abs/2509.03312) ↗
  51. Pengsong Zhang, Heng Zhang, Huazhe Xu, Renjun Xu, Zhenting Wang, Cong Wang, Animesh Garg, Zhibin Li, Arash Ajoudani, and Xinyu Liu. Scaling laws in scientific discovery with ai and robot scientists. arXiv preprint arXiv:2503.22444, 2025b. — [arXiv:2503.22444](https://arxiv.org/abs/2503.22444) find
  52. Shaokun Zhang, Ming Yin, Jieyu Zhang, Jiale Liu, Zhiguang Han, Jingyang Zhang, Beibin Li, Chi Wang, Huazheng Wang, Yiran Chen, and Qingyun Wu. Which agent causes task failures and when? on automated failure attribution of LLM multi-agent systems. In Forty-second International Conference on Machine L… find
  53. Minjun Zhu, Yixuan Weng, Linyi Yang, and Yue Zhang. Deepreview: Improving llm-based paper review with human-like deep thinking process. arXiv preprint arXiv:2503.08569, 2025a.
  54. Minjun Zhu, Yixuan Weng, Linyi Yang, and Yue Zhang. Personality alignment of large language models. In The Thirteenth International Conference on Learning Representations, 2025b.
  55. Minjun Zhu, Qiujie Xie, Yixuan Weng, Jian Wu, Zhen Lin, Linyi Yang, and Yue Zhang. Ai scientists fail without strong implementation capability. arXiv preprint arXiv:2506.01372, 2025c.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/weng-2025-deepscientist-advancing-frontier-pushing/