Elamur Research
☰
home / papers / 2025 / The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search

The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2504.08066

Summary

The AI Scientist-v2, from Sakana AI and collaborators, is an end-to-end agent that carries out machine learning research with no human involvement. It generates ideas, runs experiments, plots results and writes papers. Its predecessor, The AI Scientist-v1, needed a hand-written code template for every new topic and tested hypotheses in a single shallow sequence. Version 2 drops the templates. It begins each project with broad, proposal-style idea generation and checks novelty against Semantic Scholar. Experiments then run as a staged, parallel tree search, overseen by an experiment manager agent and inspired by AIDE. A vision-language model critiques the figures and captions. To test the system, the authors entered three fully generated papers in an ICLR 2025 workshop. One of them scored well enough to have been accepted had a human written it. This is presented as the first wholly AI-produced paper to clear peer review. The authors note that workshop standards fall below those of main conferences. They have open-sourced the code and the experiment data.

Key points

  • Removed bottlenecks: v2 needs no human-authored code templates. Exploration is tree-based rather than linear, and experiments run in parallel. Paper writing changed from v1's step-by-step Aider workflow to a single drafting pass followed by reflection with reasoning models such as o1.
  • Peer-review result: Of three papers submitted to an ICLR 2025 workshop, one received an average reviewer score of 6.33. That placed it in roughly the top 45% of submissions, above the typical acceptance threshold.
  • Content of the accepted paper: It tested a regularizer that penalizes large changes in embeddings between consecutive time steps, aiming to improve compositional generalization. On synthetic arithmetic-expression datasets this gave no significant gains and sometimes hurt performance, a negative result. The authors' own review found unclear method details, possible dataset overlap and inaccurate figure captions.
  • Tree search design: Each node in the search holds a code state with a score. The best-performing checkpoints are carried forward through successive experiment stages. The approach adapts AIDE, which performed strongly on MLEBench.
  • Resources: The code is available at github.com/SakanaAI/AI-Scientist-v2, together with the workshop experiment data. The paper also discusses ethics and AI safety, including the risk of using such systems to game peer review.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Cited by

91 papers of our library cite it (of the 140673 whose bibliographies we read).

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2504.08066 · arXiv 2504.08066

@misc{yamada2025ai,
  title = {{The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search}},
  author = {Yamada, Yutaro and Lange, Robert Tjarko and Lu, Cong and Hu, Shengran and Lu, Chris and Foerster, Jakob and Clune, Jeff and Ha, David},
  year = {2025},
  doi = {10.48550/arxiv.2504.08066},
  eprint = {2504.08066},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2504.08066}
}

References

31 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AutoScience AI. Meet Carl: The First AI System to Produce Academically Peer-Reviewed Research, 2025. URL https://www.autoscience.ai/blog/meet-carl-the-first-ai-system-t o-produce-academically-peer-reviewed-research . Accessed: 2025-03-21. ↗
  2. Joeran Beel, Min-Yen Kan, and Moritz Baumgart. An evaluation of sakana’s ai scientist for autonomous research: Wishful thinking or an emerging reality towards’ artificial general research intelligence’(agri)? arXiv preprint arXiv:2502.14297, 2025. find
  3. Yoshua Bengio, Michael Cohen, Damiano Fornasiere, Joumana Ghosn, Pietro Greiner, Matt MacDer-mott, Sören Mindermann, Adam Oberman, Jesse Richardson, Oliver Richardson, Marc-Antoine Rondeau, Pierre-Luc St-Charles, and David Williams-King. Superintelligent agents pose catastrophic risks: Can scientist… find
  4. Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Aleksander Madry, and Lilian Weng. MLE-bench: Evaluating machine learning agents on machine learning engineering. In The Thirteenth International Conference o… find our card
  5. Jeff Clune. Ai-gas: Ai-generating algorithms, an alternate paradigm for producing general artificial intelligence. CoRR, abs/1905.10985, 2019. URL http://arxiv.org/abs/1905.10985 . — [arXiv:1905.10985](https://arxiv.org/abs/1905.10985) ↗
  6. Cristina Cornelio, Sanjeeb Dash, Vernon Austel, Tyler R. Josephson, Joao Goncalves, Kenneth L. Clarkson, Nimrod Megiddo, Bachir El Khadir, and Lior Horesh. Combining data and theory for derivable scientific discovery with ai-descartes. Nature Communications, 14(1):1777, Apr 2023. ISSN 2041-1723. doi… — [doi:10.1038/s41467-023-37236-y](https://doi.org/10.1038/s41467-023-37236-y) find
  7. The University of Hong Kong Data Intelligence Lab. Ai-researcher: Fully-automated scientific discovery with llm agents, 2025. URL https://github.com/HKUDS/AI-Researcher . ↗
  8. Steffen Eger, Yong Cao, Jennifer D’Souza, Andreas Geiger, Christian Greisinger, Stephanie Gross, Yufang Hou, Brigitte Krenn, Anne Lauscher, Yizhi Li, et al. Transforming science with large language models: A survey on ai-assisted scientific discovery, experimentation, content generation, and evaluat… find
  9. Tao Feng, Yihang Sun, and Jiaxuan You. Grapheval: A lightweight graph-based LLM framework for idea evaluation. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=5RUM1aIdok . ↗
  10. Paul Gauthier. Aider is ai pair programming in your terminal, 2024. URL https://aider.chat/ . https://aider.chat/ . ↗
  11. Yolanda Gil, Mark Greaves, James Hendler, and Haym Hirsh. Amplify scientific discovery with artificial intelligence. Science, 346(6206):171–172, 2014. doi: 10.1126/science.1259439. URL https://www.science.org/doi/abs/10.1126/science.1259439 . — [doi:10.1126/science.1259439](https://doi.org/10.1126/science.1259439) ↗
  12. Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Anil Palepu, Petar Sirkovic, Artiom Myaskovsky, Felix Weissenberger, Keran Rong, Ryutaro Tanno, et al. Towards an ai co-scientist. arXiv preprint arXiv:2502.18864, 2025. find our card
  13. Intology AI. Zochi Tech Report, 2025. URL https://www.intology.ai/blog/zochi-tech-r eport . Accessed: 2025-03-21. ↗
  14. Zhengyao Jiang, Dominik Schmidt, Dhruv Srikanth, Dixing Xu, Ian Kaplan, Deniss Jacenko, and Yuxiang Wu. Aide: Ai-driven exploration in the space of code, 2025. URL https://arxiv.org/ abs/2502.13138 . ↗
  15. Ross D. King, Jem Rowland, Stephen G. Oliver, Michael Young, Wayne Aubrey, Emma Byrne, Maria Liakata, Magdalena Markham, Pinar Pir, Larisa N. Soldatova, Andrew Sparkes, Kenneth E. Whelan, and Amanda Clare. The automation of science. Science, 324(5923):85–89, 2009. doi: 10.1126/sc ience.1165620. URL… — [doi:10.1126/sc](https://doi.org/10.1126/sc) find
  16. Hiroaki Kitano. Nobel turing challenge: creating the engine for scientific discovery. npj Systems Biology and Applications, 7(1):29, Jun 2021. ISSN 2056-7189. doi: 10.1038/s41540-021-00189-3. URL https://doi.org/10.1038/s41540-021-00189-3 . — [doi:10.1038/s41540-021-00189-3](https://doi.org/10.1038/s41540-021-00189-3) ↗
  17. Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292, 2024. find our card
  18. Ludovico Mitchener, Jon M Laurent, Benjamin Tenmann, Siddharth Narayanan, Geemi P Wellawatte, Andrew White, Lorenzo Sani, and Samuel G Rodriques. Bixbench: a comprehensive benchmark for llm-based agents in computational biology. arXiv preprint arXiv:2503.00096, 2025. find
  19. Jean-Baptiste Mouret and Jeff Clune. Illuminating search spaces by mapping elites. ArXiv, abs/1504.04909, 2015. URL https://api.semanticscholar.org/CorpusID:14759751 . — [arXiv:1504.04909](https://arxiv.org/abs/1504.04909) ↗
  20. OpenAI. Openai o1 system card, 2024. URL https://api.semanticscholar.org/CorpusID: 272648256 . ↗ our card
  21. Samuel Schmidgall and Michael Moor. Agentrxiv: Towards collaborative autonomous research. arXiv preprint arXiv:2503.18102, 2025. find
  22. Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. Agent laboratory: Using llm agents as research assistants. arXiv preprint arXiv:2501.04227, 2025. find our card
  23. Yu-Zhe Shi, Mingchen Liu, Fanxu Meng, Qiao Xu, Zhangqian Bi, Kun He, Lecheng Ruan, and Qining Wang. Hierarchically encapsulated representation for protocol design in self-driving labs. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openrevi ew.net/forum?id=… find
  24. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024. find our card
  25. Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can LLMs generate novel research ideas? a large-scale human study with 100+ NLP researchers. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=M23dTGWCZy . ↗ our card
  26. Minyang Tian, Luyu Gao, Shizhuo Dylan Zhang, Xinan Chen, Cunwei Fan, Xuefei Guo, Roland Haas, Pan Ji, Kittithat Krongchon, Yao Li, Shengyan Liu, Di Luo, Yutao Ma, Hao Tong, Kha Trinh, Chenyu Tian, Zihan Wang, Bohao Wu, Yanyu Xiong, Shengzhu Yin, Minhui Zhu, Kilian Lieret, Yanxin Lu, Genglin Liu, Yuf… find
  27. Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, Anima Anandkumar, Karianne Bergen, Carla P. Gomes, Shirley Ho, Pushmeet Kohli, Joan Lasenby, Jure Leskovec, Tie-Yan Liu, Arjun Manrai, Debora Marks, Bharath Ramsund… find
  28. Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better llm agents. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org, 2024. find our card
  29. Yixuan Weng, Minjun Zhu, Guangsheng Bao, Hongbo Zhang, Jindong Wang, Yue Zhang, and Linyi Yang. Cycleresearcher: Improving automated research via automated review. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/f orum?id=bjcsVLoHYs . ↗
  30. Hjalmar Wijk, Tao R. Lin, Joel Becker, Sami Jawhar, Neev Parikh, Thomas Broadley, Lawrence Chan, Michael Chen, Josh Clymer, Jai Dhyani, Elena Ericheva, Katharyn Garcia, Brian Goodrich, Nikola Jurkovic, Megan Kinniment, Aron Lajko, Seraphina Nix, Lucas Jun Koba Sato, William Saunders, Maksym Taran, B… find
  31. Yongjun Xu, Xin Liu, Xin Cao, Changping Huang, Enke Liu, Sen Qian, Xingchen Liu, Yanjun Wu, Fengliang Dong, Cheng-Wei Qiu, Junjun Qiu, Keqin Hua, Wentao Su, Jian Wu, Huiyu Xu, Yong Han, Chenguang Fu, Zhigang Yin, Miao Liu, Ronald Roepman, Sabine Dietmann, Marko Virta, Fredrick Kengara, Ze Zhang, Lif… — [doi:10.1016/j.xinn.2021.100179](https://doi.org/10.1016/j.xinn.2021.100179) find