📄 An AI system to help scientists write expert-level empirical software

Paper · 2025 · Eser Aygün, Anastasiya Belyaeva, Gheorghe Comanici et al.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

🏛️ Institutions

🏢GoogleMountain View · United States430 works · 424,363 citations🎓Massachusetts Institute of TechnologyCambridge · United States211 works · 69,000 citations🎓Harvard UniversityCambridge · United States112 works · 42,023 citations🎓McGill UniversityMontreal · Canada87 works · 11,027 citations🎓California Institute of TechnologyPasadena · United States8 works · 745 citations

🧭 Fields, topics and concepts

Cite

Cite as Eser Aygün, Anastasiya Belyaeva, Gheorghe Comanici, Marc Coram, Hao Cui, Jake Garrison, Renee Johnston Anton Kast, Cory Y. McLean, Peter Norgaard, Zahra Shamsi, David Smalling, James Thompson, Subhashini Venugopalan, Brian P. Williams, Chujun He, Sarah Martinson, Martyna Plomecka, Lai Wei, Yuchen Zhou, Qian-Ze Zhu, Matthew Abraham, Erica Brand, Anna Bulanova, Jeffrey A. Cardille, Chris Co, Scott Ellsworth, Grace Joseph, Malcolm Kane, Ryan Krueger, Johan Kartiwa, Dan Liebling, Jan-Matthis Lueckmann, Paul Raccuglia, Xuefei (Julie) Wang, Katherine Chou, James Manyika, Yossi Matias, John C. Platt, Lizzie Dorfman, Shibl Mourad and Michael P. Brenner. 2025. An AI system to help scientists write expert-level empirical software. arXiv:2509.06503.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2509.06503 · arXiv 2509.06503

@misc{aygun2025ai,
  title = {{An AI system to help scientists write expert-level empirical software}},
  author = {Eser Aygün, Anastasiya Belyaeva, Gheorghe Comanici, Marc Coram, Hao Cui, Jake Garrison, Renee Johnston Anton Kast, Cory Y. McLean, Peter Norgaard, Zahra Shamsi, David Smalling, James Thompson, Subhashini Venugopalan, Brian P. Williams, Chujun He, Sarah Martinson, Martyna Plomecka, Lai Wei, Yuchen Zhou, Qian-Ze Zhu, Matthew Abraham, Erica Brand, Anna Bulanova, Jeffrey A. Cardille, Chris Co, Scott Ellsworth, Grace Joseph, Malcolm Kane, Ryan Krueger, Johan Kartiwa, Dan Liebling, Jan-Matthis Lueckmann, Paul Raccuglia, Xuefei (Julie) Wang, Katherine Chou, James Manyika, Yossi Matias, John C. Platt, Lizzie Dorfman, Shibl Mourad and Michael P. Brenner},
  year = {2025},
  doi = {10.48550/arxiv.2509.06503},
  eprint = {2509.06503},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2509.06503}
}

References

92 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Hannay, J. E. et al. How do scientists develop and use scientific software? In 2009 ICSE Workshop on Software Engineering for Computational Science and Engineering, 1–8 (IEEE, 2009). find
2. Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016). find
3. Hohenberg, P. & Kohn, W. Inhomogeneous electron gas. Phys. Rev. 136, B864 (1964). find
4. Kohn, W. & Sham, L. J. Self-consistent equations including exchange and correlation effects. Phys. Rev. 140, A1133 (1965). find
5. Warshel, A. & Levitt, M. Theoretical studies of enzymic reactions: dielectric, electrostatic and steric stabilization of the carbonium ion in the reaction of lysozyme. J. Mol. Biol. 103, 227–249 (1976). find
6. Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021).
7. Baek, M. et al. Accurate prediction of protein structures and interactions using a three-track neural network. Science 373, 871–876 (2021). find
8. Hourdin, F. et al. The art and science of climate model tuning. Bull. Am. Meteorol. Soc. 98, 589–602 (2017). find
9. Anderson Jr., J. Basic philosophy of CFD. In Computational Fluid Dynamics, 3–14 (Springer, 2009). find
10. Silver, N. The signal and the noise: why so many predictions fail-but some don’t (Penguin, 2012). find
11. Farmer, J. D. Making sense of chaos: a better economics for a better world (Yale Univ. Press, 2024). find
12. Sculley, D. et al. Hidden technical debt in machine learning systems. In Advances in Neural Information Processing Systems, vol. 28 (2015). find
13. Jiang, Z. et al. AIDE: AI-driven exploration in the space of code. arXiv preprint arXiv:2502.13138 (2025). find
14. Novikov, A. et al. AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131 (2025).
15. Romera-Paredes, B. et al. Mathematical discoveries from program search with large language models. Nature 625, 468–475 (2024).
16. Hu, S., Lu, C. & Clune, J. Automated design of agentic systems. arXiv preprint arXiv:2408.08435 (2024). find
17. Xu, C. et al. Automatic cell-type harmonization and integration across Human Cell Atlas datasets. Cell 186, 5876–5891.e20 (2023). find
18. Centers for Disease Control and Prevention. COVID-19 forecast hub (2025). https://github.com/cdcgov/covid19-forecast-hub?tab=readme-ov-file. ↗
19. Aksu, T. et al. GIFT-Eval: a benchmark for general time series forecasting model evaluation. arXiv preprint arXiv:2410.10393 (2024). https://huggingface.co/spaces/Salesforce/GIFT-Eval. — [arXiv:2410.10393](https://arxiv.org/abs/2410.10393) ↗
20. Shao, Z., Yang, K. & Zhou, W. Performance evaluation of single-label and multi-label remote sensing image retrieval using a dense labeling dataset. Remote Sens. 10, 964 (2018). find
21. Lueckmann, J.-M. et al. ZAPBench: a benchmark for whole-brain activity prediction in zebrafish. arXiv preprint arXiv:2503.02618 (2025). — [arXiv:2503.02618](https://arxiv.org/abs/2503.02618) find
22. Jovic, D. et al. Single-cell RNA sequencing technologies and applications: a brief overview. Clin. and Transl. Med. 12, e694 (2022). find
23. Svensson, V., Vento-Tormo, R. & Teichmann, S. A. Exponential scaling of single-cell RNA-seq in the past decade. Nat. Protoc. 13, 599–604 (2018). find
24. Regev, A. et al. The Human Cell Atlas. eLife 6, e27041 (2017). find
25. CZI Cell Science Program et al. CZ CELLxGENE Discover: a single-cell data platform for scalable exploration, analysis and modeling of aggregated data. Nucleic Acids Res. 53, D886–D900 (2025). find
26. Stuart, T. & Satija, R. Integrative single-cell analysis. Nat. Rev. Genet. 20, 257–272 (2019). find
27. Zappia, L., Phipson, B. & Oshlack, A. Exploring the single-cell RNA-seq analysis landscape with the scRNA-tools database. PLoS Comput. Biol. 14, e1006245 (2018). find
28. Tran, H. T. N. et al. A benchmark of batch-effect correction methods for single-cell RNA sequencing data. Genome Biol. 21, 1–32 (2020). find
29. Chazarra-Gil, R., van Dongen, S., Kiselev, V. Y. & Hemberg, M. Flexible comparison of batch correction methods for single-cell RNA-seq using BatchBench. Nucleic Acids Res. 49, e42 (2021). find
30. Luecken, M. D. et al. Defining and benchmarking open problems in single-cell analysis. Nat. Biotechnol. 43, 1035–1040 (2025). find
31. Google. Gemini Deep Research (2025). https://gemini.google/overview/deep-research/ ?hl=en. ↗
32. Gottweis, J. et al. Towards an AI co-scientist. arXiv preprint arXiv:2502.18864 (2025).
33. Johnson, W. E., Li, C. & Rabinovic, A. Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics 8, 118–127 (2007). find
34. Polański, K. et al. BBKNN: fast batch alignment of single cell transcriptomes. Bioinformatics 36, 964–965 (2019). find
35. Chandrashekar, A. et al. TabVI: leveraging lightweight transformer architectures to learn biologically meaningful cellular representations. bioRxiv 2025–02 (2025). find
36. Koza, J. R. Genetic programming as a means for programming computers by natural selection. Stat. Comput. 4, 87–112 (1994). find
37. Schweim, D., Hemberg, E., Sobania, D., O’Reilly, U.-M. & Rothlauf, F. Using knowledge of human-generated code to bias the search in program synthesis with grammatical evolution. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, 331–332 (2021). find
38. Mernik, M., Heering, J. & Sloane, A. M. When and how to develop domain-specific languages. ACM computing surveys (CSUR) 37, 316–344 (2005). find
39. Czarnecki, K. Generative programming: Methods, techniques, and applications tutorial abstract. In International Conference on Software Reuse, 351–352 (Springer, 2002). find
40. Li, Y. et al. Competition-level code generation with AlphaCode. Science 378, 1092–1097 (2022).
41. Chen, M. et al. Evaluating large language models trained on code. arXiv:2107.03374 (2021). arXiv preprint
42. Hutter, F., Kotthoff, L. & Vanschoren, J. Automated machine learning: methods, systems, challenges (Springer Nature, 2019). find
43. Wu, X., Wu, S.-h., Wu, J., Feng, L. & Tan, K. C. Evolutionary computation in the era of large language model: survey and roadmap. IEEE Trans. Evol. Comput. (2024). find
44. Ma, Z., Guo, H., Gong, Y., Zhang, J. & Tan, K. Toward automated algorithm design: A survey and practical guide to meta-black-box-optimization. IEEE Transactions on Evolutionary Computation (2025). find
45. Ifargan, T., Hafner, L., Kern, M., Alcalay, O. & Kishony, R. Autonomous LLM-driven research— from data to human-verifiable research papers. NEJM AI 2, AIoa2400555 (2024). find
46. Xiao, Y. et al. CellAgent: An LLM-driven multi-agent framework for automated single-cell data analysis. arXiv preprint arXiv:2407.09811 (2024). — [arXiv:2407.09811](https://arxiv.org/abs/2407.09811) find
47. Silver, D. et al. Mastering the game of Go without human knowledge. Nature 550, 354–359 (2017). find
48. Coquelin, P.-A. & Munos, R. Bandit algorithms for tree search. arXiv preprint cs/0703062 (2007). find
49. Hemberg, E., Moskal, S. & O’Reilly, U.-M. Evolving code with a large language model. Genetic Programming and Evolvable Machines 25, 21 (2024). find
50. Meyerson, E. et al. Language model crossover: Variation through few-shot prompting. ACM Transactions on Evolutionary Learning 4, 1–40 (2024). find
51. Grishina, A., Liventsev, V., Härmä, A. & Moonen, L. Fully autonomous programming using iterative multi-agent debugging with large language models. ACM Transactions on Evolutionary Learning 5, 1–37 (2025). find
52. He, X., Zhao, K. & Chu, X. Automl: A survey of the state-of-the-art. Knowledge-Based Systems 212, 106622 (2021). find
53. Fang, Y. et al. MLZero: A multi-agent system for end-to-end machine learning automation. arXiv preprint arXiv:2505.13941 (2024). — [arXiv:2505.13941](https://arxiv.org/abs/2505.13941) find
54. Li, Z., Zang, Q., Ma, D. et al. AutoKaggle: A multi-agent framework for autonomous data science competitions. arXiv preprint arXiv:2410.20424 (2024). find
55. Grosnit, A. et al. Agent K v1.0: End-to-end autonomous data science agent. arXiv preprint arXiv:2410.02598 (2024). find
56. Perplexity. Perplexity Deep Research (2025). introducing-perplexity-deep-research. https://www.perplexity.ai/hub/blog/ ↗
57. Du, M., Xu, B., Zhu, C., Wang, X. & Mao, Z. DeepResearch Bench: a comprehensive benchmark for deep research agents. arXiv preprint arXiv:2506.11763 (2025).
58. Coelho, J. et al. DeepResearchGym: A free, transparent, and reproducible evaluation sandbox for deep research. arXiv preprint arXiv:2505.19253 (2025). — [arXiv:2505.19253](https://arxiv.org/abs/2505.19253) find
59. Xu, R. & Peng, J. A comprehensive survey of deep research: Systems, methodologies, and applications. arXiv preprint arXiv:2506.12594 (2025). — [arXiv:2506.12594](https://arxiv.org/abs/2506.12594) find
60. Baek, J., Jauhar, S. K., Cucerzan, S. & Hwang, S. J. ResearchAgent: iterative research idea generation over scientific literature with large language models. arXiv preprint arXiv:2404.07738 (2024).
61. Lehman, J. et al. Evolution through large models. In Handbook of evolutionary machine learning, 331–366 (Springer, 2023). find
62. van Stein, N. & Bäck, T. Llamea: A large language model evolutionary algorithm for automatically generating metaheuristics. IEEE Transactions on Evolutionary Computation (2024). find
63. Liu, F. et al. Evolution of heuristics: towards efficient automatic algorithm design using large language model. In Proceedings of the 41st International Conference on Machine Learning (2024). find
64. Ye, H., Wang, J., Cao, Z., Wang, H. & Song, G. ReEvo: Large language models as hyper-heuristics with reflective evolution. In Advances in Neural Information Processing Systems, vol. 37, 43571–43608 (2024). find
65. Meyerson, E., Francon, O., Sargent, D., Hodjat, B. & Miikkulainen, R. Unlocking the potential of global human expertise. Advances in Neural Information Processing Systems 37, 119227–119259 (2024). find
66. Lee, J. et al. Gemini Embedding: Generalizable embeddings from Gemini. arXiv preprint arXiv:2503.07891 (2025).
67. Gigante, S., Cannoodt, R. et al. openproblems (2025). https://github.com/openproblems-bio/ openproblems. ↗
68. Cannoodt, R., Zappia, L., Burkhardt, D. et al. task_batch_integration (2025). https://github.com/openproblems-bio/task_batch_integration. ↗
69. Akiba, T., Sano, S., Yanase, T., Ohta, T. & Koyama, M. Optuna: A next-generation hyperparameter optimization framework. In Proc. 25th ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. (2019).
70. Centers for Disease Control and Prevention. Weekly Hospital Respiratory Data (HRD) Metrics by Jurisdiction (2024). https://data.cdc.gov/Public-Health-Surveillance/ Weekly-Hospital-Respiratory-Data-HRD-Metrics-by-Ju/mpgq-jmmr. Dataset ID: mpgq-jmmr. Last updated: June 14, 2024. ↗
71. Yang, Y. & Newsam, S. Bag-of-visual-words and spatial extensions for land-use classification. In Proc. 18th SIGSPATIAL Int. Conf. on Adv. in Geogr. Inf. Syst., 270–279 (Association for Computing Machinery, 2010). find
72. Russakovsky, O. et al. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015).
73. Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25 (2012).
74. Immer, A. et al. Forecasting whole-brain neuronal activity from volumetric video. arXiv preprint arXiv:2503.00073 (2025). — [arXiv:2503.00073](https://arxiv.org/abs/2503.00073) find
75. Zeng, A., Chen, M., Zhang, L. & Xu, Q. Are transformers effective for time series forecasting? In Proc AAAI Conf. Artif. Intell., vol. 37, 11121–11128 (2023). find
76. Das, A. et al. Long-term forecasting with TiDE: Time-series Dense Encoder. Trans. Mach. Learn. Res. (2023). find
77. Chen, S.-A., Li, C.-L., Yoder, N., Arik, S. O. & Pfister, T. TSMixer: An All-MLP architecture for time series forecasting. Trans. Mach. Learn. Res. (2023). find
78. Perez, E., Strub, F., De Vries, H., Dumoulin, V. & Courville, A. FiLM: Visual reasoning with a general conditioning layer. In Proc AAAI Conf. Artif. Intell., vol. 32 (2018).
79. Deistler, M. et al. Differentiable simulation enables large-scale training of detailed biophysical models of neural dynamics. bioRxiv 2024–08 (2024). find
80. Hodgkin, A. L. & Huxley, A. F. A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. 117, 500 (1952). find
81. Piessens, R., de Doncker-Kapenga, E., Überhuber, C. W. & Kahaner, D. QUADPACK: a subroutine package for automatic integration (Springer-Verlag, 1983). find
82. Gradshteyn, I. & Ryzhik, I. Table of integrals, series, and products, 8th edn (Academic Press, 1994). find
83. Meurer, A. et al. SymPy: symbolic computing in Python. PeerJ Comput. Sci. 3, e103 (2017). find
84. Brenner, M. P., Cohen-Addad, V. & Woodruff, D. Solving an open problem in theoretical physics using ai-assisted discovery. arXiv preprint arXiv:2603.04735 (2026). — [arXiv:2603.04735](https://arxiv.org/abs/2603.04735) find
85. Lueckmann, J.-M., Jain, V. & Januszewski, M. Discovering mechanistic models of neural activity: System identification in an in silico zebrafish (2026). https://arxiv.org/abs/2602.04492. 2602.04492. — [arXiv:2602.04492](https://arxiv.org/abs/2602.04492) ↗
86. McInnes, L., Healy, J. & Melville, J. UMAP: uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426 (2018).
87. Ho, S. L. & Xie, M. The use of ARIMA models for reliability forecasting and analysis. Comput. Ind. Eng. 35, 213–216 (1998). find
88. Zhong, B., Du, J., Liu, M., Yang, A. & Wu, J. Region-enhancing network for semantic segmentation of remote-sensing imagery. Sensors 21 (2021). find
89. Zhang, Z., Liu, B. & Li, Y. FURSformer: semantic segmentation network for remote sensing images with fused heterogeneous features. Electronics 12 (2023). find
90. Atiampo, A. K. & Diédié, G. H. F. New fusion approach of spatial and channel attention for semantic segmentation of very high spatial resolution remote sensing images. Open J. Appl. Sci. 14, 288–319 (2024). find
91. Sun, Y., Bi, F., Gao, Y., Chen, L. & Feng, S. A multi-attention UNet for semantic segmentation in remote sensing images. Symmetry 14, 906 (2022). find
92. Elgamily, K. M., Mohamed, M. A., Abou-Taleb, A. M. & Ata, M. M. A novel W13 deep CNN structure for improved semantic segmentation of multiple objects in remote sensing imagery. Neural Comput. Appl. 37, 5397–5427 (2025). Code Availability find

Updated 1 November 2026

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/aygun-2025-ai-system-help-scientists/