Elamur Research⌕
☰
home / papers / 2023 / Emergent autonomous scientific research capabilities of large language models

Emergent autonomous scientific research capabilities of large language models

Paper · 2023 · Daniil A. Boiko, Robert MacKnight and Gabe Gomes

Summary

The paper asks whether large language models can do more than answer chemistry questions and instead run experimental research largely on their own. The authors build an LLM-driven "Agent" whose central GPT-4 Planner coordinates several helper modules. These modules handle Google-based web search (summarised by GPT-3.5), retrieval over hardware documentation, sandboxed Python execution in Docker, and an automation layer that sends generated code to lab equipment. To get around GPT-4's September 2021 knowledge cutoff, documentation is supplied at run time through embedding-based search or through summarised function guides. The system plans syntheses of common drugs, finds the right functions in an unfamiliar cloud-lab language, and steers an Opentrons liquid handler from plain-language instructions. Its hardest task combines several instruments to carry out palladium-catalysed cross-coupling reactions, and GC-MS confirms the products. The work shows that general-purpose LLMs connected to tools can close the loop from planning to physical experiment, which raises both opportunities and misuse concerns. The authors examine these risks and suggest safeguards.

Key points

  • Architecture: a Planner chooses among four actions (GOOGLE, PYTHON, DOCUMENTATION, EXPERIMENT). Code and instrument outputs are fed back so the Planner can fix its own errors.
  • Documentation retrieval:
    • For the OT-2 Python API, ada embeddings were built for all 14 documentation sections, and relevant sections are selected by vector distance within a 7,800-token budget.
    • For Emerald Cloud Lab's Symbolic Lab Language (SLL), which GPT-4 did not know, a separate GPT-4 model condenses the function documentation.
  • Synthesis planning:
    • For ibuprofen, the Agent correctly proposed the opening Friedel-Crafts acylation step.
    • Aspirin was planned well.
    • The aspartame plan omitted the methyl group.
  • UV-Vis task: the Agent located red, yellow and blue samples on a plate from their absorbance maxima (530, 430 and 630 nm).
  • Cross-coupling: the Agent executed Suzuki and Sonogashira reactions, and GC-MS showed product peaks at m/z 154 and 178. The Nature version names the system Coscientist.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States598 works · 191,225 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksMaterials Science · 31 worksMaterials Chemistry · 31 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Daniil A. Boiko, Robert MacKnight and Gabe Gomes. 2023. Emergent autonomous scientific research capabilities of large language models. arXiv:2304.05332 · published as "Autonomous chemical research with large language models", Nature (2023), doi:10.1038/s41586-023-06792-0.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2304.05332 · arXiv 2304.05332

@misc{boiko2023emergent,
  title = {{Emergent autonomous scientific research capabilities of large language models}},
  author = {Daniil A. Boiko, Robert MacKnight and Gabe Gomes},
  year = {2023},
  doi = {10.48550/arxiv.2304.05332},
  eprint = {2304.05332},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2304.05332}
}

References

18 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Brown, T. et al. Language Models are Few-Shot Learners. in Advances in Neural Information Processing Systems (eds. Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F. & Lin, H.) vol. 33 1877–1901 (Curran Associates, Inc., 2020).
  2. Thoppilan, R. et al. LaMDA: Language Models for Dialog Applications. (2022).
  3. Touvron, H. et al. LLaMA: Open and Efficient Foundation Language Models. (2023).
  4. Hoffmann, J. et al. Training Compute-Optimal Large Language Models. (2022).
  5. Chowdhery, A. et al. PaLM: Scaling Language Modeling with Pathways. (2022).
  6. Lin, Z. et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science (1979) 379, 1123–1130 (2023). find
  7. Luo, R. et al. BioGPT: generative pre-trained transformer for biomedical text generation and mining. Brief Bioinform 23, (2022). find
  8. Irwin, R., Dimitriadis, S., He, J. & Bjerrum, E. J. Chemformer: a pre-trained transformer for computational chemistry. Mach Learn Sci Technol 3, 015022 (2022). find
  9. Kim, H., Na, J. & Lee, W. B. Generative Chemical Transformer: Neural Machine Learning of Molecular Geometric Structures from Chemical Language via Attention. J Chem Inf Model 61, 5804–5814 (2021). find
  10. OpenAI. GPT-4 Technical Report. (2023).
  11. Ziegler, D. M. et al. Fine-Tuning Language Models from Human Preferences. (2019). find
  12. Ouyang, L. et al. Training language models to follow instructions with human feedback. (2022).
  13. Bran, A. M., Cox, S., White, A. D. & Schwaller, P. ChemCrow: Augmenting large-language models with chemistry tools. (2023).
  14. Reaxys. https://www.reaxys.com. ↗
  15. SciFinder. https://scifinder.cas.org. ↗
  16. Opentrons. Opentrons Python Protocol API. https://docs.opentrons.com/v2/. ↗
  17. Emerald Cloud Lab. Running https://www.emeraldcloudlab.com/guides/runningexperiments. Experiments. ↗
  18. Urbina, F., Lentzos, F., Invernizzi, C. & Ekins, S. Dual use of artificial-intelligence-powered drug discovery. Nat Mach Intell 4, 189–191 (2022). find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/boiko-2023-emergent-autonomous-scientific-research/