Elamur Research⌕
☰
home / papers / 2024 / SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning

SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning

Paper · 2024 · Alireza Ghafarollahi and Markus J. Buehler

Summary

SciAgents addresses whether AI can carry out early-stage scientific ideation without human direction: surfacing non-obvious links across large bodies of literature and turning them into testable research proposals. Single LLMs tend to struggle here because they hallucinate outside their training scope and reason shallowly. The authors therefore pair an ontological knowledge graph with a team of role-specialised LLM agents. The graph was built from roughly 1,000 papers on biological materials. A path-sampling routine pulls subgraphs linking two concepts, and agents use these as in-context grounding to draft, expand, critique and refine hypotheses.

Two orchestration modes are compared:

  • Fixed pipeline: agent turns follow a scripted sequence.
  • Self-organising group chat: agents coordinate on their own, with optional human input and external tool calls, including a literature-based novelty check.

The output is a structured research plan with suggested modelling and experimental steps. The authors argue this shows that graph-grounded multi-agent reasoning can scale interdisciplinary hypothesis generation, especially in bio-inspired materials design.

Key points

  • The knowledge graph covers biological materials and was derived from about 1,000 scientific papers. Subgraphs sampled from paths between concepts serve as context for the agents.
  • Agents run on GPT-4 family models inside the AutoGen framework. The roles are Planner, Assistant, Ontologist, Scientist 1, Scientist 2 and Critic, and a human proxy agent enables human-in-the-loop input.
  • A "novelty assistant" queries the Semantic Scholar API three times with different keyword sets and reads the top 10 abstracts per query. It then scores the idea's novelty and feasibility on a 1–10 scale.
  • Each final report contains the hypothesis, expanded research aspects, a critical review, and priorities for molecular modelling and synthetic biology experiments. Reports are exported as PDF and CSV.
  • Code and data are released on GitHub (lamm-mit/SciAgentsDiscovery, lamm-mit/GraphReasoning).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksMaterials Science · 58 worksMaterials Chemistry · 58 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,948 works

Cite

Cite as Alireza Ghafarollahi and Markus J. Buehler. 2024. SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning. arXiv:2409.05556.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2409.05556 · arXiv 2409.05556

@misc{ghafarollahi2024sciagents,
  title = {{SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning}},
  author = {Alireza Ghafarollahi and Markus J. Buehler},
  year = {2024},
  doi = {10.48550/arxiv.2409.05556},
  eprint = {2409.05556},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2409.05556}
}

References

46 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. van der Zant, T., Kouw, M. & Schomaker, L. Generative artificial intelligence. Studies in Applied Philosophy, Epistemology and Rational Ethics 5, 107–120 (2013). URL https://link.springer.com/chapter/10.1007/978-3-642-31674-6_8. — [doi:10.1007/978-3-642-31674-6_8](https://doi.org/10.1007/978-3-642-31674-6_8) ↗
  2. Guo, K., Yang, Z., Yu, C.-H. & Buehler, M. J. Artificial intelligence and machine learning in design of mechanical materials. Materials Horizons 8, 1153–1172 (2021). find
  3. Liu, Y., Zhao, T., Ju, W. & Shi, S. Materials discovery and design using machine learning. Journal of Materiomics 3, 159–177 (2017). find
  4. Hu, Y. & Buehler, M. J. Deep language models for interpretative and predictive materials science. APL Machine Learning 1, 010901 (2023). URL https://aip.scitation.org/doi/abs/10.1063/5.0134317. — [doi:10.1063/5.0134317](https://doi.org/10.1063/5.0134317) ↗
  5. Matsumoto, M. Materials exploration: The next generation. MRS Bulletin 2022 1–2 (2022). URL https://link.springer.com/article/10.1557/s43577-022-00435-x. — [doi:10.1557/s43577-022-00435-x](https://doi.org/10.1557/s43577-022-00435-x) ↗
  6. Buehler, M. J. Accelerating scientific discovery with generative knowledge extraction, graph-based representation, and multimodal intelligent graph reasoning. Machine Learning: Science and Technology (2024). URL http://iopscience.iop.org/article/10.1088/2632-2153/ad7228. — [doi:10.1088/2632-2153/ad7228](https://doi.org/10.1088/2632-2153/ad7228) ↗
  7. Zhang, Q. et al. Large-language-model-based ai agent for organic semiconductor device research. Advanced Materials 36, 2405163 (2024). URL https://onlinelibrary.wiley.com/doi/abs/10.1002/adma.202405163. https://onlinelibrary.wiley.com/doi/pdf/10.1002/adma.202405163. — [doi:10.1002/adma.202405163](https://doi.org/10.1002/adma.202405163) ↗
  8. Lu, C. et al. The AI Scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292 (2024).
  9. Ng, W. L., Goh, G. L., Goh, G. D., Ten, J. S. J. & Yeong, W. Y. Progress and opportunities for machine learning in materials and processes of additive manufacturing. Advanced Materials 36, 2310006 (2024). URL https://onlinelibrary.wiley.com/doi/abs/10.1002/adma.202310006. https://onlinelibrary. wile… — [doi:10.1002/adma.202310006](https://doi.org/10.1002/adma.202310006) ↗
  10. Lei, G., Docherty, R. & Cooper, S. J. Materials science in the era of large language models: a perspective. Digital Discovery (2024). find
  11. OpenAI. GPT-4 technical report (2024). URL https://arxiv.org/abs/2303.08774. 2303.08774. ↗
  12. Vaswani, A. et al. Attention is all you need. Advances in neural information processing systems 30 (2017).
  13. Wei, J. et al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 (2022).
  14. Touvron, H. et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023).
  15. Teubner, T., Flath, C. M., Weinhardt, C., van der Aalst, W. & Hinz, O. Welcome to the era of chatgpt et al. the prospects of large language models. Business & Information Systems Engineering 65, 95–101 (2023). find
  16. Zhao, W. X. et al. A survey of large language models. arXiv preprint arXiv:2303.18223 (2023).
  17. Chowdhery, A. et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research 24, 1–113 (2023).
  18. Gunasekar, S. et al. Textbooks are all you need. arXiv preprint arXiv:2306.11644 (2023). find
  19. Jiang, A. Q. et al. Mistral 7b. arXiv preprint arXiv:2310.06825 (2023). find
  20. Girotra, K., Meincke, L., Terwiesch, C. & Ulrich, K. T. Ideas are dimes a dozen: Large language models for idea generation in innovation. Available at SSRN 4526071 (2023). find
  21. Buehler, M. J. Generative retrieval-augmented ontologic graph and multi-agent strategies for interpretive large language model-based materials design. ACS Engineering AU (2023). URL https://doi.org/10.1021/acsengineeringau.3c00058. — [doi:10.1021/acsengineeringau.3c00058](https://doi.org/10.1021/acsengineeringau.3c00058) ↗
  22. Jablonka, K. M. et al. 14 examples of how llms can transform materials science and chemistry: a reflection on a large language model hackathon. Digital Discovery 2, 1233–1250 (2023). find
  23. M. Bran, A. et al. Augmenting large language models with chemistry tools. Nature Machine Intelligence 1–11 (2024). find
  24. Luu, R. K. & Buehler, M. J. Bioinspiredllm: Conversational large language model for the mechanics of biological and bio-inspired materials. Advanced Science 11, 2306724 (2024). find
  25. Lu, W., Luu, R. K. & Buehler, M. J. Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities (2024). URL https://arxiv.org/abs/2409.03444. 2409.03444. — [arXiv:2409.03444](https://arxiv.org/abs/2409.03444) ↗
  26. Buehler, M. J. Cephalo: Multi-modal vision-language models for bio-inspired materials analysis and design. Advanced Functional Materials n/a, 2409531. URL https://onlinelibrary.wiley.com/doi/abs/10.1002/adfm.202409531. https://onlinelibrary.wiley.com/doi/pdf/10.1002/adfm.202409531. — [doi:10.1002/adfm.202409531](https://doi.org/10.1002/adfm.202409531) ↗
  27. Wei, J. et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35, 24824–24837 (2022).
  28. White, J. et al. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382 (2023).
  29. Zhou, Y. et al. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910 (2022).
  30. Sun, J. et al. Think-on-graph: Deep and responsible reasoning of large language model with knowledge graph. arXiv preprint arXiv:2307.07697 (2023). find
  31. Shetty, P. et al. A general-purpose material property data extraction pipeline from large polymer corpora using natural language processing. npj Computational Materials 9, 52 (2023). find
  32. Pan, S. et al. Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering (2024).
  33. Dagdelen, J. et al. Structured information extraction from scientific text with large language models. Nature Communications 15, 1418 (2024).
  34. Schilling-Wilhelmi, M. et al. From text to insight: Large language models for materials science data extraction. arXiv preprint arXiv:2407.16867 (2024). — [arXiv:2407.16867](https://arxiv.org/abs/2407.16867) find
  35. Ni, B. & Buehler, M. J. MechAgents: Large language model multi-agent collaborations can solve mechanics problems, generate new data, and integrate knowledge. Extreme Mechanics Letters 67, 102131 (2024). find
  36. Stewart, I. & Buehler, M. Molecular analysis and design using multimodal generative artificial intelligence via multi-agent modeling. ChemRxiv, 10.26434/chemrxiv-2024-nwm7n (2024). — [doi:10.26434/chemrxiv-2024-nwm7n](https://doi.org/10.26434/chemrxiv-2024-nwm7n) find
  37. Ghafarollahi, A. & Buehler, M. J. ProtAgents: Protein discovery via large language model multi-agent collaborations combining physics and machine learning. Digital Discovery 1389–1409 (2024). find
  38. Ghafarollahi, A. & Buehler, M. J. Atomagents: Alloy design and discovery through physics-aware multi-modal multi-agent artificial intelligence. arXiv preprint arXiv:2407.10022 (2024). — [arXiv:2407.10022](https://arxiv.org/abs/2407.10022) find
  39. Giesa, T., Spivak, D. & Buehler, M. Category theory based solution for the building block replacement problem in materials design. Advanced Engineering Materials 14 (2012). find
  40. Spivak, D., Giesa, T., Wood, E. & Buehler, M. Category theoretic analysis of hierarchical protein materials and social networks. PLoS ONE 6 (2011). find
  41. Wu, Q. et al. AutoGen: Enabling next-gen llm applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155 (2023).
  42. Wu, Y., Yue, T., Zhang, S., Wang, C. & Wu, Q. Stateflow: Enhancing llm task-solving through state-driven workflows (2024). 2403.11322.
  43. Wu, Y. et al. An empirical study on challenging math problem solving with gpt-4. In ArXiv preprint arXiv:2306.01337 (2023).
  44. OpenAI API. URL https://openai.com/blog/openai-api. ↗
  45. Dijkstra, E. W. A note on two problems in connexion with graphs. Numerische Mathematik 1, 269–271 (1959). find
  46. Wu, Q. et al. Autogen: Enabling next-gen LLM applications via multi-agent conversation. https://arxiv.org/abs/2308.08155 (2023). ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ghafarollahi-2024-sciagents-automating-scientific-discovery/