Elamur Research⌕
☰
home / papers / 2024 / MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses

MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses

Paper · 2024 · Zonglin Yang, Wanhao Liu, Ben Gao et al.

Summary

The paper asks whether LLMs can independently propose new, valid chemistry hypotheses when given only a research background (a question and/or a short survey) from any subfield. Modeling a hypothesis directly from the background looks intractable. The authors therefore assume that most chemistry hypotheses arise from combining a background with one or more inspirations. From this they derive a formal decomposition into three subtasks: retrieving inspiration papers, composing hypotheses from them, and ranking the results.

MOOSE-Chem, an agentic multi-agent framework, implements this decomposition. It uses an evolutionary algorithm to vary how inspirations are applied, multi-step retrieval to gather several inspirations, and an efficient ranking stage.

For evaluation, the authors built TOMATO-Chem, a set of 51 top-venue chemistry papers released after January 2024 and annotated by chemistry PhD students. The LLM's training data ends in December 2023, which rules out data contamination. Choosing inspirations from up to 3,000 candidate papers, the system recovered many hypotheses that capture the ground truth's core ideas. The results suggest LLMs could support natural-science discovery and may encode knowledge links humans have not yet noticed.

Key points

  • Benchmark: TOMATO-Chem has 51 chemistry and material science papers (Nature, Science or similar level), all public only after January 2024. Each paper is split into:
    • background question and survey, each also in a "strict" variant with fewer hints;
    • 1–3 inspiration titles;
    • hypothesis, experiments and reasoning process.
  • Decomposition: P(hypothesis | research background) is broken into three subtasks: inspiration retrieval, hypothesis composition and hypothesis ranking. Ranking is introduced as an evaluation task that earlier work largely overlooked.
  • Method: Unlike the earlier social-science approach of Yang et al. (2024b), MOOSE-Chem adds evolutionary mutation of hypotheses, multi-step collection of several inspirations, and efficient ranking.
  • Evaluation setting: The LLM used training data up to December 2023 and looked for inspirations among as many as 3,000 chemistry papers. Experts graded generated hypotheses against the ground truth. Examples include Rh(I)-catalyzed [2+2+1] cycloaddition and photoredox reactions of phosphorus ylides, both scored 3 by experts.
  • Proposed assumption: Inspiration retrieval is inherently out-of-distribution, yet the LLM performed it surprisingly accurately. The authors take this as a sign that LLMs may hold latent scientific associations not yet recognized by researchers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Science and Technology of ChinaHefei · China152 works · 114,627 citations🎓Nanyang Technological UniversitySingapore · Singapore193 works · 27,739 citations🎓Singapore University of Technology and DesignSingapore · Singapore56 works · 11,583 citations🔬Shanghai Artificial Intelligence LaboratoryShanghai · China164 works · 13,544 citations🎓Wuhan UniversityWuhan · China52 works · 1,456 citations🎓UNSW SydneySydney · Australia40 works · 1,630 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 worksComputational Theory and Mathematics · 496 worksMaterials Science · 58 worksMaterials Chemistry · 58 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 4,145 works

Cite

Cite as Zonglin Yang, Wanhao Liu, Ben Gao, Tong Xie, Yuqiang Li, Wanli Ouyang, Soujanya Poria, Erik Cambria and Dongzhan Zhou. 2024. MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses. arXiv:2410.07076.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2410.07076 · arXiv 2410.07076

@misc{yang2024moose,
  title = {{MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses}},
  author = {Zonglin Yang, Wanhao Liu, Ben Gao, Tong Xie, Yuqiang Li, Wanli Ouyang, Soujanya Poria, Erik Cambria and Dongzhan Zhou},
  year = {2024},
  doi = {10.48550/arxiv.2410.07076},
  eprint = {2410.07076},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2410.07076}
}

References

41 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. Researchagent: Iterative research idea generation over scientific literature with large language models. arXiv preprint arXiv:2404.07738, 2024.
  2. Mathias Benedek, Tanja Könen, and Aljoscha C Neubauer. Associative abilities underlying creativity. Psychology of Aesthetics, Creativity, and the Arts, 6(3):273, 2012. find
  3. Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen-tau Yih, and Yejin Choi. Abductive commonsense reasoning. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020.
  4. OpenReview.net, 2020. URL https://openreview.net/forum?id= Byg1v1HKDB. ↗
  5. Daniil A. Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. Autonomous chemical research with large language models. Nat., 624(7992):570–578, 2023. doi: 10.1038/S41586-023-06792-0. URL https://doi.org/10.1038/s41586-023-06792-0. ↗
  6. Faxiang Bu, Yuqi Deng, Jie Xu, Dali Yang, Yan Li, Wu Li, and Aiwen Lei. Electrocatalytic reductive deuteration of arenes and heteroarenes. Nature, pp. 1–2, 2024. find
  7. Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. Universal self-consistency for large language model generation. CoRR, abs/2311.17311, 2023. doi: 10.48550/ARXIV.2311.17311. URL https://doi.org/10.48550/arXiv.2311.… ↗
  8. Peter Clark, Oyvind Tafjord, and Kyle Richardson. Transformers as soft reasoners over language. In Christian Bessiere (ed.), Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, pp. 3882–3890. ijcai.org, 2020. doi: 10.24963/ijcai.2020/537. URL https:…
  9. Mario Coccia. Why do nations produce science advances and new technology? society, 59:101124, 2019. find
  10. Technology in Joseph F Hair, Arthur H Money, Philip Samouel, and Mike Page. Research methods for business. Education+ Training, 49(4):336–337, 2007. find
  11. Arthur Koestler. The act of creation. London: Hutchinson, 1964. find
  12. Sandeep Kumar, Tirthankar Ghosal, Vinayak Goyal, and Asif Ekbal. Can large language models unlock novel scientific research ideas? arXiv preprint arXiv:2409.06185, 2024.
  13. Byung Cheol Lee and Jaeyeon Chung. An empirical investigation of the impact of chatgpt on creativity. Nature Human Behaviour, pp. 1–9, 2024. find
  14. Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Hugo Larochelle, Marc’Aurelio Ran…
  15. Ruochen Li, Teerth Patel, Qingyun Wang, and Xinya Du. Mlr-copilot: Autonomous machine learning research based on large language models agents. arXiv preprint arXiv:2408.14033, 2024.
  16. Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The AI scientist: Towards fully automated open-ended scientific discovery. CoRR, abs/2408.06292, 2024. doi: 10. 48550/ARXIV.2408.06292. URL https://doi.org/10.48550/arXiv.2408.06292. ↗
  17. Ziming Luo, Zonglin Yang, Zexin Xu, Wei Yang, and Xinya Du. LLM4SR: A survey on large language models for scientific research. CoRR, abs/2501.04306, 2025. doi: 10.48550/ARXIV. 2501.04306. URL https://doi.org/10.48550/arXiv.2501.04306. ↗
  18. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-… find
  19. Biqing Qi, Kaiyan Zhang, Haoxiang Li, Kai Tian, Sihang Zeng, Zhang-Ren Chen, and Bowen Zhou. Large language models are zero shot hypothesis proposers. CoLM, abs/2311.05965, 2024. doi: 10.48550/ARXIV.2311.05965. URL https://doi.org/10.48550/arXiv.2311. 05965. ↗
  20. Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emilien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi. Mathematical discoveries from program search with large language models. Nat.…
  21. Kaito Shibahara, Yoshihito Kayaki, Kairi Yamashiro, Yuki Nagashima, Kohei Fujii, and Ken Tanaka. Rh-catalysed enantioselective [2+ 2+ 1] cycloaddition reactions using three different 2π-components. Nature Synthesis, pp. 1–13, 2024. find
  22. Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can llms generate novel research ideas? a large-scale human study with 100+ nlp researchers. arXiv preprint arXiv:2409.04109, 2024.
  23. Henry Sprueill, Carl Edwards, Mariefel V. Olarte, Udishnu Sanyal, Heng Ji, and Sutanay Choudhury. Monte carlo thought search: Large language model querying for complex scientific reasoning in catalyst design. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Comput… find
  24. Henry W. Sprueill, Carl Edwards, Khushbu Agarwal, Mariefel V. Olarte, Udishnu Sanyal, Conrad Johnston, Hongbin Liu, Heng Ji, and Sutanay Choudhury. CHEMREASONER: heuristic search over a large language model’s knowledge space using quantum-chemical feedback. In Forty-first International Conference on… — ChemReasoner: Heuristic Search over a Large Language Model's Knowledge Space using Quantum-Chemical Feedback
  25. OpenReview.net, 2024. URL https://openreview.net/forum?id=3tJDnEszco. ↗
  26. Ryuhei Suzuki, Taiga Ando, Fritz Deufel, Kohsuke Ohmatsu, and Takashi Ooi. Photocatalytic carbyne reactivity of phosphorus ylides for three-component formal cycloaddition reactions. Nature Synthesis, pp. 1–7, 2024. find
  27. Don R Swanson. Undiscovered public knowledge. The Library Quarterly, 56(2):103–118, 1986. find
  28. Trieu H. Trinh, Yuhuai Wu, Quoc V. Le, He He, and Thang Luong. Solving olympiad geometry without human demonstrations. Nat., 625(7995):476–482, 2024. doi: 10.1038/S41586-023-06747-5. URL https://doi.org/10.1038/s41586-023-06747-5. ↗
  29. Vahe Tshitoyan, John Dagdelen, Leigh Weston, Alexander Dunn, Ziqin Rong, Olga Kononova, Kristin A. Persson, Gerbrand Ceder, and Anubhav Jain. Unsupervised word embeddings capture latent knowledge from materials science literature. Nat., 571(7763):95–98, 2019. doi: 10.1038/ S41586-019-1335-8. URL htt… — [doi:10.1038/s41586-019-1335-8](https://doi.org/10.1038/s41586-019-1335-8) find
  30. Jinpei Wang, Yuxin Song, Fanfei Yu, Yijun Zeng, Chenyang Wu, Xuezhi Qin, Liang Peng, Yitan Li, Yongsen Zhou, Ran Tao, et al. Ultrastrong, flexible thermogalvanic armor with a carnot-relative efficiency over 8%. Nature Communications, 15(1):6704, 2024a. find
  31. Qingyun Wang, Doug Downey, Heng Ji, and Tom Hope. Scimon: Scientific inspiration machines optimized for novelty. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok…
  32. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023.
  33. OpenReview.net, 2023. URL https://openreview.net/ forum?id=1PL1NIMMrw. ↗
  34. Tong Xie, Yuwei Wan, Haoran Wang, Ina Østrøm, Shaozhou Wang, Mingrui He, Rong Deng, Xinyuan Wu, Clara Grazian, Chunyu Kit, and Bram Hoex. Opinion mining by convolutional neural networks for maximizing discoverability of nanomaterials. J. Chem. Inf. Model., 64(7): 2746–2759, 2024. doi: 10.1021/ACS.JC… — [doi:10.1021/acs.jcim.3c00746](https://doi.org/10.1021/acs.jcim.3c00746) find
  35. Zonglin Yang, Xinya Du, Alexander M. Rush, and Claire Cardie. Improving event duration prediction via time-aware pre-training. In Trevor Cohn, Yulan He, and Yang Liu (eds.), Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020, volume EMNLP 2020 of… find
  36. Zonglin Yang, Xinya Du, Erik Cambria, and Claire Cardie. End-to-end case-based reasoning for commonsense knowledge base completion. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pp. 3509–3522, Dubrovnik, Croatia, May 2023a. Associatio… find
  37. Zonglin Yang, Xinya Du, Rui Mao, Jinjie Ni, and Erik Cambria. Logical reasoning over natural language as knowledge representation: A survey. In 1st Workshop on Natural Language Reasoning and Structured Explanations (@ACL 2023), 2023b.
  38. Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, Erik Cambria, Xiaodong Liu, Jianfeng Gao, and Furu Wei. Language models as inductive reasoners. In Yvette Graham and Matthew Purver (eds.), Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 20…
  39. Zonglin Yang, Xinya Du, Junxian Li, Jie Zheng, Soujanya Poria, and Erik Cambria. Large language models for automated open-domain scientific hypotheses discovery. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Tha…
  40. Ruiqi Zhong, Peter Zhang, Steve Li, Jinwoo Ahn, Dan Klein, and Jacob Steinhardt. Goal driven discovery of distributional differences via language descriptions. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing…
  41. Research A.2 P ROOF OF THE R IGOROUS D ECOMPOSITION OF P (h \| b) B ASED ON THE F UNDAMENTAL A SSUMPTION Question (Ranked) \#\# Background Chemistry Survey Research Inspiration Hypotheses Corpus find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2024-moose-chem-large-language/