Elamur Research⌕
☰
home / papers / 2024 / LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery

LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery

Paper · 2024 · Pingchuan Ma, Tsun-Hsuan Wang, Minghao Guo et al.

Summary

Pretrained LLMs hold broad scientific knowledge and can reason about hypotheses. However, they cannot reliably compute what a proposed physical law predicts or check it against observations. The authors address this with Scientific Generative Agent (SGA), which couples an LLM with a differentiable simulator in a bilevel optimization loop. In the outer level, the LLM reviews the best earlier candidates stored in a top-K heap. It then writes new candidates as Python code, changing both the discrete structure (an equation or a molecular structure) and the set of continuous parameters to tune. In the inner level, gradient descent through the simulation fits those parameters, such as material stiffness or atom coordinates, and reports loss and feedback. Sampling at both low and high temperature balances refining known solutions against riskier exploration. SGA was tested on two tasks: discovering constitutive laws from motion trajectories and designing molecules with target quantum mechanical properties. It beats other LLM-based baselines and improves almost monotonically. It also sometimes finds unconventional solutions that domain experts judged plausible. Adapting it to a new field mostly means rewriting prompts.

Key points

  • Published at ICML 2024 (PMLR 235) by authors from MIT CSAIL, CMU LTI, UMass Amherst and the MIT-IBM Watson AI Lab.
  • Formulation: the outer problem searches over a symbolic expression E and a parameterization Θ, subject to the constraint that E can actually be simulated. The inner problem finds the optimal θ̂ through a differentiable simulator Φ.
  • Algorithm: the heap is seeded with an initial guess, for example code for a purely elastic material when the true behavior is a weakly compressible fluid. Each iteration draws M_l "exploit" samples at temperature T_l and M_h "explore" samples at T_h. After N iterations, the top-1 solution is returned.
  • The authors place SGA alongside OPRO, FunSearch and AlphaGeometry. They also note it resembles neural architecture search, where a discrete outer search wraps a continuous inner optimization.
  • Limitations:
    • One task costs about $10 with GPT-4.
    • Generated code runs without filtering, which poses a safety risk.
    • Results depend on the code being differentiable, so the authors suggest zero-order optimizers as an alternative.
    • LLM-generated solutions are hard to interpret.
    • Reusing the KV cache across iterations could speed the method up.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksPhysics and Astronomy · 70 worksStatistical and Nonlinear Physics · 60 worksMaterials Science · 58 worksMaterials Chemistry · 58 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 4,145 works

Cite

Cite as Pingchuan Ma, Tsun-Hsuan Wang, Minghao Guo, Zhiqing Sun, Joshua B. Tenenbaum, Daniela Rus, Chuang Gan and Wojciech Matusik. 2024. LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery. arXiv:2405.09783.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2405.09783 · arXiv 2405.09783

@misc{ma2024llm,
  title = {{LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery}},
  author = {Pingchuan Ma, Tsun-Hsuan Wang, Minghao Guo, Zhiqing Sun, Joshua B. Tenenbaum, Daniela Rus, Chuang Gan and Wojciech Matusik},
  year = {2024},
  doi = {10.48550/arxiv.2405.09783},
  eprint = {2405.09783},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2405.09783}
}

References

71 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AI4Science, M. R. and Quantum, M. A. The impact of large language models on scientific discovery: a preliminary study using gpt-4. arXiv preprint arXiv:2311.07361, 2023. — [arXiv:2311.07361](https://arxiv.org/abs/2311.07361) find
  2. Colson, B., Marcotte, P., and Savard, G. An overview of bilevel optimization. Annals of operations research, 153: 235–256, 2007. find
  3. Du, T., Wu, K., Ma, P., Wah, S., Spielberg, A., Rus, D., and Matusik, W. Diffpd: Differentiable projective dynamics. ACM Transactions on Graphics (TOG), 41(2):1–21, 2021. find
  4. Anthropic. Introducing the next generation of claude, 2024. URL https://www.anthropic.com/ news/claude-3-family. ↗
  5. Fang, X., Liu, L., Lei, J., He, D., Zhang, S., Zhou, J., Wang, F., Wu, H., and Wang, H. Geometry-enhanced molecular representation learning for property prediction. Nature Machine Intelligence, 4(2):127–134, 2022. find
  6. Arnaldo, I., Krawiec, K., and O’Reilly, U.-M. Multiple regression genetic programming. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, pp. 879–886, 2014. find
  7. Fortunato, S., Bergstrom, C. T., Börner, K., Evans, J. A., Helbing, D., Milojević, S., Petersen, A. M., Radicchi, F., Sinatra, R., Uzzi, B., et al. Science of science. Science, 359(6379):eaao0185, 2018. find
  8. Bender, G., Kindermans, P.-J., Zoph, B., Vasudevan, V., and Le, Q. Understanding and simplifying one-shot architecture search. In International conference on machine learning, pp. 550–559. PMLR, 2018. find
  9. Halgren, T. A. Merck molecular force field. i. basis, form, scope, parameterization, and performance of mmff94. Journal of computational chemistry, 17(5-6):490–519, 1996. find
  10. Biggio, L., Bendinelli, T., Neitz, A., Lucchi, A., and Parascandolo, G. Neural symbolic regression that scales. In International Conference on Machine Learning, pp. 936– 945. Pmlr, 2021. find
  11. Hansen, N. The cma evolution strategy: a comparing review. Towards a new evolutionary computation: Advances in the estimation of distribution algorithms, pp. 75–102, 2006. find
  12. Boiko, D. A., MacKnight, R., Kline, B., and Gomes, G. Autonomous chemical research with large language models. Nature, 624(7992):570–578, 2023.
  13. Huang, Q., Vora, J., Liang, P., and Leskovec, J. Benchmarking large language models as ai research agents. arXiv preprint arXiv:2310.03302, 2023.
  14. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877–1901, 2020.
  15. Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., Casas, D. d. l., Hanna, E. B., Bressand, F., et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024. find
  16. Cai, H., Zhu, L., and Han, S. ProxylessNAS: Direct neural architecture search on target task and hardware. In International Conference on Learning Representations, 2019.
  17. Jiang, C., Schroeder, C., Teran, J., Stomakhin, A., and Selle, A. The material point method for simulating continuum materials. In Acm siggraph 2016 courses, pp. 1–52. 2016. find
  18. Jin, W., Barzilay, R., and Jaakkola, T. Junction tree variational autoencoder for molecular graph generation. In International conference on machine learning, pp. 2323– 2332. PMLR, 2018. find
  19. Liu, R., Mu, P., Yuan, X., Zeng, S., and Zhang, J. A general descent aggregation framework for gradient-based bi-level optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1):38–57, 2022. find
  20. Jin, Y., Fu, W., Kang, J., Guo, J., and Guo, J. Bayesian symbolic regression. arXiv preprint arXiv:1910.08892, 2019. — [arXiv:1910.08892](https://arxiv.org/abs/1910.08892) find
  21. Liu, Z., Roberts, R. A., Lal-Nag, M., Chen, X., Huang, R., and Tong, W. Ai-based language models powering drug discovery and development. Drug Discovery Today, 26 (11):2593–2607, 2021. find
  22. Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.-Y. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30, 2017. find
  23. Ma, P., Du, T., Tenenbaum, J. B., Matusik, W., and Gan, C. Risp: Rendering-invariant state predictor with differentiable simulation and rendering for cross-domain parameter estimation. In International Conference on Learning Representations, 2021. find
  24. Kingma, D. and Ba, J. Adam: A method for stochastic optimization. In International Conference on Learning Representations, San Diega, CA, USA, 2015.
  25. Ma, P., Chen, P. Y., Deng, B., Tenenbaum, J. B., Du, T., Gan, C., and Matusik, W. Learning neural constitutive laws from motion observations for generalizable pde dynamics. In International Conference on Machine Learning. PMLR, 2023. find
  26. Kommenda, M., Burlacu, B., Kronberger, G., and Affen-zeller, M. Parameter identification for symbolic regression using nonlinear least squares. Genetic Programming and Evolvable Machines, 21(3):471–501, 2020. find
  27. Ma, Y. J., Liang, W., Wang, G., Huang, D.-A., Bastani, O., Jayaraman, D., Zhu, Y., Fan, L., and Anandkumar, A. Eureka: Human-level reward design via coding large language models. In International Conference on Learning Representations, 2024.
  28. Kramer, S., Cerrato, M., Džeroski, S., and King, R. Automated scientific discovery: From equation discovery to autonomous discovery systems. arXiv preprint arXiv:2305.02251, 2023.
  29. Macklin, M. Warp: A high-performance python framework for gpu simulation and graphics, March 2022. NVIDIA GPU Technology Conference. find
  30. La Cava, W., Helmuth, T., Spector, L., and Moore, J. H. A probabilistic and multi-objective analysis of lexicase selection and ε-lexicase selection. Evolutionary Computation, 27(3):377–402, 2019. find
  31. McConaghy, T. Ffx: Fast, scalable, deterministic symbolic regression technology. Genetic Programming Theory and Practice IX, pp. 235–260, 2011. find
  32. La Cava, W., Orzechowski, P., Burlacu, B., de Franca, F., Virgolin, M., Jin, Y., Kommenda, M., and Moore, J. Contemporary symbolic regression methods and their relative performance. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, 2021.
  33. Mundhenk, T. N., Landajuela, M., Glatt, R., Santiago, C. P., Faissol, D. M., and Petersen, B. K. Symbolic regression via neural-guided genetic programming population seeding. In Advances in Neural Information Processing Systems, 2021. find
  34. Landrum, G. et al. Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling. Greg Landrum, 8:31, 2013. find
  35. OpenAI. OpenAI: Introducing ChatGPT, 2022. URL https://openai.com/blog/chatgpt. ↗
  36. Li, J., Liu, Y., Fan, W., Wei, X.-Y., Liu, H., Tang, J., and Li, Q. Empowering molecule discovery for molecule-caption translation with large language models: A chatgpt perspective. arXiv preprint arXiv:2306.06615, 2023. — [arXiv:2306.06615](https://arxiv.org/abs/2306.06615) find
  37. OpenAI. OpenAI: GPT-4, 2023. URL https://openai.com/research/gpt-4. ↗
  38. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022.
  39. Li, W., Li, W., Sun, L., Wu, M., Yu, L., Liu, J., Li, Y., and Tian, S. Transformer-based model for symbolic regression via joint supervised learning. In The Eleventh International Conference on Learning Representations, 2022. find
  40. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
  41. Liu, H., Simonyan, K., and Yang, Y. DARTS: Differentiable architecture search. In International Conference on Learning Representations, 2019. find
  42. Petersen, B. K., Larma, M. L., Mundhenk, T. N., Santiago, C. P., Kim, S. K., and Kim, J. T. Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients. In International Conference on Learning Representations, 2020. find
  43. Udrescu, S.-M., Tan, A., Feng, J., Neto, O., Wu, T., and Tegmark, M. Ai feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity. Advances in Neural Information Processing Systems, 33:4860–4871, 2020. find
  44. Valipour, M., You, B., Panju, M., and Ghodsi, A. Symbolicgpt: A generative transformer model for symbolic regression. arXiv preprint arXiv:2106.14131, 2021. — [arXiv:2106.14131](https://arxiv.org/abs/2106.14131) find
  45. Popper, K. The logic of scientific discovery. Routledge, 2005. find
  46. Ramakrishnan, R., Dral, P. O., Rupp, M., and Von Lilienfeld, O. A. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):1–7, 2014. find
  47. Virgolin, M., Alderliesten, T., and Bosman, P. A. Linear scaling with and within semantic backpropagation-based genetic programming for symbolic regression. In Proceedings of the genetic and evolutionary computation conference, pp. 1084–1092, 2019. find
  48. Riniker, S. and Landrum, G. A. Better informed distance geometry: using what we know to improve conformation generation. Journal of chemical information and modeling, 55(12):2562–2574, 2015. find
  49. Virgolin, M., Alderliesten, T., Witteveen, C., and Bosman, P. A. Improving model-based genetic programming for symbolic regression of small expressions. Evolutionary computation, 29(2):211–237, 2021. find
  50. Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J., Ellenberg, J. S., Wang, P., Fawzi, O., et al. Mathematical discoveries from program search with large language models. Nature, pp. 1–3, 2023.
  51. Wang, H., Fu, T., Du, Y., Gao, W., Huang, K., Liu, Z., Chandak, P., Liu, S., Van Katwyk, P., Deac, A., et al. Scientific discovery in the age of artificial intelligence. Nature, 620(7972):47–60, 2023. find
  52. Rosenberg, A. and McIntyre, L. Philosophy of science: A contemporary introduction. Routledge, 2019. find
  53. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35: 24824–24837, 2022.
  54. Schapire, R. E. The boosting approach to machine learning: An overview. Nonlinear estimation and classification, pp. 149–171, 2003. find
  55. Schneider, G. Automating drug discovery. Nature reviews drug discovery, 17(2):97–113, 2018. find
  56. Weininger, D. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28(1):31–36, 1988. find
  57. Sharma, G. and Thakur, A. Chatgpt in drug discovery. 2023. find
  58. Sinha, A., Malo, P., and Deb, K. Evolutionary algorithm for bilevel optimization using approximations of the lower level optimal solution mapping. European Journal of Operational Research, 257(2):395–411, 2017a. find
  59. Wuestman, M., Hoekman, J., and Frenken, K. A typology of scientific breakthroughs. Quantitative Science Studies, 1(3):1203–1222, 2020. find
  60. Sinha, A., Malo, P., and Deb, K. A review on bilevel optimization: From classical to evolutionary approaches and applications. IEEE Transactions on Evolutionary Computation, 22(2):276–295, 2017b. find
  61. Xue, C., Wang, X., Yan, J., Hu, Y., Yang, X., and Sun, K. Rethinking bi-level optimization in neural architecture search: A gibbs sampling perspective. In AAAI Conference on Artificial Intelligence, volume 35, pp. 10551– 10559, 2021. find
  62. Sulsky, D., Zhou, S.-J., and Schreyer, H. L. Application of a particle-in-cell method to solid mechanics. Computer physics communications, 87(1-2):236–252, 1995. find
  63. Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., and Chen, X. Large language models as optimizers. In International Conference on Learning Representations, 2024.
  64. Sumers, T., Yao, S., Narasimhan, K., and Griffiths, T. Cognitive architectures for language agents. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. Survey Certification. find
  65. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., and Narasimhan, K. R. Tree of thoughts: Deliberate problem solving with large language models. In Conference on Neural Information Processing Systems, 2023a. find
  66. Trinh, T. H., Wu, Y., Le, Q. V., He, H., and Luong, T. Solving olympiad geometry without human demonstrations. Nature, 625(7995):476–482, 2024.
  67. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations, 2023b.
  68. Yoshikawa, N., Terayama, K., Sumita, M., Homma, T., Oono, K., and Tsuda, K. Population-based de novo molecule generation, using grammatical evolution. Chemistry Letters, 47(11):1431–1434, 2018. find
  69. Zheng, L., Yin, L., Xie, Z., Huang, J., Sun, C., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., et al. Efficiently programming large language models using sglang. arXiv preprint arXiv:2312.07104, 2023.
  70. Zhou, G., Gao, Z., Ding, Q., Zheng, H., Xu, H., Wei, Z., Zhang, L., and Ke, G. Uni-mol: A universal 3d molecular representation learning framework. In International Conference on Learning Representations, 2023. find
  71. Zhou, Z., Kearnes, S., Li, L., Zare, R. N., and Riley, P. Optimization of molecules via deep reinforcement learning. Scientific reports, 9(1):10752, 2019. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ma-2024-llm-simulation-bilevel-optimizers-new/