Elamur Research⌕
☰
home / papers / 2025 / DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience

DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience

Paper · 2025 · Runxiang Wang, Boxiao Wang, Kai Li et al.

Summary

The paper addresses symbolic regression: finding compact, interpretable formulas that explain observed data. Recent LLM-based systems such as LLM-SR lean mainly on the scientific knowledge stored inside the model. They rarely examine the data directly and do not learn from their own failures, so they often produce broken or numerically unstable expressions. DrSR (Dual Reasoning Symbolic Regression) keeps LLM-SR's loop of generating, fitting and scoring equations and adds two reasoning channels that use the same LLM backbone. The first has the model inspect sampled data and residuals and write structured notes on trends, nonlinearities and correlations, refining them whenever a better equation appears. The second sorts each new candidate by its outcome and turns those outcomes into reusable "ideas" kept in a growing library. On six benchmarks from physics, chemistry, biology and materials science, DrSR outperforms classical and LLM-based baselines in accuracy, generalization, convergence speed and the share of valid equations. This suggests LLM-driven equation discovery benefits from grounding in data and from self-reflection.

Key points

  • The architecture has three role-specific modules on one backbone: π_data analyzes the data, π_idea summarizes experience, and π_main generates equation skeletons, whose parameters are then optimized with BFGS.
  • π_data receives 100 uniformly sampled input-output pairs. In later rounds it also gets residuals from the current best equation, so its insights move from global patterns to local and higher-order structure.
  • Each candidate is labeled positive, negative or invalid (syntax errors, numerical faults, variable mismatches). Each label has its own prompt template for extracting heuristics.
  • The backbones were Mixtral-8x7B-Instruct-v0.1 and LLaMA3.1-8B-Instruct. On harder tasks such as E. coli Growth and Stress-Strain, DrSR kept about 0.4–0.6 valid equations, while LLM-SR stayed below 0.3.
  • Ablations on Oscillation 2 varied how often data insights are used (p = 0, 0.5, 1.0) and removed parts of the idea library. Every removal hurt performance, and turning off both mechanisms reduces DrSR to LLM-SR.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksPhysics and Astronomy · 70 worksStatistical and Nonlinear Physics · 60 worksMaterials Science · 58 worksMaterials Chemistry · 58 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,948 works

Cite

Cite as Runxiang Wang, Boxiao Wang, Kai Li, Yifan Zhang and Jian Cheng. 2025. DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience. arXiv:2506.04282.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2506.04282 · arXiv 2506.04282

@misc{wang2025drsr,
  title = {{DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience}},
  author = {Runxiang Wang, Boxiao Wang, Kai Li, Yifan Zhang and Jian Cheng},
  year = {2025},
  doi = {10.48550/arxiv.2506.04282},
  eprint = {2506.04282},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2506.04282}
}

References

50 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nour Makke and Sanjay Chawla. Interpretable scientific discovery with symbolic regression: a review. Artificial Intelligence Review, 57(1):2, 2024. find
  2. Pablo Lemos, Niall Jeffrey, Miles Cranmer, Shirley Ho, and Peter Battaglia. Rediscovering orbital mechanics with machine learning. Machine Learning: Science and Technology, 4(4):045002, 2023. find
  3. Benjamin L Davis and Zehao Jin. Discovery of a planar black hole mass scaling relation for spiral galaxies. The Astrophysical Journal Letters, 956(1):L22, 2023. find
  4. Rohit Batra, Le Song, and Rampi Ramprasad. Emerging materials intelligence ecosystems propelled by machine learning. Nature Reviews Materials, 6(8):655–678, 2021. find
  5. Alberto Hernandez, Adarsh Balasubramanian, Fenglin Yuan, Simon AM Mason, and Tim Mueller. Fast, accurate, and transferable many-body interatomic potentials by symbolic regression. npj Computational Materials, 5(1):112, 2019. find
  6. Shuo Yu, Hongyan Xue, Xiang Ao, Feiyang Pan, Jia He, Dandan Tu, and Qing He. Generating synergistic formulaic alpha collections via reinforcement learning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5476–5486, 2023. find
  7. Wenyan Xu, Rundong Wang, Chen Li, Yonghong Hu, and Zhonghua Lu. Hrft: Mining high-frequency risk factor collections end-to-end via transformer. arXiv preprint arXiv:2408.01271, 2024. — [arXiv:2408.01271](https://arxiv.org/abs/2408.01271) find
  8. Michael Schmidt and Hod Lipson. Distilling free-form natural laws from experimental data. science, 324(5923):81–85, 2009. find
  9. Miles Cranmer. Interpretable machine learning for science with pysr and symbolicregression. jl. arXiv preprint arXiv:2305.01582, 2023.
  10. Brenden K Petersen, Mikel Landajuela, T Nathan Mundhenk, Claudio P Santiago, Soo K Kim, and Joanne T Kim. Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients. arXiv preprint arXiv:1912.04871, 2019. — [arXiv:1912.04871](https://arxiv.org/abs/1912.04871) find
  11. Luca Biggio, Tommaso Bendinelli, Alexander Neitz, Aurelien Lucchi, and Giambattista Parascandolo. Neural symbolic regression that scales. In International Conference on Machine Learning, pages 936–945. Pmlr, 2021. find
  12. Pierre-Alexandre Kamienny, Stéphane d’Ascoli, Guillaume Lample, and François Charton. End-to-end symbolic regression with transformers. Advances in Neural Information Processing Systems, 35:10269–10281, 2022.
  13. Parshin Shojaee, Kazem Meidani, Shashank Gupta, Amir Barati Farimani, and Chandan K Reddy. Llm-sr: Scientific equation discovery via programming with large language models. arXiv preprint arXiv:2404.18400, 2024.
  14. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024. find
  15. Paul Kassianik, Baturay Saglam, Alexander Chen, Blaine Nelson, Anu Vellore, Massimo Aufiero, Fraser Burch, Dhruv Kedia, Avi Zohary, Sajana Weerawardhena, et al. Llama-3.1-foundationai-securityllm-base-8b technical report. arXiv preprint arXiv:2504.21039, 2025. — [arXiv:2504.21039](https://arxiv.org/abs/2504.21039) find
  16. Matteo Merler, Katsiaryna Haitsiukevich, Nicola Dainese, and Pekka Marttinen. In-context symbolic regression: Leveraging large language models for function discovery. arXiv preprint arXiv:2404.19094, 2024. find
  17. Parshin Shojaee, Ngoc-Hieu Nguyen, Kazem Meidani, Amir Barati Farimani, Khoa D Doan, and Chandan K Reddy. Llm-srbench: A new benchmark for scientific equation discovery with large language models. arXiv preprint arXiv:2504.10415, 2025.
  18. Monod and J. The growth of bacterial cultures. 3(1):371–394, 1949. find
  19. L Rosso, JR Lobry, S Bajard, and Jean-Pierre Flandrois. Convenient model to describe the combined effects of temperature and ph on microbial growth. Applied and environmental microbiology, 61(2):610–616, 1995. find
  20. Silviu-Marian Udrescu and Max Tegmark. Ai feynman: A physics-inspired method for symbolic regression. Science advances, 6(16):eaay2631, 2020.
  21. Mikel Landajuela, Chak Shing Lee, Jiachen Yang, Ruben Glatt, Claudio P Santiago, Ignacio Aravena, Terrell Mundhenk, Garrett Mulcahy, and Brenden K Petersen. A unified framework for deep symbolic regression. Advances in Neural Information Processing Systems, 35:33985– 33998, 2022. find
  22. Arya Grayeli, Atharva Sehgal, Omar Costilla Reyes, Miles Cranmer, and Swarat Chaudhuri. Symbolic regression with a learned concept library. Advances in Neural Information Processing Systems, 37:44678–44709, 2024.
  23. Donald Gerwin. Information processing, data inferences, and scientific generalization. Behavioral Science, 19(5):314–325, 1974. find
  24. Pat Langley. Bacon: A production system that discovers empirical laws. In IJCAI, page 344. Citeseer, 1977.
  25. John R Koza. Genetic programming as a means for programming computers by natural selection. Statistics and computing, 4:87–112, 1994. find
  26. Fangzheng Sun, Yang Liu, Jian-Xun Wang, and Hao Sun. Symbolic physics learner: Discovering governing equations via monte carlo tree search. arXiv preprint arXiv:2205.13134, 2022. — [arXiv:2205.13134](https://arxiv.org/abs/2205.13134) find
  27. Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the national academy of sciences, 113(15):3932–3937, 2016. find
  28. Kathleen Champion, Bethany Lusch, J Nathan Kutz, and Steven L Brunton. Data-driven discovery of coordinates and governing equations. Proceedings of the National Academy of Sciences, 116(45):22445–22451, 2019. find
  29. Georg Martius and Christoph H Lampert. Extrapolation and learning equations. arXiv preprint arXiv:1610.02995, 2016. — [arXiv:1610.02995](https://arxiv.org/abs/1610.02995) find
  30. Subham Sahoo, Christoph Lampert, and Georg Martius. Learning equations for extrapolation and control. In International Conference on Machine Learning, pages 4442–4450. Pmlr, 2018. find
  31. Samuel Kim, Peter Y Lu, Srijon Mukherjee, Michael Gilbert, Li Jing, Vladimir Čeperić, and Marin Soljačić. Integration of neural network-based symbolic regression in deep learning for scientific discovery. IEEE transactions on neural networks and learning systems, 32(9):4166– 4177, 2020. find
  32. Min Wu, Weijun Li, Lina Yu, Wenqiang Li, Jingyi Liu, Yanjie Li, and Meilan Hao. Prunesymnet: A symbolic neural network and pruning algorithm for symbolic regression. arXiv preprint arXiv:2401.15103, 2024. — [arXiv:2401.15103](https://arxiv.org/abs/2401.15103) find
  33. Yanjie Li, Weijun Li, Lina Yu, Min Wu, Jingyi Liu, Shu Wei, Yusong Deng, and Meilan Hao. Metasymnet: A tree-like symbol network with adaptive architecture and activation functions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 27081–27089, 2025. find
  34. John R Koza. Genetic programming: A paradigm for genetically breeding populations of computer programs to solve problems, volume 34. Stanford University, Department of Computer Science Stanford, CA, 1990. find
  35. Fabricio Olivetti de Franca and Guilherme Seidyo Imai Aldeia. Interaction–transformation evolutionary algorithm for symbolic regression. Evolutionary computation, 29(3):367–390, 2021. find
  36. Baihe He, Qiang Lu, Qingyun Yang, Jake Luo, and Zhiguang Wang. Taylor genetic programming for symbolic regression. In Proceedings of the genetic and evolutionary computation conference, pages 946–954, 2022. find
  37. Hengzhe Zhang, Aimin Zhou, Qi Chen, Bing Xue, and Mengjie Zhang. Sr-forest: a genetic programming based heterogeneous ensemble learning method. IEEE Transactions on Evolutionary Computation, 2023. find
  38. T Nathan Mundhenk, Mikel Landajuela, Ruben Glatt, Claudio P Santiago, Daniel M Faissol, and Brenden K Petersen. Symbolic regression via neural-guided genetic programming population seeding. arXiv preprint arXiv:2111.00053, 2021. — [arXiv:2111.00053](https://arxiv.org/abs/2111.00053) find
  39. Mikel Landajuela, Brenden K Petersen, Soo K Kim, Claudio P Santiago, Ruben Glatt, T Nathan Mundhenk, Jacob F Pettit, and Daniel M Faissol. Improving exploration in policy gradient search: Application to symbolic optimization. arXiv preprint arXiv:2107.09158, 2021. — [arXiv:2107.09158](https://arxiv.org/abs/2107.09158) find
  40. Laure Crochepierre, Lydia Boudjeloud-Assala, and Vincent Barbesant. Interactive reinforcement learning for symbolic regression from multi-format human-preference feedbacks. In IJCAI, pages 5900–5903, 2022. find
  41. Wenqiang Li, Weijun Li, Lina Yu, Min Wu, Linjun Sun, Jingyi Liu, Yanjie Li, Shu Wei, Yusong Deng, and Meilan Hao. A neural-guided dynamic symbolic network for exploring mathematical expressions from data. arXiv preprint arXiv:2309.13705, 2023. — [arXiv:2309.13705](https://arxiv.org/abs/2309.13705) find
  42. Mengge Du, Yuntian Chen, and Dongxiao Zhang. Discover: Deep identification of symbolically concise open-form partial differential equations via enhanced reinforcement learning. Physical Review Research, 6(1):013182, 2024. find
  43. Jacob F Pettit, Chak Shing Lee, Jiachen Yang, Alex Ho, Daniel Faissol, Brenden Petersen, and Mikel Landajuela. Disco-dso: Coupling discrete and continuous optimization for efficient generative design in hybrid spaces. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages… find
  44. Mojtaba Valipour, Bowen You, Maysum Panju, and Ali Ghodsi. Symbolicgpt: A generative transformer model for symbolic regression (2021). arXiv preprint arXiv:2106.14131, 2021. — [arXiv:2106.14131](https://arxiv.org/abs/2106.14131) find
  45. Martin Vastl, Jonáš Kulhánek, Jiří Kubalík, Erik Derner, and Robert Babuška. Symformer: End-to-end symbolic regression using transformer-based architecture. IEEE Access, 2024. find
  46. Wenqiang Li, Weijun Li, Linjun Sun, Min Wu, Lina Yu, Jingyi Liu, Yanjie Li, and Songsong Tian. Transformer-based model for symbolic regression via joint supervised learning. In The Eleventh International Conference on Learning Representations, 2022. find
  47. Zihan Yu, Jingtao Ding, and Yong Li. Symbolic regression via mdlformer-guided search: from minimizing prediction error to minimizing description length. arXiv preprint arXiv:2411.03753, 2024. — [arXiv:2411.03753](https://arxiv.org/abs/2411.03753) find
  48. Shizhe Ding, Boyang Xia, Milong Ren, and Dongbo Bu. Niert: Accurate numerical interpolation through unifying scattered data representations using transformer encoder. IEEE Transactions on Knowledge and Data Engineering, 2024. find
  49. Elliot Meyerson, Mark J Nelson, Herbie Bradley, Adam Gaier, Arash Moradi, Amy K Hoover, and Joel Lehman. Language model crossover: Variation through few-shot prompting. ACM Transactions on Evolutionary Learning, 4(4):1–40, 2024. find
  50. Ping Guo, Qingfu Zhang, and Xi Lin. Coevo: Continual evolution of symbolic solutions using large language models. arXiv preprint arXiv:2412.18890, 2024. — [arXiv:2412.18890](https://arxiv.org/abs/2412.18890) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2025-drsr-llm-based-scientific-equation/