Elamur Research
☰
home / papers / 2025 / From System 1 to System 2: A Survey of Reasoning Large Language Models

From System 1 to System 2: A Survey of Reasoning Large Language Models

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2502.17419

Summary

Conventional LLMs respond quickly and intuitively, much like the fast "System 1" of dual-process cognitive theory, but they often fail on problems that require careful multi-step analysis. This survey examines how newer reasoning models, such as OpenAI's o1/o3 and DeepSeek's R1, approximate deliberate "System 2" thinking. The authors trace these models to two roots: the growth of foundational LLMs and older System 2 techniques, namely symbolic logic systems, Monte Carlo Tree Search (MCTS) and Reinforcement Learning (RL). They characterize reasoning LLMs by output behavior and training dynamics. They then sort the ways of building these models into five families: searching over reasoning structures, learning reward models, having models improve themselves, operating with macro-level actions, and fine-tuning through reinforcement. A benchmarking part reviews text-only and multimodal reasoning benchmarks and evaluation metrics, and compares reasoning models with their foundational counterparts. Later sections cover large-scale RL training, safety, agents, adaptive and efficient reasoning, and open limitations. The survey's value is breadth: earlier reviews focused only on test-time methods or RL, while this one maps the whole area. A continuously updated GitHub repository accompanies it.

Key points

  • The survey organizes its core methods into five categories: search over reasoning structures, reward models, self-driven improvement, macro-level actions, and reinforcement-based fine-tuning. Each category comes with representative reasoning LLMs.
  • It names three early System 2 technologies that, combined with foundational LLMs, led to reasoning LLMs: symbolic logic, MCTS and RL.
  • The background covers foundational models including BERT, GPT-2 (1.5B parameters), GPT-3 (175B), CLIP, DALL-E, GPT-4/4o, LLaMA, LLaVA and DeepSeek-V3 (671B Mixture-of-Experts).
  • The evaluation covers both plain-text and multimodal benchmarks and compares reasoning LLMs against their base models. The provided extract does not include the actual comparison scores.
  • This summary is based on version v6 (25 Jun 2025). That version adds sections on large-scale RL, safety of reasoning models, agent integration, and adaptive/efficient reasoning, and it is backed by a live GitHub tracker.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Cited by

143 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 141 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2502.17419 · arXiv 2502.17419

@misc{li2025system,
  title = {{From System 1 to System 2: A Survey of Reasoning Large Language Models}},
  author = {Li, Zhong-Zhi and Zhang, Duzhen and Zhang, Ming-Liang and Zhang, Jiaxin and Liu, Zengyan and Yao, Yuxuan and Xu, Haotian and Zheng, Junhao and Wang, Pei-Jie and Chen, Xiuyi and Zhang, Yingying and Yin, Fei and Dong, Jiahua and Li, Zhiwei and Bi, Bao-Long and Mei, Ling-Rui and Fang, Junfeng and Liang, Xiao and Guo, Zhijiang and Song, Le and Liu, Cheng-Lin},
  year = {2025},
  doi = {10.48550/arxiv.2502.17419},
  eprint = {2502.17419},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2502.17419}
}

References

23 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. in International conference on machine learning. Pmlr, 2021, pp. 8821–8831. OpenAI, “GPT-4 Technical Report,” 2023. J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds et al., “Flamingo: a visual language model for few-shot learning,” Advance… find
  2. PMLR, 2022, pp. 5547–5569. D. Dai, C. Deng, C. Zhao, R. Xu, H. Gao, D. Chen, J. Li, W. Zeng, X. Yu, Y. Wu et al., “DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V… find
  3. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020. [84] S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer, “Reth… find our card
  4. Courier Corporation, 2012. [90] A. Colmerauer, “An introduction to Prolog III,” Communications of the ACM, vol. 33, no. 7, pp. 69–90, 1990. [91] W. F. Clocksin and C. S. Mellish, Programming in PROLOG. find
  5. Springer Science & Business Media, 2003. [92] K. R. Apt et al., From logic programming to Prolog. find
  6. Prentice Hall London, 1997, vol. 362. [93] M. P. Singh, A. S. Rao, and M. P. Georgeff, Formal methods in DAI: Logic-based representation and reasoning. find
  7. MIT Press Cambridge, 1999, vol. 8. [94] R. G. Jeroslow, “Computation-oriented reductions of predicate to propositional logic,” Decision Support Systems, vol. 4, no. 2, pp. 183–197, 1988. [95] J. McCarthy, “History of LISP,” in History of programming languages, 1978, pp. 173–185. [96] L. Bachmair and… find
  8. IEEE, 2018, pp. 1–8. [105] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot et al., “Mastering the game of Go with deep neural networks and tree search,” nature, vol. 529, no. 7587, pp. 484–489, 2016. [106] D… find
  9. [107] O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev et al., “Grandmaster level in StarCraft II using multi-agent reinforcement learning,” nature, vol. 575, no. 7782, pp. 350– 354, 2019. [108] A. Y. Ng, D. Harada, and S. Rus… find
  10. Citeseer, 1999, pp. 278–287. [109] H. Luo, Q. Sun, C. Xu, P. Zhao, J. Lou, C. Tao, X. Geng, Q. Lin, S. Chen, and D. Zhang, “Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct,” arXiv preprint arXiv:2308.09583, 2023. [110] Z. Shao, P. Wang, Q. Zhu, R.… find our card
  11. [128] J. Li, H. Le, Y. Zhou, C. Xiong, S. Savarese, and D. Sahoo, “CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models,” CoRR, vol. abs/2411.04329, 2024. [129] J. Qiu, Y. Lu, Y. Zeng, J. Guo, J. Geng, H. Wang, K. Huang, Y. Wu, and M. Wang, “TreeBoN: Enhancing Inference-… find
  12. [148] J. Liu, A. Cohen, R. Pasunuru, Y. Choi, H. Hajishirzi, and A. Celikyilmaz, “Don’t throw away your value model! generating more preferable text with value-guided monte-carlo tree search decoding,” in First Conference on Language Modeling, 2024. [Online]. Available: https://openreview.net/forum?… ↗
  13. OpenReview.net, 2024. [195] Y. Dubois, C. X. Li, R. Taori, T. Zhang, I. Gulrajani, J. Ba, C. Guestrin, P. Liang, and T. B. Hashimoto, “AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback,” in Advances in Neural Information Processing Systems 36: Annual Conference on Neural… find our card
  14. OpenReview.net, 2024. [207] Z. Gou, Z. Shao, Y. Gong, Y. Shen, Y. Yang, N. Duan, and W. Chen, “CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing,” in The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. find our card
  15. OpenReview.net, 2024. [208] D. Huang, J. Dai, H. Weng, P. Wu, Y. Qing, H. Cui, Z. Guo, and J. Zhang, “Effilearner: Enhancing efficiency of generated code via self-optimization,” in Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024,… find
  16. PMLR, 2023, pp. 10 764– 10 799. [234] T. Khot, H. Trivedi, M. Finlayson, Y. Fu, K. Richardson, P. Clark, and A. Sabharwal, “Decomposed prompting: A modular approach for solving complex tasks,” arXiv preprint arXiv:2210.02406, 2022. [235] D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Sc… find
  17. [248] K. Gandhi, D. Lee, G. Grand, M. Liu, W. Cheng, A. Sharma, and N. D. Goodman, “Stream of search (sos): Learning to search in language, 2024,” URL https://arxiv. org/abs/2404.03683, 2024. [249] S. Leng, J. Wang, J. Li, H. Zhang, Z. Hu, B. Zhang, H. Zhang, Y. Jiang, X. Li, D. Zhao, F. Wang, Y. Ro… find
  18. [269] Z. Xie, L. Chen, W. Mao, J. Xu, L. Kong et al., “Teaching language models to critique via reinforcement learning,” arXiv preprint arXiv:2502.03492, 2025. [270] T. Xie, Z. Gao, Q. Ren, H. Luo, Y. Hong, B. Dai, J. Zhou, K. Qiu, Z. Wu, and C. Luo, “Logic-rl: Unleashing llm reasoning with rule-bas… find our card
  19. [289] Z. Li, Z. Ma, M. Li, S. Li, Y. Rong, T. Xu, Z. Zhang, D. Zhao, and W. Huang, “Star-r1: Spatial transformation reasoning by reinforcing multimodal llms,” 2025. [Online]. Available: https://arxiv.org/abs/2505.15804 [290] E. Yeo, Y. Tong, M. Niu, G. Neubig, and X. Yue, “Demystifying Long Chain-of… ↗ our card
  20. [311] AI-MO, “Aime 2024,” 2024. [Online]. Available: https:// huggingface.co/datasets/AI-MO/aimo-validation-aime [312] M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou et al., “Challenging big-bench tasks and whether chain-of-thought can… find our card
  21. [412] I. Team, “InternLM2 Technical Report,” 2024. [413] N. Muennighoff, Z. Yang, W. Shi, X. L. Li, L. Fei-Fei, H. Hajishirzi, L. Zettlemoyer, P. Liang, E. Candès, and T. Hashimoto, “s1: Simple test-time scaling,” arXiv preprint arXiv:2501.19393, 2025. [414] OpenAI, “OpenAI o1-mini,” September 2024.… find our card
  22. [493] D. Zhang, S. Zhoubian, M. Cai, F. Li, L. Yang, W. Wang, T. Dong, Z. Hu, J. Tang, and Y. Yue, “Datascibench: An llm agent benchmark for data science,” 2025. [Online]. Available: https://arxiv.org/abs/2502.13897 [494] Y. Chai, H. Li, J. Zhang, L. Liu, G. Liu, G. Wang, S. Ren, S. Huang, and H. Li… ↗
  23. [515] B. Peng, E. Alcaide, Q. Anthony, A. Albalak, S. Arcadinho, S. Biderman, H. Cao, X. Cheng, M. Chung, M. Grella, K. K. GV, X. He, H. Hou, J. Lin, P. Kazienko, J. Kocon, J. Kong, B. Koptyra, H. Lau, K. S. I. Mantri, F. Mom, A. Saito, G. Song, X. Tang, B. Wang, J. S. Wind, S. Wozniak, R. Zhang, Z.… find