Let's Verify Step by Step

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Mathematics & statistics

Cited by

1328 papers of our library cite it (of the 140673 whose bibliographies we read); 60 of them have a page here, 1268 not yet.

• Many-Shot In-Context Learning
• Large Language Models for Mathematical Reasoning: Progresses and Challenges
• Llemma: An Open Language Model For Mathematics
• Qwen Technical Report
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
• Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
• InternLM2 Technical Report
• AlphaMath Almost Zero: Process Supervision without Process
• Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
• Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
• FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios
• Process Reinforcement through Implicit Rewards
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• JudgeMeNot: Personalizing Large Language Models to Emulate Judicial Reasoning in Hebrew
• Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs
• ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
• rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
• V-STaR: Training Verifiers for Self-Taught Reasoners
• OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework
• Large Language Models Can Self-Improve
• Large Language Models Cannot Self-Correct Reasoning Yet
• Survey of Hallucination in Natural Language Generation
• Kimi k1.5: Scaling Reinforcement Learning with LLMs
• Training Language Models to Self-Correct via Reinforcement Learning
• Tulu 3: Pushing Frontiers in Open Language Model Post-Training
• Generative Judge for Evaluating Alignment
• LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
• Search-o1: Agentic Search-Enhanced Large Reasoning Models
• MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
• WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct
• Improve Mathematical Reasoning in Language Models by Automated Process Supervision
• Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
• s1: Simple test-time scaling
• OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
• To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
• ReFT: Reasoning with Reinforced Fine-Tuning
• Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts
• Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
• Large Language Models are Better Reasoners with Self-Verification
• Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
• The Rise and Potential of Large Language Model Based Agents: A Survey
• Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
• How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark
• Qwen3 Technical Report
• Demystifying Long Chain-of-Thought Reasoning in LLMs
• MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
• Scaling Relationship on Learning Mathematical Reasoning with Large Language Models
• SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild
• Instruction Tuning for Large Language Models: A Survey
• Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
• Generative Verifiers: Reward Modeling as Next-Token Prediction
• ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
• The Lessons of Developing Process Reward Models in Mathematical Reasoning
• A Survey of Large Language Models
• ProcessBench: Identifying Process Errors in Mathematical Reasoning
• Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification
• InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2305.20050 · arXiv 2305.20050

@misc{lightman2023let,
  title = {{Let's Verify Step by Step}},
  author = {Lightman, Hunter and Kosaraju, Vineet and Burda, Yura and Edwards, Harri and Baker, Bowen and Lee, Teddy and Leike, Jan and Schulman, John and Sutskever, Ilya and Cobbe, Karl},
  year = {2023},
  doi = {10.48550/arxiv.2305.20050},
  eprint = {2305.20050},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.20050}
}

References

26 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. Askell, Y. Bai, A. Chen, D. Drain, D. Ganguli, T. Henighan, A. Jones, N. Joseph, B. Mann, N. DasSarma, et al. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861, 2021. find our card
2. S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023. find our card
3. P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017. find
4. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. find our card
5. A. Cotra. Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover. https://www.alignmentforum.org/posts/pRkFkzwKZ2zfa3R6H/without-specific-countermeasures-the-easiest-path-to, 2022. ↗
6. A. Creswell, M. Shanahan, and I. Higgins. Selection-inference: Exploiting large language models for interpretable logical reasoning. arXiv preprint arXiv:2205.09712, 2022. find
7. T. Everitt, V. Krakovna, L. Orseau, M. Hutter, and S. Legg. Reinforcement learning with a corrupted reward channel. arXiv preprint arXiv:1705.08417, 2017. — [arXiv:1705.08417](https://arxiv.org/abs/1705.08417) find
8. L. Gao, J. Schulman, and J. Hilton. Scaling laws for reward model overoptimization. arXiv preprint arXiv:2210.10760, 2022. find
9. D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. find our card
10. T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa. Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916, 2022. find our card
11. A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, et al. Solving quantitative reasoning problems with language models. arXiv preprint arXiv:2206.14858, 2022. find our card
12. Y. Li, Z. Lin, S. Zhang, Q. Fu, B. Chen, J.-G. Lou, and W. Chen. On the advance of making language models better reasoners. arXiv preprint arXiv:2206.02336, 2022. — [arXiv:2206.02336](https://arxiv.org/abs/2206.02336) find
13. J. Maynez, S. Narayan, B. Bohnet, and R. McDonald. On faithfulness and factuality in abstractive summarization. arXiv preprint arXiv:2005.00661, 2020. find
14. R. Nakano, J. Hilton, S. Balaji, J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V. Kosaraju, W. Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021. find our card
15. E. Nichols, L. Gao, and R. Gomez. Collaborative storytelling with large-scale neural language models. In Proceedings of the 13th ACM SIGGRAPH Conference on Motion, Interaction and Games, pages 1–10, 2020. find
16. M. Nye, A. J. Andreassen, G. Gur-Ari, H. Michalewski, J. Austin, D. Bieber, D. Dohan, A. Lewkowycz, M. Bosma, D. Luan, et al. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114, 2021. find our card
17. OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. find our card
18. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155, 2022. find
19. J. Shen, Y. Yin, L. Li, L. Shang, X. Jiang, M. Zhang, and Q. Liu. Generate & rank: A multi-task framework for math word problems. arXiv preprint arXiv:2109.03034, 2021. find
20. N. Stiennon, L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. Voss, A. Radford, D. Amodei, and P. F. Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008–3021, 2020. find
21. A. Stuhlmüller and J. Byun. Supervise process, not outcomes. https://ought. org/updates/2022-04-06-process, 2022. find
22. J. Uesato, N. Kushman, R. Kumar, F. Song, N. Siegel, L. Wang, A. Creswell, G. Irving, and I. Higgins. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14275, 2022. find our card
23. X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, and D. Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022. find our card
24. J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, and D. Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022. find our card
25. E. Zelikman, Y. Wu, J. Mu, and N. Goodman. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35: 15476–15488, 2022. find our card
26. D. M. Ziegler, N. Stiennon, J. Wu, T. B. Brown, A. Radford, D. Amodei, P. Christiano, and G. Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019. find