Elamur Research
☰
home / papers / 2025 / ProcessBench: Identifying Process Errors in Mathematical Reasoning

ProcessBench: Identifying Process Errors in Mathematical Reasoning

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2025.acl-long.50.pdf

Summary

Language models still make mistakes in multi-step math solutions. Automatically locating where their reasoning breaks down is seen as a building block for scalable oversight. Existing resources are either too easy for current models or check only final answers.

The Qwen Team at Alibaba built ProcessBench, a set of 3,400 step-by-step solutions to problems from GSM8K, MATH, OlympiadBench and Omni-MATH. Twelve open-source Qwen and LLaMA models generated the solutions. Multiple human experts then marked the position of the first faulty step, or −1 if the solution has no error. Targeting only the earliest error avoids the ambiguity of judging steps that build on wrong premises. It also keeps the protocol simple enough to apply to both process reward models (PRMs) and general LLMs prompted as critics.

Published PRMs degrade sharply on harder problems. They lose to prompted critics and to a simple PRM fine-tuned on PRM800K. Among open models, QwQ-32B-Preview rivals GPT-4o but trails o1-mini. The benchmark exposes weaknesses in how current PRM training data is synthesized and offers a testbed for reasoning verification.

Key points

  • Dataset: ProcessBench has 3,400 test cases. Most are competition- or Olympiad-level, with GSM8K as the only grade-school source. It is released on Hugging Face (Qwen/ProcessBench) and GitHub (QwenLM/ProcessBench).
  • Task: Given a problem and its solution steps, a model outputs the index of the earliest wrong step (0-based) or −1. Annotators considered mathematical, logical, conceptual and completeness errors but did not have to label the error type.
  • Main finding on PRMs: Existing PRMs, e.g. Math-Shepherd-style models trained on synthesized labels, fall behind prompted critic models. The authors attribute this to two causes:
    • Labels estimated from final-answer success depend on the model that generated the solutions.
    • Solutions often reach correct answers despite flawed steps, which corrupts those estimated labels.
  • PRM800K baseline: The authors' Qwen2.5-Math-7B-PRM800K, trained on fully human-annotated PRM800K, generalizes notably better than other PRMs.
  • Critic models: In the overview figure, o1-mini leads (87.9) and GPT-4o-0806 scores 61.9. QwQ-32B-Preview is the best open-source critic and is close to GPT-4o. Within model families, critique ability improves with model size. Open models were evaluated with majority voting over eight samples.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Mathematics & statistics

Cited by

104 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 100 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.50 · arXiv 2412.06559

@inproceedings{zheng2025processbench,
  title = {{ProcessBench: Identifying Process Errors in Mathematical Reasoning}},
  author = {Zheng, Chujie and Zhang, Zhenru and Zhang, Beichen and Lin, Runji and Lu, Keming and Yu, Bowen and Liu, Dayiheng and Zhou, Jingren and Lin, Junyang},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.50},
  eprint = {2412.06559},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2025.acl-long.50}
}

References

28 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. 2016. Concrete problems in ai safety. arXiv preprint arXiv:1606.06565. find our card
  2. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2023. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798. find our card
  3. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2.5-coder technical report. arXiv preprint arXiv:2409.12186. find our card
  4. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. find our card
  5. Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. 2024. When can LLMs actually correct their own mistakes? a critical survey of self-correction of LLMs. Transactions of the Association for Computational Linguistics, 12:1417–1440. find
  6. Samuel R Bowman, Jeeyoon Hyun, Ethan Perez, Edwin Chen, Craig Pettit, Scott Heiner, Kamilė Lukošiūtė, Amanda Askell, Andy Jones, Anna Chen, et al. 2022. Measuring progress on scalable oversight for large language models. arXiv preprint arXiv:2211.03540. find
  7. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, pages 611–626. find
  8. Boxi Cao, Keming Lu, Xinyu Lu, Jiawei Chen, Mengjie Ren, Hao Xiang, Peilin Liu, Yaojie Lu, Ben He, Xianpei Han, et al. 2024. Towards scalable automated alignment of llms: A survey. arXiv preprint arXiv:2406.01252. — [arXiv:2406.01252](https://arxiv.org/abs/2406.01252) find
  9. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. arXiv preprint arXiv:2305.20050. find our card
  10. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. find our card
  11. Zicheng Lin, Zhibin Gou, Tian Liang, Ruilin Luo, Haowei Liu, and Yujiu Yang. 2024. Criticbench: Benchmarking llms for critique-correct reasoning. arXiv preprint arXiv:2402.14809. find
  12. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. find
  13. Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, Jiao Sun, and Abhinav Rastogi. 2024. Improve mathematical reasoning in language models by automated process supervision. arXiv preprint arXiv:2406.06592. find our card
  14. Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, et al. 2024. Omni-math: A universal olympiad level mathematic benchmark for large language models. arXiv preprint arXiv:2410.07985. find our card
  15. Nat McAleese, Rai Michael Pokorny, Juan Felipe Ceron Uribe, Evgenia Nitishinskaya, Maja Trebacz, and Jan Leike. 2024. Llm critics help catch llm bugs. arXiv preprint arXiv:2407.00215. — [arXiv:2407.00215](https://arxiv.org/abs/2407.00215) find
  16. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. 2024. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402.14008. find our card
  17. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874. find our card
  18. OpenAI. 2024. Openai o1-mini: Advancing cost-efficient reasoning. find
  19. Team Qwen. 2024a. Qwen2.5: A party of foundation models. find
  20. Team Qwen. 2024b. Qwq: Reflect deeply on the boundaries of the unknown. find
  21. William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. 2022. Self-critiquing models for assisting human evaluators. arXiv preprint arXiv:2206.05802. o1 Team Skywork. 2024. Skywork-o1 open series. https://huggingface.co/Skywork. ↗ our card
  22. Alan Wake, Albert Wang, Bei Chen, CX Lv, Chao Li, Chengen Huang, Chenglin Cai, Chujie Zheng, Daniel Cooper, Ethan Dai, et al. 2024. Yi-lightning technical report. arXiv preprint arXiv:2412.01253. find
  23. Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2024. Math-shepherd: Verify and reinforce LLMs step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper… find our card
  24. Wei Xiong, Chengshuai Shi, Jiaming Shen, Aviv Rosenberg, Zhen Qin, Daniele Calandriello, Misha Khalman, Rishabh Joshi, Bilal Piot, Mohammad Saleh, et al. 2024a. Building math agents with multi-turn iterative preference learning. arXiv preprint arXiv:2409.02392. — [arXiv:2409.02392](https://arxiv.org/abs/2409.02392) find
  25. Wei Xiong, Hanning Zhang, Nan Jiang, and Tong Zhang. 2024b. An implementation of generative prm. https://github.com/RLHFlow/ RLHF-Reward-Modeling. ↗
  26. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024a. Qwen2 technical report. arXiv preprint arXiv:2407.10671. find
  27. An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. 2024b. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122. find our card
  28. Zihao Zhou, Shudong Liu, Maizhen Ning, Wei Liu, Jindong Wang, Derek F Wong, Xiaowei Huang, Qiufeng Wang, and Kaizhu Huang. 2024. Is your model really a good math reasoner? evaluating mathematical reasoning with checklist. arXiv preprint arXiv:2407.08733. — [arXiv:2407.08733](https://arxiv.org/abs/2407.08733) find