Elamur Research⌕
☰
home / papers / 2026 / PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering

PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering

Paper · 2026 · Xiangfeng Wang, Hangyu Guo, Yanlin Lai et al.

Summary

RLVR (reinforcement learning with verifiable rewards) relies on verifiers that usually check only whether a model's final answer matches the reference. Such verifiers reward "lucky guesses," where a correct answer comes from flawed reasoning. In a human-annotated sample of responses to 2,500 STEM prompts, the authors found this in 16.98% of cases. To address this, they built PRIME, a benchmark that tests whether a verifier checks both answer correctness and whether the derivation logically supports the answer.

The construction pipeline had five stages:

  • collecting college-level STEM problems;
  • filtering them with models for verifiability and answer correctness;
  • generating solutions with many different LRMs;
  • keeping the cases that are hard to verify;
  • having domain experts label each sample.

Many existing verifiers often miss derivation errors. Using verifiers selected via PRIME as process-aware reward models in RLVR clearly beat outcome-only verification. Verifier accuracy on PRIME also tracked downstream training gains almost linearly. This makes PRIME a practical tool for choosing reward models.

Key points

  • Data: PRIME contains 2,530 expert-annotated samples, distilled from a pool of over 7M textbook and exam problems. It covers 16 STEM categories and about 480 sub-disciplines, and 18 domain experts did the labeling.
  • Filtering: GPT-OSS-120B checked whether each problem was verifiable. Reference answers were then cross-checked with Gemini-3-Pro, GPT-5 and Claude-Sonnet-4.5.
  • Training gains: With process-aware verification, Qwen3-14B-Base improved over the outcome-only baseline by 8.29% on AIME24, 9.12% on AIME25 and 7.31% on Beyond-AIME (absolute).
  • Verifier comparison: Five verifiers were compared: CompassVerifier-32B, Qwen3-4B, Qwen3-8B, GPT-OSS-20B and GPT-OSS-120B. With GPT-OSS-120B, the average score of Qwen3-14B-Base rose from 24.79% to 47.07%.
  • Correlation: PRIME accuracy correlates linearly with downstream RLVR performance, with R² = 0.937 for Qwen3-8B-Base and 0.920 for Qwen3-14B-Base. The benchmark and code are released on GitHub and Hugging Face.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🎓University of Science and Technology of ChinaHefei · China108 works · 112,698 citations🏢StepFun AIShanghai · China3 works · 131 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputational Theory and Mathematics · 252 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksEvaluation & benchmarks · 1,220 works

Cite

Cite as Xiangfeng Wang, Hangyu Guo, Yanlin Lai, Mitt Huang, Liang Zhao, Chengyuan Yao, Yinmin Zhang, Qi Han, Xiaoxiaoren, Chun Yuan, Tong Xu, Zheng Ge, Xiangyu Zhang and Daxin Jiang. 2026. PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.683

@inproceedings{wang2026prime,
  title = {{PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering}},
  author = {Xiangfeng Wang, Hangyu Guo, Yanlin Lai, Mitt Huang, Liang Zhao, Chengyuan Yao, Yinmin Zhang, Qi Han, Xiaoxiaoren, Chun Yuan, Tong Xu, Zheng Ge, Xiangyu Zhang and Daxin Jiang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.683},
  url = {https://doi.org/10.18653/v1/2026.acl-long.683}
}

References

32 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, and 1 others. 2024. Phi-4 technical report. arXiv preprint arXiv:2412.08905.
  2. Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, and 1 others. 2025. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925.
  3. Anthropic. 2025a. Claude Opus 4 and Claude Sonnet 4 System Card. System card, Anthropic. Accessed: 2026-01-05. find
  4. Anthropic. 2025b. Claude Sonnet 4.5 System Card. System card, Anthropic. Accessed: 2026-01-05. find
  5. ByteDance-Seed. 2025. Beyondaime: Advancing math reasoning evaluation beyond high school olympiads. [https://huggingface.co/ datasets/ByteDance-Seed/BeyondAIME](https://huggingface.co/datasets/ByteDance-Seed/ BeyondAIME). find
  6. ByteDance Seed Team. 2025. Seed 1.6: Multimodal models with adaptive deep thinking. https://seed.bytedance.com/en/seed1_6. Technical Report. Accessed: 2026-01-05. ↗
  7. Ding Chen, Qingchen Yu, Pengyuan Wang, Wentao Zhang, Bo Tang, Feiyu Xiong, Xinchi Li, Minchuan Yang, and Zhiyu Li. 2025. xverify: Efficient answer verifier for reasoning model evaluations. CoRR, abs/2504.10481. — [arXiv:2504.10481](https://arxiv.org/abs/2504.10481) find
  8. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  9. Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, King Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, and 1 others. 2025. Supergpqa: Scaling llm evaluation across 285 graduate disciplines. arXiv preprint arXiv:2502.14739. find
  10. Ruixiang Feng, Zhenwei An, Yuntao Wen, Ran Le, Yiming Jia, Chen Yang, Zongchao Chen, Lisi Chen, Shen Gao, Shuo Shang, and 1 others. 2025. Cosin-everifier: Tool-augmented answer verification for computation-oriented scientific questions. arXiv preprint arXiv:2512.01224. — [arXiv:2512.01224](https://arxiv.org/abs/2512.01224) find
  11. Google DeepMind. 2025. Gemini: The Most Capable and General Model We’ve Ever Built. Accessed: 2026-01-05. find
  12. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  13. Hynek Kydlíček and Hugging Face Team. 2025. Math-Verify: A robust mathematical expression evaluation system. Accessed: 2026-01-05. find
  14. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. arXiv preprint arXiv:2305.20050.
  15. Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, and 1 others. 2025a. Deepseek-v3. 2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556.
  16. Shudong Liu, Hongwei Liu, Junnan Liu, Linchen Xiao, Songyang Gao, Chengqi Lyu, Yuzhe Gu, Wenwei Zhang, Derek F. Wong, Songyang Zhang, and Kai Chen. 2025b. Compassverifier: A unified and robust verifier for llms evaluation and outcome reward. CoRR, abs/2508.03686.
  17. Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. 2024. Generative reward models. arXiv preprint arXiv:2410.12832. find
  18. OpenAI. 2025. GPT-5 System Card. System card, OpenAI. Accessed: 2026-01-05. find
  19. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling.
  20. ByteDance Seed, Jiaze Chen, Tiantian Fan, Xin Liu, Lingjun Liu, Zhiqi Lin, Mingxuan Wang, Chengyi Wang, Xiangpeng Wei, Wenyuan Xu, and 1 others. 2025. Seed1. 5-thinking: Advancing superb reasoning models with reinforcement learning. arXiv preprint arXiv:2504.13914. — Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning
  21. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  22. Yi Su, Dian Yu, Linfeng Song, Juntao Li, Haitao Mi, Zhaopeng Tu, Min Zhang, and Dong Yu. 2025. Crossing the reward bridge: Expanding RL with verifiable rewards across diverse domains. CoRR, abs/2503.23829. — [arXiv:2503.23829](https://arxiv.org/abs/2503.23829) find
  23. Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, and 1 others. 2025. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534.
  24. Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14275.
  25. Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2024a. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pape…
  26. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, and 1 others. 2024b. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266–9529…
  27. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  28. An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, and 1 others. 2024. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122. — Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
  29. Z.ai Team. 2025. GLM-4.6: Advanced Agentic, Reasoning and Coding Capabilities. Accessed: 2026-01-05. find
  30. Kaiyan Zhang, Yuxin Zuo, Bingxiang He, Youbang Sun, Runze Liu, Che Jiang, Yuchen Fan, Kai Tian, Guoli Jia, Pengfei Li, and 1 others. 2025. A survey of reinforcement learning for large reasoning models. arXiv preprint arXiv:2509.08827. — [arXiv:2509.08827](https://arxiv.org/abs/2509.08827) find
  31. Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. 2024. Generative verifiers: Reward modeling as next-token prediction, 2024. URL https://arxiv.org/abs/2408.15240, 1. ↗
  32. Shenghe Zheng, Chenyu Huang, Fangchen Yu, Junchi Yao, Jingqi Ye, Tao Chen, Yun Luo, Ning Ding, Lei Bai, Ganqu Cui, and Peng Ye. 2025. Sci-verifier: Scientific verifier with thinking. CoRR, abs/2509.24285. *Figure 6: Answer type distribution of P RIME .* CompassVerifier xVerify-9B-C Tencent-Qwen-7B-R… — [arXiv:2509.24285](https://arxiv.org/abs/2509.24285) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-prime-process-outcome-alignment-benchmark/