Elamur Research⌕
☰
home / papers / 2025 / ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning

ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning

Paper · 2025 · Bairu Hou, Yang Zhang, Jiabao Ji et al.

Summary

Reasoning models trained to produce long chains of thought often generate redundant, inefficient reasoning, which inflates inference cost. Earlier fixes such as budget forcing (e.g., S1) cut the reasoning off at a token limit at test time. This leaves the wasteful reasoning style intact and causes noticeable accuracy loss under tight budgets. ThinkPrune instead adapts the model itself. It continues RL training with GRPO, following the DeepSeek-R1 recipe, but truncates every sampled output at a fixed token limit before scoring. A response whose answer falls beyond the cutoff therefore earns zero reward. A system prompt also informs the model of the limit. To limit accuracy loss, the authors apply the limit in stages, tightening it over several RL rounds (e.g., 4k → 3k → 2k tokens). On math benchmarks this roughly halves or better the output length with little or no accuracy loss. Analysis shows the pruned models drop hesitation and repeated checking while keeping core computation. This makes long-CoT models considerably cheaper to run.

Key points

  • DeepSeek-R1-Distill-Qwen-1.5B: average output length fell from 10,355 to 3,574 tokens, and mean accuracy across four math benchmarks rose slightly. On AIME24, reasoning length was cut roughly in half for about a 2% accuracy loss.
  • DeepScaleR-1.5B-Preview: average length dropped from 5,914 to 3,370 tokens. QwQ-32B: average length dropped from 8,763 to 4,494 tokens. Both models lost about 2% accuracy.
  • Reward design: the reward is binary. It is 1 only if a correct answer can be extracted from the clipped output, which avoids extra hyperparameters or reward engineering. The staged schedule is inspired by iterative weight pruning (Han et al., 2015).
  • Behavior analysis on MATH500: hesitation markers such as "wait" became much rarer, and verification phrases decreased slightly. GPT-4o phase labeling showed a larger share of steps devoted to applying theorems and computing expressions.
  • Readability: perplexity of reasoning traces, measured with Qwen2.5-Math-7B, stayed about the same after pruning (e.g., 1.91 → 1.90 for the R1-Distill 1.5B model). Code is released at github.com/UCSB-NLP-Chang/ThinkPrune.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 5 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Computer Vision and Pattern Recognition · 1,282 works

🧭 Topics 2 Language models & pretraining · 1,343 works Reasoning · 510 works

Cite

Cite as Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas and Shiyu Chang. 2025. ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning. arXiv:2504.01296.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2504.01296 · arXiv 2504.01296

@misc{hou2025thinkprune,
  title = {{ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning}},
  author = {Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas and Shiyu Chang},
  year = {2025},
  doi = {10.48550/arxiv.2504.01296},
  eprint = {2504.01296},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2504.01296}
}

References

34 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AMC. American invitational mathematics examination. https://artofproblemsolving.com/wiki/index.php/American Invitational Mathematics Examination, 2025. ↗
  2. AMC. American mathematics competitions (amc). https://maa.org/student-programs/ amc/, 2025. ↗
  3. Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wangxiang Che. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567, 2025. our card
  4. Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187, 2024. our card
  5. Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. Process reinforcement through implicit rewards. arXiv preprint arXiv:2502.01456, 2025. our card
  6. DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv: 2501.12948, 2025. our card
  7. Yichao Fu, Junda Chen, Siqi Zhu, Zheyu Fu, Zhongdongming Dai, Aurick Qiao, and Hao Zhang. Efficiently serving llm reasoning programs with certaindex. arXiv preprint arXiv: 2412.20993, 2024. — [arXiv:2412.20993](https://arxiv.org/abs/2412.20993) find
  8. Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint arXiv:2503.01307, 2025. our card
  9. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Ani… find
  10. Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time compute with latent reasoning: A recurrent depth approach. arXiv preprint arXiv: 2502.05171, 2025. find
  11. Song Han, Jeff Pool, John Tran, and William J. Dally. Learning both weights and connections for efficient neural networks, 2015. find
  12. Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. arXiv preprint arXiv: 2412.06769, 2024. find
  13. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. Annual Me… our card
  14. Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron Courville, Alessandro Sordoni, and Rishabh Agarwal. V-star: Training verifiers for self-taught reasoners. arXiv preprint arXiv:2402.06457, 2024. our card
  15. Ke Ji, Jiahao Xu, Tian Liang, Qiuzhi Liu, Zhiwei He, Xingyu Chen, Xiaoyuan Liu, Zhijie Wang, Junying Chen, Benyou Wang, et al. The first few tokens are all you need: An efficient and effective unsupervised prefix fine-tuning method for reasoning models. arXiv preprint arXiv:2503.02875, 2025. — [arXiv:2503.02875](https://arxiv.org/abs/2503.02875) find
  16. KimiTeam, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025. our card
  17. Abhinav Kumar, Jaechul Roh, Ali Naseh, Marzena Karpinska, Mohit Iyyer, Amir Houmansadr, and Eugene Bagdasarian. Overthink: Slowdown attacks on reasoning llms. arXiv e-prints, pp. arXiv–2502, 2025. find
  18. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. arXiv preprint arXiv:2305.20050, 2023. our card
  19. Jiawei Liu and Lingming Zhang. Code-r1: Reproducing r1 for code with reliable rewards. 2025. find
  20. Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. https://pretty-radio-b75.notion.site/ DeepScaleR-Surpassing-O1-P… ↗
  21. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025. our card
  22. OpenAI. Openai o1 system card. arXiv preprint arXiv: 2412.16720, 2024. our card
  23. OpenAI. Introducing operator, 2024. introducing-operator/. Accessed: 2025-03-23. find
  24. URL https://openai.com/index/OpenAI. Competitive programming with large reasoning models. arXiv preprint arXiv: 2502.06807, 2025. ↗
  25. QwenTeam. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL https://qwenlm.github.io/blog/qwq-32b/. ↗
  26. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv: 2402.03300, 2024. our card
  27. Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. Codi: Compressing chain-of-thought into continuous space via self-distillation. arXiv preprint arXiv: 2502.21074, 2025. our card
  28. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256, 2024. our card
  29. Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Hanjie Chen, Xia Hu, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419, 2025. our card
  30. Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, et al. Thoughts are all over the place: On the underthinking of o1-like llms. arXiv preprint arXiv:2501.18585, 2025. find
  31. Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning. arXiv preprint arXiv: 2502.03387, 2025. our card
  32. Eric Zelikman, Georges Raif Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah Goodman. Quiet-star: Language models can teach themselves to think before speaking. In First Conference on Language Modeling, 2024. our card
  33. Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild. arXiv preprint arXiv: 2503.18892, 2025a. our card
  34. Zhiyuan Zeng, Qinyuan Cheng, Zhangyue Yin, Yunhua Zhou, and Xipeng Qiu. Revisiting the test-time scaling of o1-like models: Do they truly possess test-time scaling capabilities? arXiv preprint arXiv: 2502.12215, 2025b. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hou-2025-thinkprune-pruning-long-chain-thought/