Elamur Research⌕
☰
home / papers / 2026 / Pru-CoT: Towards Efficient Reasoning Distillation via Pruning Chain-of-Thought

Pru-CoT: Towards Efficient Reasoning Distillation via Pruning Chain-of-Thought

Paper · 2026 · Han Liu, Shuotian Ma, Hui Li et al.

Summary

Large reasoning models such as DeepSeek-R1 produce long Chain-of-Thought (CoT) traces that often include repeated checks and trivial restatements ("overthinking"). When these traces are used to distill reasoning into smaller models, the students tend to imitate the verbosity instead of the core logic. Earlier pruning methods either use local rules with narrow search, such as SPIRIT and Binary Cutting, or rely on expensive global search, such as A*-Thought.

Pru-CoT addresses this in two stages:

  1. Scoring steps. A frozen student LLM is used, and each reasoning step gets a learnable weight that blends its token embeddings with a learnable filler-token noise embedding. The weights are tuned by backpropagating the loss on the final solution, so steps the answer depends on are pushed toward 1.
  2. Rewriting the trace. An LLM agent uses the high-scoring steps as anchors and removes low-scoring ones, while keeping the narrative coherent and faithful to the original reasoning.

Students trained on the pruned traces were more accurate and produced shorter reasoning on math benchmarks. This makes Pru-CoT a cheaper, scalable way to curate distillation data.

Key points

  • Step scoring: each step's embeddings are mixed with a filler embedding (e.g., ".") using a per-step weight in [0, 1]. Steps scoring below roughly 0.5 become candidates for pruning.
  • Accuracy: the reported average gain is 2.4% on DeepSeek-R1-Distill-Qwen-7B, with experiments covering the DeepSeek-R1-Distill-Qwen model series.
  • Cost: total processing is about 4.4× faster than A*-Thought (1.82 vs. 7.99 GPU hours).
  • Case study: the LLM agent sometimes keeps steps that received a zero weight when later text refers back to them, which preserves coherence.
  • Limitations: evaluation used only modest model and dataset sizes, and there was no thorough hyperparameter search or comparison of filler tokens. The code is released on GitHub (Co-Cl2/Pru-CoT).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Pennsylvania State UniversityState College · United States56 works · 5,360 citations🎓Dalian University of TechnologyDalian · China12 works · 46 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Han Liu, Shuotian Ma, Hui Li, Xiaotong Zhang, Fenglong Ma and Hong Yu. 2026. Pru-CoT: Towards Efficient Reasoning Distillation via Pruning Chain-of-Thought. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1684

@inproceedings{liu2026pru,
  title = {{Pru-CoT: Towards Efficient Reasoning Distillation via Pruning Chain-of-Thought}},
  author = {Han Liu, Shuotian Ma, Hui Li, Xiaotong Zhang, Fenglong Ma and Hong Yu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1684},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1684}
}

References

31 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. 2025. Do NOT think that much for 2+3=? on the overthinking of long reasoning models. In International Conference on Machine Le… find
  2. Tri Dao. 2024. Flashattention-2: Faster attention with better parallelism and work partitioning. In International Conference on Learning Representations (ICLR).
  3. DeepSeek-AI. 2025a. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv, abs/2501.12948.
  4. Mengru Ding, Hanmeng Liu, Zhizhang Fu, Jian Song, Wenbo Xie, and Yue Zhang. 2024. Break the chain: Large language models can be shortcut reasoners. arXiv, abs/2406.06580. — [arXiv:2406.06580](https://arxiv.org/abs/2406.06580) find
  5. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, and 5 others.… find
  6. Etash Kumar Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, and 31 others. 2025.… — OpenThoughts: Data Recipes for Reasoning Models
  7. Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang. 2026. Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning.
  8. Trans. Mach. Learn. Res., 2026. find
  9. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Symposium on Operating Systems Principles (SOSP), pages 611–626.
  10. Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang, Ziju Shen, and 1 others. 2024. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13(9):… find
  11. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s verify step by step. In International Conference on Learning Representations (ICLR).
  12. Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. 2025. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning. arXiv, abs/2501.12570. find
  13. Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. 2025. CoT-valve: Length-compressible chain-of-thought tuning. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 6025–6035.
  14. OpenAI. 2024. Openai o1 system card. abs/2412.16720. arXiv, Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. 2024. LLMLingua-2: Data distillation for efficient and faithful t…
  15. Jacob Pfau, William Merrill, and Samuel R. Bowman. 2024. Let’s think dot by dot: Hidden computation in transformer language models. arXiv, abs/2404.15758. find
  16. Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pages 3505–3506. find
  17. Joykirat Singh, Justin Chih-Yao Chen, Archiki Prasad, Elias Stengel-Eskin, Akshay Nambi, and Mohit Bansal. 2025. Think right: Learning to mitigate under-over thinking via adaptive, attentive compression. arXiv, abs/2510.01581. find
  18. Kimi Team. 2025a. Kimi k1.5: Scaling reinforcement learning with llms. arXiv, abs/2501.12599.
  19. Qwen Team. 2025b. Qwen3 technical report. arXiv, abs/2505.09388.
  20. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark…
  21. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Conference on Neural Information Processing Systems (NeurIPS), pages 24824–24837.
  22. Heming Xia, Chak Tou Leong, Wenjie Wang, Yongqi Li, and Wenjie Li. 2025. TokenSkip: Controllable chain-of-thought compression in LLMs. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3351–3363.
  23. Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. 2025a. Chain of draft: Thinking faster by writing less. arXiv, abs/2502.18600. find
  24. Xiaoang Xu, Shuo Wang, Xu Han, Zhenghao Liu, Huijia Wu, Pei Pei Li, Zhiyuan Liu, Maosong Sun, and Zhaofeng He. 2025b. A*-thought: Efficient reasoning via bidirectional compression for low-resource settings. In Conference on Neural Information Processing Systems (NeurIPS). find
  25. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, and 43 others. 2024a. Qwen2 technical report. arXiv, abs/2407.10… find
  26. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 22 others. 2024b. Qwen2.5 technical report. arXiv, abs/2412.15… find
  27. Wenhao Zeng, Yaoning Wang, Chao Hu, Yuling Shi, Chengcheng Wan, Hongyu Zhang, and Xiaodong Gu. 2025. Pruning the unsurprising: Efficient code reasoning via first-token surprisal. arXiv, abs/2508.05988. — [arXiv:2508.05988](https://arxiv.org/abs/2508.05988) find
  28. Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, and Ningyu Zhang. 2025. LightThinker: Thinking step-by-step compression. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 13307–13328.
  29. Yifan Zhang and Team Math-AI. 2025. American invitational mathematics examination (aime) 2025. find
  30. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. 2024. LlamaFactory: Unified efficient fine-tuning of 100+ language models. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 400–410.
  31. Zhaoyang Wang, Jinqi Jiang, Tian Qiu, Hui Liu, Xianfeng Tang, and Huaxiu Yao. 2025. Efficient long cot reasoning in small language models. arXiv, abs/2505.18440. — [arXiv:2505.18440](https://arxiv.org/abs/2505.18440) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-pru-cot-efficient-reasoning-distillation/