Your Reasoning Model Knows What Counts: Self-Guided Chain-of-Thought Pruning for Efficient Reasoning
Paper · 2026 · Zi-Ao Ma, Xian-Ling Mao, Tian Lan et al.
Summary
Large Reasoning Models often produce long chain-of-thought traces full of detours, which raise inference cost and can hurt robustness. Earlier compression methods rely on length penalties, truncation heuristics, external verifiers or teacher-curated data. Because of this, they may cut steps that a particular model actually needs. SGP-CoT (Self-Guided Pruning) instead lets each model judge its own reasoning.
The method works in three steps:
- It samples several correct responses per prompt.
- It splits each trace into reasoning units, using line-initial trigger tokens learned from frequency statistics.
- It scores every unit with the model's own likelihoods. One score captures how much the unit supports the final answer, and another captures how much it supports local coherence.
Low-impact units are removed. The pruned and original traces then form preference pairs for DPO training, so no external supervision is required.
On AIME, GPQA-Diamond, MATH-500 and GSM8K, the tuned models give shorter answers with equal or better accuracy. The approach suits accuracy-sensitive deployments where cheaper inference matters.
Key points
- Experiments use DeepSeek-R1-Distill-Qwen-1.5B and 7B. Average end-to-end latency on one H800 GPU dropped from 640 s to 328 s (about 49%) and from 834 s to 470 s (about 44%), respectively.
- LC-R1 and L1 compress more aggressively but lose more accuracy. SGP-CoT trades some compression for more consistent accuracy preservation.
- About 1.5K preference pairs are enough. Scaling to 10K–30K adds little compression, and larger sets reduce accuracy on hard benchmarks, most visibly AIME 2024.
- Ablations show that replacing DPO with plain SFT lengthens outputs on three benchmarks and lowers accuracy. Of the two scores, the answer-impact threshold matters more than the coherence threshold.
- Impact scoring for a 7B–8B model costs under 10 H800 GPU hours. Code is available at github.com/HammerScholar/SGP-CoT.
Institutions
ChinaFields, topics and concepts
📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works
🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,685 worksEfficient models & compression · 297 works
Cite
Cite as Zi-Ao Ma, Xian-Ling Mao, Tian Lan, Chen Xu and Zhijing Wu. 2026. Your Reasoning Model Knows What Counts: Self-Guided Chain-of-Thought Pruning for Efficient Reasoning. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
@inproceedings{ma2026your,
title = {{Your Reasoning Model Knows What Counts: Self-Guided Chain-of-Thought Pruning for Efficient Reasoning}},
author = {Zi-Ao Ma, Xian-Ling Mao, Tian Lan, Chen Xu and Zhijing Wu},
booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
year = {2026},
doi = {10.18653/v1/2026.acl-long.25},
url = {https://doi.org/10.18653/v1/2026.acl-long.25}
}References
- Pranjal Aggarwal and Sean Welleck. 2025. L1: Controlling how long a reasoning model thinks with reinforcement learning. arXiv preprint arXiv:2503.04697.
- While SGP-CoT demonstrates strong performance across diverse reasoning tasks, we acknowledge several limitations that suggest directions for future work: find
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/papers/ma-2026-your-reasoning-model-knows-what/