Elamur Research⌕
☰
home / papers / 2026 / Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning

Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning

Paper · 2026 · Yi Sui, Chaozhuo Li and Dawei Song

Summary

Large Reasoning Models tend to "overthink." They produce very long Chain-of-Thought (CoT) traces regardless of how hard a problem is, which raises latency and compute cost. Earlier compression methods usually tune brevity per task and often lose accuracy. The authors present STACK, which compresses reasoning step by step and decides how to compress based on the model's current state.

STACK tracks local token entropy against a sliding-window baseline to spot "hesitation" steps where the model is uncertain:

  • Uncertain steps: retrieved external evidence is injected through Knowledge-Guided Contrastive Decoding (KGCD).
  • Confident but verbose stretches: the model is prompted to trim its own redundant steps.
  • Late re-verification: an early-stopping rule halts reasoning once the answer distribution converges.

The compressed chains are paired online with the original long chains as contrastive samples. Training uses a DPO objective augmented with PPO-style reward differences. On three math benchmarks, STACK cuts average response length by 59.9% and raises accuracy by 4.8 points over prior methods. This suggests that reasoning efficiency and reliability can be improved together.

Key points

  • Hesitation detection: entropy-based detection triggers retrieval only when needed. Always-on retrieval (fusion weight λ_w ≈ 0.98) gave lower accuracy (92.9 on MATH500, 56.3 on AIME24) than gated STACK (93.5 / 57.4).
  • Robustness to noisy retrieval: with random top-1 replacement, accuracy dropped only to 92.4 on MATH500 and 55.4 on AIME24, while λ_w fell to 0.11. This indicates that unreliable evidence is down-weighted automatically.
  • Early stopping: it affects efficiency rather than correctness. Removing it barely changes accuracy (93.3 / 57.1) but lengthens outputs to 2475 tokens on MATH500 and 9130 on AIME24.
  • Training ablation: SFT followed by reward-difference-augmented DPO ("MDPO") balances accuracy and length better than SFT alone or SFT+DPO.
  • Hyperparameters and limitations: the best settings across model scales were µ = 0.25 for the contrastive decoding coefficient and α = 0.5 for the compression reward weight. The listed limitations are higher training cost, imperfect retrieval quality, and reliance on text-only knowledge.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Beijing University of Posts and TelecommunicationsBeijing · China57 works · 32,843 citations🎓Beijing Institute of TechnologyBeijing · China34 works · 3,652 citations🎓The Open UniversityMilton Keynes · United Kingdom10 works · 338 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksEfficient models & compression · 242 works

Cite

Cite as Yi Sui, Chaozhuo Li and Dawei Song. 2026. Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1128

@inproceedings{sui2026think,
  title = {{Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning}},
  author = {Yi Sui, Chaozhuo Li and Dawei Song},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1128},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1128}
}

References

53 references, 28 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  2. Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. 2025. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567.
  3. Jeffrey Cheng and Benjamin Van Durme. 2024. Compressed chain of thought: Efficient reasoning through dense representations. arXiv preprint arXiv:2412.13171. find
  4. Cheng-Han Chiang and Hung-yi Lee. 2024. Over-reasoning and redundant calculation of large language models. arXiv preprint arXiv:2401.11467.
  5. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
  6. Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A survey on rag meeting llms: Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6491– 6501.
  7. Gongfan Fang, Xinyin Ma, and Xinchao Wang. 2025. Thinkless: Llm learns when to think. arXiv preprint arXiv:2505.13379. — [arXiv:2505.13379](https://arxiv.org/abs/2505.13379) find
  8. Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, and 1 others. 2024. Omni-math: A universal olympiad level mathematic benchmark for large language models. arXiv preprint arXiv:2410.07985.
  9. Aryo Pradipta Gema, Chen Jin, Ahmed Abdulaal, Tom Diethe, Philip Teare, Beatrice Alex, Pasquale Minervini, and Amrutha Saseendran. 2024. Decore: Decoding by contrasting retrieval heads to mitigate hallucinations. arXiv preprint arXiv:2410.18860. find
  10. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  11. Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. 2025. Token-budget-aware llm reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, pages 24842–24855.
  12. Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang. 2025. Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning, 2025. arXiv preprint arXiv:2504.01296.
  13. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and 1 others. 2024. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems.
  14. Lightman Hunter, Kosaraju Vineet, Burda Yuri, Edwards Harrison, Baker Bowen, Lee Teddy, Leike Jan, Schulman John, Sutskever Ilya, and Cobbe Karl. 2023. Let’s verify step by step. In The Twelfth International Conference on Learning Representations.
  15. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720.
  16. Chaozhuo Li, Pengbo Wang, Chenxu Wang, Litian Zhang, Zheng Liu, Qiwei Ye, Yuanbo Xu, Feiran Huang, Xi Zhang, and Philip S Yu. 2025a. Loki’s dance of illusions: A comprehensive survey of hallucination in large language models. arXiv preprint arXiv:2507.02870. — [arXiv:2507.02870](https://arxiv.org/abs/2507.02870) find
  17. Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori B Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2023. Contrastive decoding: Open-ended text generation as optimization. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1:…
  18. Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025b. Search-o1: Agentic search-enhanced large reasoning models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 5420–5438.
  19. Yingqian Min, Zhipeng Chen, Jinhao Jiang, Jie Chen, Jia Deng, Yiwen Hu, Yiru Tang, Jiapeng Wang, Xiaoxue Cheng, Huatong Song, and 1 others. 2024. Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning systems. arXiv preprint arXiv:2412.09413. find
  20. Tergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang, Yujin Kim, and Se-Young Yun. 2025. Self-training elicits concise reasoning in large language models. arXiv preprint arXiv:2502.20122. find
  21. Agada Joseph Oche, Ademola Glory Folashade, Tirthankar Ghosal, and Arpan Biswas. 2025. A systematic review of key retrieval-augmented generation (rag) systems: Progress, gaps, and future directions. arXiv preprint arXiv:2507.18910. — [arXiv:2507.18910](https://arxiv.org/abs/2507.18910) find
  22. Songyang Liu, Chaozhuo Li, Jiameng Qiu, Xi Zhang, Feiran Huang, Litian Zhang, Yiming Hei, and Philip S Yu. 2025a. The scales of justitia: A comprehensive survey on safety evaluation of llms. arXiv preprint arXiv:2506.11094. — [arXiv:2506.11094](https://arxiv.org/abs/2506.11094) find
  23. Ziqing Qiao, Yongheng Deng, Jiali Zeng, Dong Wang, Lai Wei, Guanbo Wang, Fandong Meng, Jie Zhou, Ju Ren, and Yaoxue Zhang. 2025. Concise: Confidence-guided compression in step-by-step efficient reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, page…
  24. Songyang Liu, Chaozhuo Li, Chenxu Wang, Jinyu Hou, Zejian Chen, Litian Zhang, Zheng Liu, Qiwei Ye, Yiming Hei, Xi Zhang, and 1 others. 2026. Clawkeeper: Comprehensive safety protection for openclaw agents through skills, plugins, and watchers. arXiv preprint arXiv:2603.24414. — [arXiv:2603.24414](https://arxiv.org/abs/2603.24414) find
  25. Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang, Junxian He, and 1 others. 2025. A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond. arXiv preprint arXiv:2503.21614.
  26. Xin Liu and Lu Wang. 2025. Answer convergence as a signal for early stopping in reasoning. arXiv preprint arXiv:2506.02536.
  27. Yue Liu, Jiaying Wu, Yufei He, Ruihan Gong, Jun Xia, Liang Li, Hongcheng Gao, Hongyu Chen, Baolong Bi, Jiaheng Zhang, and 1 others. 2025b. Efficient inference for large reasoning models: A survey. arXiv preprint arXiv:2503.23077. — [arXiv:2503.23077](https://arxiv.org/abs/2503.23077) find
  28. Haolang Lu, Yilian Liu, Jingxin Xu, Guoshun Nan, Yuanlong Yu, Zhican Chen, and Kun Wang. 2025. Auditing meta-cognitive hallucinations in reasoning large language models. arXiv preprint arXiv:2505.13143. — [arXiv:2505.13143](https://arxiv.org/abs/2505.13143) find
  29. Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. 2025a. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning. arXiv preprint arXiv:2501.12570. find
  30. Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, and 1 others. 2025b. Deepscaler: Surpassing o1-preview with a 1.5 b model by scaling rl. Notion Blog. find
  31. Minjia Mao, Bowen Yin, Yu Zhu, and Xiao Fang. 2025. Early stopping chain-of-thoughts in large language models. arXiv preprint arXiv:2509.14004. — [arXiv:2509.14004](https://arxiv.org/abs/2509.14004) find
  32. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  33. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First conference on language modeling.
  34. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. find
  35. Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, Zhaoxiang Liu, and Shiguo Lian. 2025a. Dast: Difficulty-adaptive slow-thinking for large reasoning models. arXiv preprint arXiv:2503.04472. find
  36. Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. 2025b. Codi: Compressing chain-of-thought into continuous space via self-distillation. arXiv preprint arXiv:2502.21074.
  37. Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. 2024. find
  38. Trusting your evidence: Hallucinate less with context-aware decoding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 783–791.
  39. Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Na Zou, and 1 others. 2025. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419.
  40. Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, and 1 others. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599. — Kimi k1.5: Scaling Reinforcement Learning with LLMs
  41. Chenxu Wang, Chaozhuo Li, Songyang Liu, Zejian Chen, Jinyu Hou, Ji Qi, Rui Li, Litian Zhang, Qiwei Ye, Zheng Liu, and 1 others. 2026. The devil behind moltbook: Anthropic safety is always vanishing in self-evolving ai societies. arXiv preprint arXiv:2602.09877. — [arXiv:2602.09877](https://arxiv.org/abs/2602.09877) find
  42. Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, and Zhenru Zhang. 2025. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning. arXiv preprint arXiv:2506.01939.
  43. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  44. Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. 2025. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600. find
  45. Jianzhi Yan, Le Liu, Youcheng Pan, Shiwei Chen, Zike Yuan, Yang Xiang, and Buzhou Tang. 2025a. From long to lean: Performance-aware and adaptive chain-of-thought compression via multi-round refinement. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 1… find
  46. Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang, Jian Shao, and Yueting Zhuang. 2025b. Inftythink: Breaking the length limits of long-context reasoning in large language models. arXiv preprint arXiv:2503.06692.
  47. Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. 2025. Demystifying long chain-of-thought reasoning in llms. arXiv preprint arXiv:2502.03373.
  48. Jiajie Zhang, Nianyi Lin, Lei Hou, Ling Feng, and Juanzi Li. 2025a. Adaptthink: Reasoning models can learn when to think. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 3716–3730.
  49. Xiaoyun Zhang, Jingqing Ruan, Xing Ma, Yawen Zhu, Haodong Zhao, Hao Li, Jiansong Chen, Ke Zeng, and Xunliang Cai. 2025b. When to continue thinking: Adaptive thinking mode switching for efficient reasoning. Findings of the Association for Computational Linguistics: EMNLP 2025, pages 5808–5828. find
  50. Yuxiang Zhang, Zhengxu Yu, Weihang Pan, Zhongming Jin, Qiang Fu, Deng Cai, Binbin Lin, and Jieping Ye. 2025c. Tokensqueeze: Performance-preserving compression for reasoning llms. arXiv preprint arXiv:2511.13223. — [arXiv:2511.13223](https://arxiv.org/abs/2511.13223) find
  51. Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang. 2022. Learning on large-scale text-attributed graphs via variational inference. arXiv preprint arXiv:2210.14709. — [arXiv:2210.14709](https://arxiv.org/abs/2210.14709) find
  52. Wenshuo Zhao, Haoxing Zhai, Xinyu Qiu, Zhenting Qi, Shuhe Li, and Linchao Zhu. 2025. Mutis: Enhancing reasoning efficiency through multi turn intervention sampling in reinforcement learning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 13680–13692. find
  53. Zheng Zhao, Emilio Monti, Jens Lehmann, and Haytham Assem. 2024. Enhancing contextual understanding in large language models through contrastive decoding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sui-2026-think-less-know-more-state/