Elamur Research⌕
☰
home / papers / 2026 / AdapThink: Adaptive Thinking Preferences for Reasoning Language Models

AdapThink: Adaptive Thinking Preferences for Reasoning Language Models

Paper · 2026 · Wenyue Xu, Xu Wan, Wei Wang et al.

Summary

RL-trained reasoning LLMs waste tokens on easy problems through redundant self-checking. On hard problems they often abandon promising lines of reasoning too early. AdapThink is a post-training framework that replaces fixed length budgets with preferences derived from each group of sampled responses.

For each prompt, the share of correct answers in the group estimates difficulty. Responses are split into correct and incorrect subsets, and each response is rewarded for being shorter and less reflective than its subset's average. A cosine weight disables this pressure when group accuracy falls below a threshold, so hard problems are trained purely for correctness. The pressure grows as problems become easier. Reflection is counted with a BERT segment classifier rather than keyword matching.

A second component oversamples candidates and keeps a subset that maximizes the log-determinant of the covariance of length and reflection features. This diversifies reasoning patterns while keeping correctness balanced.

On math and code benchmarks with 1.5B–32B models, AdapThink produces shorter outputs with higher accuracy. It also avoids the repetition-based reward hacking seen with length penalties, which makes reasoning models cheaper and more reliable.

Key points

  • Under 32K-token budgets, average response length drops by 17.1–21.4% and accuracy rises by 6.12–6.59 points versus base models.
  • The reflection detector is a BERT classifier trained on 11,620 segments of 256 tokens each, labeled by DeepSeek-R1-Distill-Qwen-32B (45.3% reflective). It reaches 98.86% F1. Over 95% of wrong answers from the DeepSeek-R1-Distill-Qwen 1.5B/7B models on MATH-500 showed heavy reflection.
  • On code with a 32B DRIVE-SFT model (24K context), φ_thr = 0.1 cuts tokens by 15.4% for +2.70 points. φ_thr = 0.25 gives +4.63 points on LiveCode-V6.
  • N-gram repetition, a sign of reward hacking, is 0.7% for AdapThink versus 10.8% for LCPO (19.8% on LCPO's incorrect answers).
  • On out-of-distribution benchmarks (LSAT, GPQA), AdapThink adds 2.77 points and uses 26.7% fewer tokens. Ablations show that penalizing "switch" words helps, while penalizing "check" words hurts performance.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China315 works · 43,268 citations🎓Zhejiang UniversityHangzhou · China224 works · 13,587 citations🏢Alibaba DAMO AcademyHangzhou · China45 works · 6,874 citations🎓Tongji UniversityShanghai · China33 works · 2,993 citations

Also on the byline: China Southern Power Grid.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksReasoning · 1,685 works

Cite

Cite as Wenyue Xu, Xu Wan, Wei Wang, Wenqi Huang, Wotao Yin, Shengjie Zhao and Mingyang Sun. 2026. AdapThink: Adaptive Thinking Preferences for Reasoning Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.477

@inproceedings{xu2026adapthink,
  title = {{AdapThink: Adaptive Thinking Preferences for Reasoning Language Models}},
  author = {Wenyue Xu, Xu Wan, Wei Wang, Wenqi Huang, Wotao Yin, Shengjie Zhao and Mingyang Sun},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.477},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.477}
}

References

33 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Pranjal Aggarwal and Sean Welleck. 2025. L1: Controlling how long a reasoning model thinks with reinforcement learning. arXiv preprint arXiv:2503.04697.
  2. Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. 2024. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187.
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
  4. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  5. Rui Ha, Chaozhuo Li, Rui Pu, and Sen Su. 2025. From" aha moments" to controllable thinking: Toward metacognitive reasoning in large reasoning models via decoupled reasoning and control. arXiv preprint arXiv:2508.04460. — [arXiv:2508.04460](https://arxiv.org/abs/2508.04460) find
  6. Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. 2024. Token-budget-aware llm reasoning. arXiv preprint arXiv:2412.18547.
  7. Mingyu Jin, Qinkai Yu, Dong Shu, Haiyan Zhao, Wenyue Hua, Yanda Meng, Yongfeng Zhang, and Mengnan Du. 2024. The impact of reasoning step length on large language models. In ACL (Findings).
  8. Yu Kang, Xianghui Sun, Liangyu Chen, and Wei Zou. 2025. C3ot: Generating shorter chain-of-thought without compromising effectiveness. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24312–24320.
  9. Amirhossein Kazemnejad, Milad Aghajohari, Eva Portelance, Alessandro Sordoni, Siva Reddy, Aaron Courville, and Nicolas Le Roux. 2024. Vineppo: Unlocking rl potential for llm reasoning through refined credit assignment. arXiv preprint arXiv:2410.01679.
  10. Komal Kumar, Tajamul Ashraf, Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, Phillip HS Torr, Fahad Shahbaz Khan, and Salman Khan. 2025. Llm post-training: A deep dive into reasoning large language models. arXiv preprint arXiv:2502.21321. — [arXiv:2502.21321](https://arxiv.org/abs/2502.21321) find
  11. Yuliang Liu, Junjie Lu, Zhaoling Chen, Chaofeng Qu, Jason Klein Liu, Chonghan Liu, Zefan Cai, Yunhui Xia, Li Zhao, Jiang Bian, Chuheng Zhang, Wei Shen, and Zhouhan Lin. 2025. Adaptivestep: Automatically dividing reasoning step through model confidence. arXiv preprint arXiv:2502.13943. — [arXiv:2502.13943](https://arxiv.org/abs/2502.13943) find
  12. Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. 2025a. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning. arXiv preprint arXiv:2501.12570. find
  13. Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. 2025b. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. Notion Blog. find
  14. Yan Ma, Steffi Chern, Xuyang Shen, Yiran Zhong, and Pengfei Liu. 2025. Rethinking rl scaling for vision language models: A transparent, from-scratch framework and comprehensive evaluation scheme. arXiv preprint arXiv:2504.02587. — [arXiv:2504.02587](https://arxiv.org/abs/2504.02587) find
  15. Ning Miao, Yee Whye Teh, and Tom Rainforth. 2023. Selfcheck: Using llms to zero-shot check their own step-by-step reasoning. arXiv preprint arXiv:2308.00436. thinking tokens improves reasoning efficiency. arXiv preprint arXiv:2506.08343.
  16. Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. 2025b. Thoughts are all over the place: On the underthinking of o1-like llms. arXiv preprint arXiv:2501.18585. find
  17. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393.
  18. Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. 2023. Large language models are better reasoners with self-verification. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2550–2575.
  19. Sania Nayab, Giulio Rossolini, Marco Simoni, Andrea Saracino, Giorgio Buttazzo, Nicolamaria Manes, and Fabrizio Giacomelli. 2024. Concise thoughts: Impact of output length on llm reasoning and cost. arXiv preprint arXiv:2407.19825. — [arXiv:2407.19825](https://arxiv.org/abs/2407.19825) find
  20. Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. 2025. Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2502.14768.
  21. OpenAI. 2024. Learning to reason with LLMs. https://openai.com/index/ learning-to-reason-with-llms. Accessed: 2024. ↗
  22. Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. 2025. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600.
  23. Matthew Renze and Erhan Guven. 2024a. The benefits of a concise chain of thought on problem-solving in large language models. In 2024 2nd International Conference on Foundation and Large Language Models (FLLM), pages 476–483. IEEE. find
  24. Matthew Renze and Erhan Guven. 2024b. Self-reflection in llm agents: Effects on problem-solving performance. arXiv preprint arXiv:2405.06682.
  25. William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. 2022. Self-critiquing models for assisting human evaluators. arXiv preprint arXiv:2206.05802.
  26. Maohao Shen, Guangtao Zeng, Zhenting Qi, Zhang-Wei Hong, Zhenfang Chen, Wei Lu, Gregory Wornell, Subhro Das, David Cox, and Chuang Gan. 2025a. Satori: Reinforcement learning with chain-of-action-thought enhances llm reasoning via autoregressive search. arXiv preprint arXiv:2502.02508. find
  27. Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Qiaowei Li, Minghui Chen, Zheng Lin, and Weiping Wang. 2025. Dynamic early exit in reasoning models. arXiv preprint arXiv:2504.15895.
  28. Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. 2025. Demystifying long chain-of-thought reasoning in llms. arXiv preprint arXiv:2502.03373.
  29. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, and 1 others. 2025. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476.
  30. Speed Zhu, Jianwei Cai, Guang Chen, Lulu Wu, Saiyong Yang, and Wiggin Zhou. 2025. Drive: Data curation best practices for reinforcement learning with verifiable reward in competitive code generation. arXiv preprint arXiv:2511.06307. — [arXiv:2511.06307](https://arxiv.org/abs/2511.06307) find
  31. Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, Zhaoxiang Liu, and Shiguo Lian. 2025b. Dast: Difficulty-adaptive slow-thinking for large reasoning models. arXiv preprint arXiv:2503.04472. find
  32. Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, and Xia Hu. 2025. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419.
  33. Chenlong Wang, Yuanning Feng, Dongping Chen, Zhaoyang Chu, Ranjay Krishna, and Tianyi Zhou. 2025a. Wait, we don’t need to" wait"! removing find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xu-2026-adapthink-adaptive-thinking-preferences-reasoning/