Elamur Research⌕
☰
home / papers / 2025 / DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models

DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models

Paper · 2025 · Yi Shen, Jian Zhang, Jieyun Huang et al.

Summary

Slow-thinking reasoning models such as DeepSeek-R1 tend to "overthink." They spend hundreds or thousands of tokens on trivial questions. Existing fixes shorten reasoning uniformly, which can hurt accuracy on hard problems that genuinely need long chains of thought. DAST (Difficulty-Adaptive Slow Thinking) instead trains a model to match its Chain-of-Thought length to the difficulty of each problem. The authors define a Token Length Budget (TLB), computed from both the accuracy and the length distribution of sampled answers. The TLB acts as a per-problem target length and also as a difficulty measure. Rule-based reward scores are then adjusted against this budget. Overlong answers to easy questions are penalized, and sufficiently extended reasoning on hard ones is rewarded. From the adjusted scores the authors build pairwise budget-preference data and fine-tune the reasoning model with SimPO. Across several STEM benchmarks and two model sizes (DS-7B, DS-32B), DAST cuts token usage by more than 30% on average while keeping or improving accuracy. This shows that efficiency gains need not sacrifice performance on difficult tasks.

Key points

  • Motivating example: for the equation 3x + 7 = 22, DeepSeek V3 needs about 58 tokens, while DeepSeek-R1 uses about 1041.
  • Pipeline:
    • TLB-based reward calibration;
    • construction of preference pairs (DCP and DICP types);
    • offline SimPO training.
  • Ablation on MATH-500 with DS-7B:
    • The base model scores 93.2 ACC with an average length of about 4039 tokens.
    • DAST reaches 93.6 ACC with an 18.0% compression ratio.
    • Without DICP, compression rises to 59.8%, but accuracy falls to 90.0.
    • Without DCP, accuracy is 94.6, but outputs grow 17.8% longer.
    • Adding CICP pairs gave no notable gain.
  • Truncation threshold δ: the method is sensitive to this setting. The chosen values were 0.15 for DS-32B (about 47% compression at peak accuracy) and 0.18 for DS-7B. With δ = 0, reward hacking occurred.
  • Stated limitations:
    • Evaluation covers only STEM tasks; code and general-domain tasks are untested.
    • The threshold needs careful tuning.
    • Off-policy training may underperform online RL.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 worksComputer Vision and Pattern Recognition · 3,621 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,948 works

Cite

Cite as Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, Zhaoxiang Liu and Shiguo Lian. 2025. DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models. arXiv:2503.04472.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2503.04472 · arXiv 2503.04472

@misc{shen2025dast,
  title = {{DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models}},
  author = {Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, Zhaoxiang Liu and Shiguo Lian},
  year = {2025},
  doi = {10.48550/arxiv.2503.04472},
  eprint = {2503.04472},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2503.04472}
}

References

34 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Pranjal Aggarwal and Sean Welleck. 2025. L1: Controlling how long a reasoning model thinks with reinforcement learning. arXiv preprint arXiv:2503.04697.
  2. Daman Arora and Andrea Zanette. 2025. Training language models to reason efficiently. arXiv preprint arXiv:2502.04463.
  3. Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. 2024. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187.
  4. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  5. Tingxu Han, Chunrong Fang, Shiyu Zhao, Shiqing Ma, Zhenyu Chen, and Zhenting Wang. 2024. Token-budget-aware llm reasoning. arXiv preprint arXiv:2412.18547.
  6. Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. 2024. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769. find
  7. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
  8. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974.
  9. Yu Kang, Xianghui Sun, Liangyu Chen, and Wei Zou. 2025. C3ot: Generating shorter chain-of-thought without compromising effectiveness. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24312–24320.
  10. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. In The Twelfth International Conference on Learning Representations.
  11. Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. 2023. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. arXiv preprint arXiv:2312.15685.
  12. Yue Liu, Jiaying Wu, Yufei He, Hongcheng Gao, Hongyu Chen, Baolong Bi, Jiaheng Zhang, Zhiqi Huang, and Bryan Hooi. 2025. Efficient inference for large reasoning models: A survey. ArXiv, abs/2503.23077. — [arXiv:2503.23077](https://arxiv.org/abs/2503.23077) find
  13. Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. 2023. \# instag: Instruction tagging for analyzing supervised fine-tuning of large language models. arXiv preprint arXiv:2308.07074.
  14. Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. 2025. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning. arXiv preprint arXiv:2501.12570. find
  15. Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. 2025. Cot-valve: Length-compressible chain-of-thought tuning. arXiv preprint arXiv:2502.09601.
  16. MAA. 2024. American invitational mathematics examination - aime. In American Invitational Mathematics Examination - AIME 2024. find
  17. Yu Meng, Mengzhou Xia, and Danqi Chen. 2025. Simpo: Simple preference optimization with a reference-free reward. Advances in Neural Information Processing Systems, 37:124198–124235. find
  18. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393.
  19. Tergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang, Yujin Kim, and Se-Young Yun. 2025. Self-training elicits concise reasoning in large language models. arXiv preprint arXiv:2502.20122. find
  20. Sania Nayab, Giulio Rossolini, Giorgio Buttazzo, Nicolamaria Manes, and Fabrizio Giacomelli. 2024. Concise thoughts: Impact of output length on llm reasoning and cost. ArXiv, abs/2407.19825. — [arXiv:2407.19825](https://arxiv.org/abs/2407.19825) find
  21. OpenAI. 2024. Learning to reason with llms. find
  22. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling.
  23. Matthew Renze and Erhan Guven. 2024. The benefits of a concise chain of thought on problem-solving in large language models. In 2024 2nd International Conference on Foundation and Large Language Models (FLLM), pages 476–483. IEEE. find
  24. Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. 2025. Codi: Compressing chain-of-thought into continuous space via self-distillation. arXiv preprint arXiv:2502.21074.
  25. Yang Sui, Yu Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, and Hanjie Chen. 2025. Stop overthinking: A survey on efficient reasoning for large language models. ArXiv.
  26. Hanshi Sun, Momin Haider, Ruiqi Zhang, Huitao Yang, Jiahao Qiu, Ming Yin, Mengdi Wang, Peter Bartlett, and Andrea Zanette. 2024. Fast best-of-n decoding via speculative rejection. arXiv preprint arXiv:2410.20290.
  27. Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599. — Kimi k1.5: Scaling Reinforcement Learning with LLMs
  28. Yuxuan Tong, Xiwen Zhang, Rui Wang, Ruidong Wu, and Junxian He. 2025. Dart-math: Difficulty-aware rejection tuning for mathematical problem-solving. Advances in Neural Information Processing Systems, 37:7821–7846.
  29. Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. 2025. Tokenskip: Controllable chain-of-thought compression in llms. arXiv preprint arXiv:2502.12067.
  30. Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. 2025. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600.
  31. Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Zheng Lin, Li Cao, and Weiping Wang. 2025. Dynamic early exit in reasoning models. ArXiv.
  32. Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. 2025. Demystifying long chain-of-thought reasoning in llms. arXiv preprint arXiv:2502.03373.
  33. Zhiyuan Zeng, Qinyuan Cheng, Zhangyue Yin, Bo Wang, Shimin Li, Yunhua Zhou, Qipeng Guo, Xuanjing Huang, and Xipeng Qiu. 2024. Scaling of search and learning: A roadmap to reproduce o1 from reinforcement learning perspective. arXiv preprint arXiv:2412.14135.
  34. Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, and Ningyu Zhang. 2025. Lightthinker: Thinking step-by-step compression. arXiv preprint arXiv:2502.15589.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shen-2025-dast-difficulty-adaptive-slow-thinking/