Elamur Research⌕
☰
home / papers / 2025 / Guiding Reasoning in Small Language Models with LLM Assistance

Guiding Reasoning in Small Language Models with LLM Assistance

Paper · 2025 · Yujin Kim, Euiin Yi, Minu Kim et al.

Summary

Small language models (SLMs) are cheap to run but often break down on problems that need long chains of reasoning. The authors propose SMART (Small Reasons, Large Hints), which lets an SLM do most of the work and calls a larger LLM only for steps that look unreliable. The idea borrows from scaffolding in cognitive science, where a learner gets help only when needed. As the SLM writes its solution step by step, each step gets a score from either a Process Reward Model (PRM) or the SLM's mean token probability, called token-level confidence (TLC). When a step scores below a threshold, the LLM writes a new version of that step, keeping all earlier steps, and the SLM then continues. SMART can be combined with Best-of-N and Beam Search. The authors test it on math reasoning (MATH500) with the Qwen and Llama model families. SLMs reach nearly LLM-level accuracy while the LLM produces far fewer tokens. This makes hybrid on-device and cloud setups more practical.

Key points

  • SMART reaches up to 98.9% of LLM accuracy and cuts LLM token usage by up to 90.0%.
  • The tested SLM–LLM pairs are Qwen 1.5B→7B and Llama 1B→8B. Results are reported as Weighted@N, sampled at temperature 0.8 and averaged over 16 samples.
  • On MATH500 with N=16, the largest gains come on the hardest problems:
    • Qwen 1.5B-7B with Best-of-N reaches about 99% of the 7B model's accuracy on the top difficulty tier, a +73% gain over the plain SLM.
    • Llama 1B-8B with Best-of-N improves by about +142% over the plain SLM on the same tier.
  • The authors' analysis shows that LLM interventions matter most in the early steps of a solution.
  • Code is available at github.com/euiin/SMART. A stated limitation is that the threshold controls LLM involvement only indirectly. The authors suggest RL-based or uncertainty-aware triggering as future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Korea Advanced Institute of Science and TechnologyDaejeon · South Korea51 works · 3,167 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 works

Cite

Cite as Yujin Kim, Euiin Yi, Minu Kim, Se-Young Yun and Taehyeon Kim. 2025. Guiding Reasoning in Small Language Models with LLM Assistance. arXiv:2504.09923.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2504.09923 · arXiv 2504.09923

@misc{kim2025guiding,
  title = {{Guiding Reasoning in Small Language Models with LLM Assistance}},
  author = {Yujin Kim, Euiin Yi, Minu Kim, Se-Young Yun and Taehyeon Kim},
  year = {2025},
  doi = {10.48550/arxiv.2504.09923},
  eprint = {2504.09923},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2504.09923}
}

References

43 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tobias Gerstenberg and Simon Stephan. 2021. A counterfactual simulation model of causation by omission. Cognition, 216:104842. find
  2. Peter W Battaglia, Jessica B Hamrick, and Joshua B Tenenbaum. 2013. Simulation as an engine of physical scene understanding. Proceedings of the National Academy of Sciences, 110(45):18327–18332. find
  3. Nathan Godey, Éric de la Clergerie, and Benoît Sagot. 2024. Why do small language models underperform? studying language model saturation via the softmax bottleneck. arXiv preprint arXiv:2404.07647.
  4. Jing Bi, Yuting Wu, Weiwei Xing, and Zhenjie Wei. 2024. Enhancing the reasoning capabilities of small language models via solution guidance fine-tuning. arXiv preprint arXiv:2412.09906.
  5. Melody Y Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Heylar, Rachel Dias, Andrea Vallone, Hongyu Ren, Jason Wei, et al. 2024. Deliberative alignment: Reasoning enables safer language models. arXiv preprint arXiv:2412.16339.
  6. IBM Think Blog. 2024. Small language models. Accessed: 2025-02-16. find
  7. Edoardo Cetin and Oya Celiktutan. 2022. Policy gradient with serial markov chain reasoning. Advances in Neural Information Processing Systems, 35:8824– 8839. find
  8. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  9. Guoxin Chen, Kexin Tang, Chao Yang, Fuying Ye, Yu Qiao, and Yiming Qian. 2024a. Seer: Facilitating structured reasoning and explanation via reinforcement learning. arXiv preprint arXiv:2401.13246. find
  10. Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, and Jiaxin Huang. 2025. Efficient test-time scaling via self-calibration. arXiv preprint arXiv:2503.00031. — [arXiv:2503.00031](https://arxiv.org/abs/2503.00031) find
  11. Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. 2024b. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187.
  12. Daniel Kahneman. 2011. Thinking, fast and slow. Farrar, Straus and Giroux. find
  13. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. find
  14. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  15. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393.
  16. Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. 2025. Llms get lost in multi-turn conversation. arXiv preprint arXiv:2505.06120.
  17. Avanika Narayan, Dan Biderman, Sabri Eyuboglu, Avner May, Scott Linderman, James Zou, and Christopher Re. 2025. Minions: Cost-efficient collaboration between on-device and cloud language models. arXiv preprint arXiv:2502.15964. — [arXiv:2502.15964](https://arxiv.org/abs/2502.15964) find
  18. Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman. 2017. Building machines that learn and think like people. Behavioral and brain sciences, 40:e253. find
  19. Yaniv Nikankin, Anja Reusch, Aaron Mueller, and Yonatan Belinkov. 2024. Arithmetic without algorithms: Language models solve math with a bag of heuristics. arXiv preprint arXiv:2410.21272. find
  20. Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274–19286. PMLR. find
  21. Allen Z Ren, Justin Lidard, Lars L Ankile, Anthony Simeonov, Pulkit Agrawal, Anirudha Majumdar, Benjamin Burchfiel, Hongkai Dai, and Max Simchowitz. 2024. Diffusion policy policy optimization. arXiv preprint arXiv:2409.00588. — [arXiv:2409.00588](https://arxiv.org/abs/2409.00588) find
  22. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023a. Let’s verify step by step. arXiv preprint arXiv:2305.20050.
  23. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  24. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023b. Let’s verify step by step. arXiv preprint arXiv:2305.20050.
  25. Pratyusha Sharma, Jordan T Ash, and Dipendra Misra. 2023. The truth is in there: Improving reasoning in language models with layer-selective rank reduction. arXiv preprint arXiv:2312.13558. — [arXiv:2312.13558](https://arxiv.org/abs/2312.13558) find
  26. Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A Smith, and Yejin Choi. 2021. Dexperts: Decoding-time controlled text generation with experts and anti-experts. arXiv preprint arXiv:2105.03023.
  27. Shannon Zejiang Shen, Hunter Lang, Bailin Wang, Yoon Kim, and David Sontag. 2024a. Learning to decode collaboratively with multiple language models. arXiv preprint arXiv:2403.03870. find
  28. Yuliang Liu, Junjie Lu, Zhaoling Chen, Chaofeng Qu, Jason Klein Liu, Chonghan Liu, Zefan Cai, Yunhui Xia, Li Zhao, Jiang Bian, et al. 2025. Adaptivestep: Automatically dividing reasoning step through model confidence. arXiv preprint arXiv:2502.13943. — [arXiv:2502.13943](https://arxiv.org/abs/2502.13943) find
  29. Weizhou Shen, Chenliang Li, Hongzhan Chen, Ming Yan, Xiaojun Quan, Hehong Chen, Ji Zhang, and Fei Huang. 2024b. Small llms are weak tool learners: A multi-llm agent. arXiv preprint arXiv:2401.07324.
  30. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314.
  31. Ali Madani, Ben Krause, Eric R Greene, Subu Subramanian, Benjamin P Mohr, James M Holton, Jose Luis Olmos, Caiming Xiong, Zachary Z Sun, Richard Socher, et al. 2023. Large language models generate functional protein sequences across diverse families. Nature Biotechnology, 41(8):1099–1106. find
  32. Subhash Talluri, Carlos Fernandez Casares, Deepak Rupakula, Mohammad Zoualfaghari, and Parham Beheshti. 2024. Aws for industries: Opportunities for telecoms with small language models. Accessed: 2025-02-16. find
  33. Sara Vera Marjanović, Arkil Patel, Vaibhav Adlakha, Milad Aghajohari, Parishad BehnamGhader, Mehar Bhatia, Aditi Khandelwal, Austin Kraft, Benno Krojer, Xing Han Lù, et al. 2025. Deepseek-r1 thoughtology: Let’s\< think\> about llm reasoning. arXiv preprint arXiv:2504.07128. find
  34. Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023. Large language models in medicine. Nature medicine, 29(8):1930– 1940. find
  35. Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. 2024. Gsm-symbolic: Understanding the limitations of mathematical reasoning in large language models. arXiv preprint arXiv:2410.05229.
  36. Xuezhi Wang and Denny Zhou. 2024. Chain-of-thought reasoning without prompting. arXiv preprint arXiv:2402.10200. find
  37. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. In The Thirty-eight Conference on Neural Information Processing Systems…
  38. Henry M Wellman and Susan A Gelman. 1992. Cognitive development: foundational theories of core domains. Annual review of psychology. find
  39. Borui Xu, Yao Chen, Zeyi Wen, Weiguo Liu, and Bingsheng He. 2025. Evaluating small language models for news summarization: Implications and factors influencing performance. arXiv preprint arXiv:2502.00641. find
  40. Shangzi Xue, Zhenya Huang, Jiayu Liu, Xin Lin, Yuting Ning, Binbin Jin, Xin Li, and Qi Liu. 2024. Decompose, analyze and rethink: Solving intricate problems with human-like reasoning cycle. Advances in Neural Information Processing Systems, 37:357–385. find
  41. Ling Yang, Zhaochen Yu, Bin Cui, and Mengdi Wang. 2025. Reasonflux: Hierarchical llm reasoning via scaling thought templates. arXiv preprint arXiv:2502.06772. find
  42. Euiin Yi, Taehyeon Kim, Hongseok Jeung, Du-Seong Chang, and Se-Young Yun. 2024. Towards fast multilingual llm inference: Speculative decoding and specialized drafters. arXiv preprint arXiv:2406.16758. find
  43. Thomas Zeng, Shuibai Zhang, Shutong Wu, Christian Classen, Daewon Chae, Ethan Ewer, Minjae Lee, Heeju Kim, Wonjun Kang, Jackson Kunde, et al. 2025. Versaprm: Multi-domain process reward model via synthetic reasoning data. arXiv preprint arXiv:2502.06737. — [arXiv:2502.06737](https://arxiv.org/abs/2502.06737) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kim-2025-guiding-reasoning-small-language-models/