Elamur Research⌕
☰
home / papers / 2025 / Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs

Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs

Paper · 2025 · Yue Wang; Qiuzhi Liu; Jiahao Xu; Tian Liang; Xingyu Chen; Zhiwei He; Linfeng Song; Dian Yu; Juntao Li; Zhuosheng Zhang; Rui Wang; Zhaopeng Tu; Haitao Mi; and Dong Yu

Summary

The paper examines a failure mode of o1-style reasoning models that the authors call "underthinking": the model keeps jumping to new solution strategies before it has fully pursued the current one. The study analyzes two open-source models with visible long chains of thought, QwQ-32B-Preview and DeepSeek-R1-671B, plus DeepSeek-R1-Preview. They are tested on MATH500, GPQA Diamond and AIME problems. A "thought" is treated as one reasoning segment, and a new one begins at markers such as "alternatively".

Failed answers turn out to be much longer and contain far more strategy switches than successful ones. This pattern grows stronger as problems get harder. To measure it, the authors propose an underthinking metric that captures how token-efficient incorrect responses are. As a remedy they introduce TIP, a decoding-time thought switching penalty that lowers the probability of switch tokens. This pushes the model to develop each line of reasoning more deeply. TIP raises accuracy on the hard test sets without any fine-tuning, which makes it a cheap fix that can be applied directly at inference.

Key points

  • Compared with correct responses, incorrect ones use about 225% more tokens on average, driven by roughly 418% more frequent thought switching.
  • The benchmarks are:
    • MATH500, with difficulty levels 1–5;
    • GPQA Diamond, 198 questions;
    • AIME 2024, 30 problems, extended with 60 more from AIME 2022–2023.
  • DeepSeek-R1-Preview was evaluated only on MATH500 and AIME because of a web-interface limit of 50 messages per day.
  • In one illustrative case, QwQ-32B-Preview produced 25 separate thoughts within a single solution and still gave a wrong final answer.
  • TIP is a training-free decoding strategy that penalizes thought-switching tokens. Conventional LLMs such as Qwen-Math-72B and Llama3.3-70B show no comparable length gap between their correct and incorrect answers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Shanghai Jiao Tong UniversityShanghai · China232 works · 20,305 citations🏢TencentShenzhen · China174 works · 21,355 citations🎓Soochow UniversitySuzhou · China71 works · 4,889 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 worksComputer Vision and Pattern Recognition · 3,621 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 4,145 works

Cite

Cite as Yue Wang; Qiuzhi Liu; Jiahao Xu; Tian Liang; Xingyu Chen; Zhiwei He; Linfeng Song; Dian Yu; Juntao Li; Zhuosheng Zhang; Rui Wang; Zhaopeng Tu; Haitao Mi; and Dong Yu. 2025. Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs. arXiv:2501.18585.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2501.18585 · arXiv 2501.18585

@misc{mi2025thoughts,
  title = {{Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs}},
  author = {Yue Wang; Qiuzhi Liu; Jiahao Xu; Tian Liang; Xingyu Chen; Zhiwei He; Linfeng Song; Dian Yu; Juntao Li; Zhuosheng Zhang; Rui Wang; Zhaopeng Tu; Haitao Mi; and Dong Yu},
  year = {2025},
  doi = {10.48550/arxiv.2501.18585},
  eprint = {2501.18585},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2501.18585}
}

References

27 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Carolyn Jane Anderson, Joydeep Biswas, Aleksander Boruch-Gruszecki, Federico Cassano, Molly Q Feldman, Arjun Guha, Francesca Lucchetti, and Zixuan Wu. Phd knowledge not required: A reasoning challenge for large language models. arXiv preprint arXiv:2502.01584, 2025. — [arXiv:2502.01584](https://arxiv.org/abs/2502.01584) find
  2. Daman Arora and Andrea Zanette. Training language models to reason efficiently. arXiv preprint arXiv:2502.04463, 2025.
  3. Franz Louis Cesista. Multimodal structured generation: Cvpr’s 2nd mmfm challenge technical report. arXiv preprint arXiv:2406.11403, 2024. — [arXiv:2406.11403](https://arxiv.org/abs/2406.11403) find
  4. Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Do not think that much for 2+3=? on the overthinking of o1-like llms, 2024. URL https://arxiv.org/abs/2412.21187. ↗
  5. DeepSeek. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. 2025. URL https://api.semanticscholar.org/CorpusID:275789950. ↗
  6. David Heineman, Yao Dou, and Wei Xu. Improving minimum bayes risk decoding with multi-prompt. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 22525–22545, 2024. find
  7. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In NeurIPS, 2021.
  8. Sébastien Jean, Orhan Firat, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. Montreal neural machine translation systems for wmt’15. In Proceedings of the tenth workshop on statistical machine translation, pp. 134–140, 2015. find
  9. Maxim Khanov, Jirayu Burapacheep, and Yixuan Li. Args: Alignment as reward-guided search. In The Twelfth International Conference on Learning Representations, 2024.
  10. Kimi. Kimi k1.5: Scaling reinforcement learning with llms. 2025.
  11. Philipp Koehn and Rebecca Knowles. Six challenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation, pp. 28–39, 2017. find
  12. Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. Making language models better reasoners with step-aware verifier. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguis…
  13. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2024. URL — Let's Verify Step by Step https://openreview.net/forum?id=v8L0pN6EOi. ↗
  14. MAA Committees. Aime problems and solutions. — s1: Simple test-time scaling https://artofproblemsolving.com/wiki/index.php/AIME_Problems_and_Solutions. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple t… ↗
  15. Kenton Murray and David Chiang. Correcting length bias in neural machine translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, pp. 212–223, 2018.
  16. OpenAI. Learning to reason learning-to-reason-with-llms, 2024. find
  17. Jacob Pfau, William Merrill, and Samuel R Bowman. Let’s think dot by dot: Hidden computation in transformer language models. arXiv preprint arXiv:2404.15758, 2024. find
  18. Qwen. Qwq: Reflect deeply on the boundaries of the unknown, November 2024. URL https://qwenlm.github.io/blog/qwq-32b-preview/. with llms. https://openai.com/index/ ↗
  19. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.12022, 2023.
  20. Abigail See, Peter J Liu, and Christopher D Manning. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1073–1083, 2017. find
  21. Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. Modeling coverage for neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 76–85, 2016.
  22. Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. Neural machine translation with reconstruction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017. find
  23. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id… ↗
  24. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022.
  25. Ian Wu, Patrick Fernandes, Amanda Bertsch, Seungone Kim, Sina Pakazad, and Graham Neubig. Better instruction-following through minimum bayes risk. arXiv preprint arXiv:2410.02902, 2024. find
  26. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, N… find
  27. Yuyang Wu, Yifei Wang, Tianqi Du, Stefanie Jegelka, and Yisen Wang. When more is less: Understanding chain-of-thought length in llms. arXiv preprint arXiv:2502.07266, 2025.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2025-thoughts-all-over-place-underthinking/