Elamur Research⌕
☰
home / papers / 2026 / Confidence Leaps in LLM Reasoning: Early Stopping and Cross-Model Transfer

Confidence Leaps in LLM Reasoning: Early Stopping and Cross-Model Transfer

Paper · 2026 · Pavel Tikhonov, Ivan Oseledets and Elena Tutubalina

Summary

Reasoning LLMs are often assumed to become more certain gradually as they think. This paper tests that assumption on multiple-choice tasks and finds that certainty often arrives abruptly. The authors split chain-of-thought traces from Qwen3-32B, QwQ-32B and DeepSeek-R1 Distill Llama-70B into chunks at shift markers such as "Wait" or "Hmm". After each chunk, they force an answer and record the probabilities over options A–D on GPQA and LogiQA. When a model switches to the right option, that option's probability usually jumps sharply. The authors call this a "confidence leap", and it can occur at almost any position in the trace. Building on this, they propose an early-stopping rule that needs no training and no access to model internals: halt once any option's probability rises by more than a threshold between consecutive chunks. Where a leap occurs, the rule cuts reasoning tokens by about a fifth to a third without losing accuracy. Reasoning up to the leap also helps a different model, gpt-oss-20b, which suggests leaps carry transferable meaning. Code and data are public.

Key points

  • On GPQA, Qwen3-32B's probability for the correct answer rises by +0.69 on average when it switches to that answer. The distribution of changes is bimodal: large positive jumps occur almost only for correct switches.
  • At accuracy-optimal thresholds, token savings on traces where a leap occurs range from 19.2% to 32.7% (e.g., 31.5% for Qwen3-32B on GPQA with unchanged accuracy). The share of traces where the rule fires ranges from 3.5% to 54.0%.
  • DeepSeek-R1 Distill Llama-70B on GPQA gains +8.7 accuracy points (68.9% → 77.6%) while saving 24.2% of tokens. This hints that late reasoning can sometimes hurt.
  • In cross-model transfer from Qwen3-32B to gpt-oss-20b, direct-answer accuracy rises from 33.33% (two chunks before the leap) to 41.79% (one chunk before) to 70.59% (at the leap chunk). Agreement with the source model rises from 44.78% to 88.24%.
  • Letting gpt-oss-20b continue for 128 tokens adds +14.93 points from the chunk before the leap but only +1.87 from the leap chunk itself. Limitations: the method requires a finite answer set and relies on trigger-word chunking.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Skolkovo Institute of Science and TechnologySkolkovo · Russia5 works · 142 citations🤝AIRI - Artificial Intelligence Research InstituteMoscow · Russia5 works · 29 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 works

Cite

Cite as Pavel Tikhonov, Ivan Oseledets and Elena Tutubalina. 2026. Confidence Leaps in LLM Reasoning: Early Stopping and Cross-Model Transfer. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-short.43

@inproceedings{tikhonov2026confidence,
  title = {{Confidence Leaps in LLM Reasoning: Early Stopping and Cross-Model Transfer}},
  author = {Pavel Tikhonov, Ivan Oseledets and Elena Tutubalina},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-short.43},
  url = {https://doi.org/10.18653/v1/2026.eacl-short.43}
}

References

17 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anum Afzal, Florian Matthes, Gal Chechik, and Yftah Ziser. 2025. Knowing before saying: Llm representations encode information about chain-of-thought success before completion. arXiv preprint arXiv:2505.24362. find
  2. Yifan Chen, Zhaojiang Liu, Shuohang Li, and 1 others. 2024. Rethinking the cost of reasoning: Adaptive depth in chain-of-thought generation. arXiv preprint arXiv:2401.02412. find
  3. Yao Fu, Xiaochuang Tang, Hao Peng, and 1 others. 2023. Chain-of-thought early exit: When to stop thinking in large language models. arXiv preprint arXiv:2310.04369. — [arXiv:2310.04369](https://arxiv.org/abs/2310.04369) find
  4. Andrey Galichin, Alexey Dontsov, Polina Druzhinina, Anton Razzhigaev, Oleg Y. Rogov, Elena Tutubalina, and Ivan Oseledets. 2026. I have covered all the bases here: Interpreting reasoning features in large language models via sparse autoencoders. In Proceedings of the AAAI Conference on Artificial In… find
  5. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  6. Andrew K. Lampinen, Ishita Dasgupta, James L. Mc-Clelland, and Noah D. Goodman. 2024. Can language models learn from explanations in context? arXiv preprint arXiv:2401.10968.
  7. Tom Lanham, Yuhuai Wu, and Noah D. Goodman. 2023. Measuring faithfulness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702.
  8. Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2020. Logiqa: A challenge dataset for machine reading comprehension with logical reasoning. arXiv preprint arXiv:2007.08124.
  9. Xin Liu and Lu Wang. 2025. Answer convergence as a signal for early stopping in reasoning. arXiv preprint arXiv:2506.02536.
  10. Minjia Mao, Bowen Yin, Yu Zhu, and Xiao Fang. 2025. Early stopping chain-of-thoughts in large language models. arXiv preprint arXiv:2509.14004. — [arXiv:2509.14004](https://arxiv.org/abs/2509.14004) find
  11. QwenTeam. 2025. Qwq-32b: Embracing the power of reinforcement learning. https://qwen.ai/blog?id= 6aed6aa257238a0b6c77a6753f180350c2fecc4a& Accessed: from=research.research-list. 2025-10-04. ↗
  12. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling.
  13. Charlie Snell, James Thorne, and Roger Grosse. 2023. Reducing the cost of reasoning via confidence-based early stopping. arXiv preprint arXiv:2310.08838. — [arXiv:2310.08838](https://arxiv.org/abs/2310.08838) find
  14. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  15. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  16. Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, and He He. 2025. Reasoning models know when they’re right: Probing hidden states for self-verification. arXiv preprint arXiv:2504.05419. find
  17. Probability Change at the Critical Chunk A This section contains figures analyzing the change in probability for the correct answer at the identified critical chunk. *Figure 9: Distribution of probability change at the critical chunk for DeepSeek-R1 on LogiQA.* *Figure 5: Scatter plot of probability… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tikhonov-2026-confidence-leaps-llm-reasoning-early/