Elamur Research⌕
☰
home / papers / 2026 / Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models

Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models

Paper · 2026 · Yuan Sui, Yufei He, Tri Cao et al.

Summary

Step-by-step reasoning in LLMs often wastes compute on dead-end solution paths. Early mistakes also tend to carry through long chains, because models seldom step back and change their overall approach. Meta-Reasoner addresses this with a separate high-level controller that runs alongside the base model during inference.

The base LLM works in rounds:

  • It writes a few chain-of-thought steps.
  • It condenses its progress into a short report.
  • A contextual multi-armed bandit (e.g., LinUCB) reads that report and picks a guiding strategy, such as continuing, backtracking, restarting, decomposing, or verifying weak steps.
  • A reward computed from the report updates the bandit policy.

This splits strategic control from token-level generation and needs no task-specific fine-tuning. On Game-of-24, TheoremQA and SciBench, the authors report accuracy 9–12% higher than prior state-of-the-art methods, with 28–35% less inference time under the same compute budget. A creative-writing experiment suggests the approach transfers beyond math and science. The work matters because it offers a cheap, plug-in way to spend a reasoning budget more wisely.

Key points

  • Bandit design: arms are diagnosis–strategy pairs. Three are standard (continue, backtrack, restart), and four more were generated by the model, e.g., resetting to a checkpoint or clarifying conflicting results. LinUCB explores less-tried arms through an uncertainty bonus.
  • Efficiency vs. baselines: with the same base models, Meta-Reasoner needs roughly 29–33% less inference time than ToT, Reflexion, MACM and Best-of-N. Compact progress reports keep the per-round token load low.
  • Base models: experiments use GPT-4o-mini and Gemini-Exp-1206.
  • Creative writing: on the ToT story task with a GPT-4 judge, it reaches a coherence score of 7.68, versus 6.19 for IO, 6.93 for CoT, and 7.91 for ToT + Prompt Refined.
  • Limitations and code: the method is limited to text-only tasks, so problems needing other modalities, such as geometry, are out of scope. Code is available at github.com/Y-Sui/Meta-reasoner.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓National University of SingaporeSingapore · Singapore180 works · 34,304 citations🎓Yale UniversityNew Haven · United States95 works · 12,765 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 works

Cite

Cite as Yuan Sui, Yufei He, Tri Cao, Sophia Simeng Han, Yulin Chen and Bryan Hooi. 2026. Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.649

@inproceedings{sui2026meta,
  title = {{Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models}},
  author = {Yuan Sui, Yufei He, Tri Cao, Sophia Simeng Han, Yulin Chen and Bryan Hooi},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.649},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.649}
}

References

41 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AI-MO Team. 2024. Aimo validation aime dataset. https://huggingface.co/datasets/ AI-MO/aimo-validation-aime. Dataset containing problems from AIME 2022-2024. ↗
  2. Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Michal Podstawski, Hubert Niewiadomski, Piotr Nyczyk, et al. 2023. Graph of thoughts: Solving elaborate problems with large language models. arXiv preprint arXiv:2308.09687. find
  3. Yuji Cao, Huan Zhao, Yuheng Cheng, Ting Shu, Guolong Liu, Gaoqi Liang, Junhua Zhao, and Yun Li. 2024. Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods. arXiv preprint. find
  4. Wenhu Chen, Ming Yin, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma, Jianyu Xu, Xinyi Wang, and Tony Xia. 2023. TheoremQA: A theorem-driven question answering dataset. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7889–7901, Singapore. Association for Comp…
  5. Aniket Rajiv Didolkar, Anirudh Goyal, Nan Rosemary Ke, Siyuan Guo, Michal Valko, Timothy P Lillicrap, Danilo Jimenez Rezende, Yoshua Bengio, Michael Curtis Mozer, and Sanjeev Arora. 2024. Metacognitive capabilities of LLMs: An exploration in mathematical problem solving. In AI for Math Workshop @ IC… find
  6. Chenghao Yang. 2024. Inference Time Compute. find
  7. Kanishk Gandhi, Denise H. J. Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, and Noah Goodman. 2024. Stream of Search (SoS): Learning to Search in Language. In First Conference on Language Modeling. find
  8. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chengg… find
  9. Peizhong Gao, Ao Xie, Shaoguang Mao, Wenshan Wu, Yan Xia, Haipeng Mi, and Furu Wei. 2024. Meta reasoning for large language models. arXiv preprint arXiv: 2406.11698. — [arXiv:2406.11698](https://arxiv.org/abs/2406.11698) find
  10. Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Yuanzhuo Wang, and Jian Guo. 2024. A survey on llm-as-a-judge. CoRR, abs/2411.15594. find
  11. Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. 2025. rstar-math: Small llms can master math reasoning with self-evolved deep thinking. arXiv preprint arXiv: 2501.04519.
  12. Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. 2023. Evoprompt: Connecting llms with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint arXiv: 2309.08532. find
  13. Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. 2025. Evoprompt: Connecting llms with evolutionary algorithms yields powerful prompt optimizers. Preprint, arXiv:2309.08532. find
  14. Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. 2023. Reasoning with language model is planning with world model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8154–8173. find
  15. Alexander Havrilla, Sharath Chandra Raparthy, Christoforos Nalmpantis, Jane Dwivedi-Yu, Maksym Zhuravinskyi, Eric Hambro, and Roberta Raileanu. 2024. Glore: When, where, and how to improve llm reasoning via global and local refinements. In ICML.
  16. Kuang-Huei Lee, Ian Fischer, Yueh-Hua Wu, Dave Marwood, Shumeet Baluja, Dale Schuurmans, and Xinyun Chen. 2025. Evolving deeper llm thinking. arXiv preprint arXiv: 2501.09891. find
  17. Bin Lei, Yi Zhang, Shan Zuo, Ali Payani, and Caiwen Ding. 2024. MACM: Utilizing a multi-agent system for condition mining in solving complex mathematical problems. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. find
  18. Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. 2024. Llms-as-judges: A comprehensive survey on llm-based evaluation methods. CoRR, abs/2412.05579.
  19. Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. 2010. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th International Conference on World Wide Web, WWW ’10, page 661–670, New York, NY, USA. Association for Computing Machinery. find
  20. Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025. Search-o1: Agentic search-enhanced large reasoning models. arXiv preprint arXiv: 2501.05366.
  21. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, I. Sutskever, and K. Cobbe. 2023. Let’s verify step by step. International Conference on Learning Representations.
  22. Zhan Ling, Yunhao Fang, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic, and Hao Su. 2023. Deductive verification of chain-of-thought reasoning. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Curran Associates Inc.
  23. Rohin Manvi, Anikait Singh, and Stefano Ermon. 2025. Adaptive inference-time compute: LLMs can predict if they can do better, even mid-generation.
  24. OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre… — OpenAI o1 System Card
  25. Bhrij Patel, Souradip Chakraborty, Wesley A. Suttle, Mengdi Wang, Amrit Singh Bedi, and Dinesh Manocha. 2024. AIME: AI system optimization via multiple LLM evaluators. find
  26. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2024. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling.
  27. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learning. In Thirty-seventh Conference on Neural Information Processing Systems.
  28. Aleksandrs Slivkins. 2019. Introduction to multi-armed bandits. Found. Trends Mach. Learn. find
  29. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv: 2408.03314.
  30. Yuan Sui, He Yufei, Ding Zifeng, and Bryan Hooi. 2025. Can knowledge graphs make large language models more trustworthy? an empirical study over open-ended question answering. In The 63rd Annual Meeting of the Association for Computational Linguistics.
  31. Xiaoxuan Wang, Ziniu Hu, Pan Lu, Yanqiao Zhu, Jieyu Zhang, Satyen Subramaniam, Arjun R. Loomba, Shichang Zhang, Yizhou Sun, and Wei Wang. 2023. Scibench: Evaluating college-level scientific problem-solving abilities of large language models. International Conference on Machine Learning.
  32. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
  33. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
  34. Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. 2023. Large language models are better reasoners with self-verification. In The 2023 Conference on Empirical Methods in Natural Language Processing.
  35. Jinyang Wu, Mingkuan Feng, Shuai Zhang, Feihu Che, Zengqi Wen, Chonghua Liao, and Jianhua Tao. 2024. Beyond examples: High-level automated reasoning paradigm in in-context learning via mcts. arXiv preprint arXiv: 2411.18478. — [arXiv:2411.18478](https://arxiv.org/abs/2411.18478) find
  36. Jinyang Wu, Mingkuan Feng, Shuai Zhang, Feihu Che, Zengqi Wen, Chonghua Liao, and Jianhua Tao. 2025. Beyond examples: High-level automated reasoning paradigm in in-context learning via mcts. Preprint, arXiv:2411.18478. — [arXiv:2411.18478](https://arxiv.org/abs/2411.18478) find
  37. Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, and Dacheng Tao. 2024. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search. arXiv preprint arXiv: 2412.18319. — [arXiv:2412.18319](https://arxiv.org/abs/2412.18319) find
  38. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik R Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. In Thirty-seventh Conference on Neural Information Processing Systems. find
  39. Ori Yoran, Tomer Wolfson, Ben Bogin, Uri Katz, Daniel Deutch, and Jonathan Berant. 2023. Answering questions by meta-reasoning over multiple chains of thought. In The 2023 Conference on Empirical Methods in Natural Language Processing.
  40. Che Zhang, Zhenyang Xiao, Chengcheng Han, Yixin Lian, and Yuejian Fang. 2024. Learning to check: Unleashing potentials for self-correction in large language models. arXiv preprint arXiv:2402.13035. — [arXiv:2402.13035](https://arxiv.org/abs/2402.13035) find
  41. Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, et al. 2025. A survey on test-time scaling in large language models: What, how, where, and how well? arXiv preprint arXiv:2503.24235. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sui-2026-meta-reasoner-dynamic-guidance-optimized/