Elamur Research⌕
☰
home / papers / 2026 / Tracing the Light of Thought: A Probabilistic Self- and Cross-Consistency Verification Mechanism Improving Mathematical Reasoning in LLMs

Tracing the Light of Thought: A Probabilistic Self- and Cross-Consistency Verification Mechanism Improving Mathematical Reasoning in LLMs

Paper · 2026 · Xiaoyang Liu, Dawei Wang, Tian Li et al.

Summary

Self Consistency (SC) samples many reasoning chains and returns the most common answer. It breaks down on hard math problems, where wrong answers outnumber right ones. The authors borrow an idea from computer graphics, Metropolis Light Transport, which concentrates sampling near light paths already known to carry high energy. Their Metropolis Self Consistency (MSC) turns SC into a sequential chain. Each iteration either draws a fresh solution (global mutation) or asks the model to refine an earlier accepted one (local mutation). A Metropolis-Hastings accept-reject test, scored only from token log-probabilities, then decides which paths join the pool, and majority voting picks the final answer. Metropolis Cross Consistency (MCC) runs two generator models whose chains refine each other's solutions, with a third LLM judging their proposals. Across 9 LLMs and 4 difficult math benchmarks, both methods beat SC. MCC built from models of very different strength nearly equals the strongest model alone, at lower cost. The approach needs no training and no external reward model, and any scalar correctness signal can replace the default score.

Key points

  • The correctness function relies on the observation that low probabilities on a step's first tokens signal likely errors. The authors frame this as diffuse versus specular reflection of a light path.
  • A shared random draw chooses between global and local mutation in each iteration, governed by the hyperparameter p_large. Mixing both types was found important, mirroring MLT practice.
  • In MCC, the judge compares the two proposals twice with their order swapped to counter positional bias. It acts only if both verdicts agree.
  • The judge is not treated as an oracle, since each chain still applies its own accept-reject step. When both generators reach the same answer, both paths are accepted directly.
  • Stated limitations:
    • Wall-clock time grows linearly with sample size because sampling is sequential.
    • One shared hyperparameter set (tolerance T, weights on E, D, P) is used for all models.
    • Only two simple mutation strategies are explored.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Newcastle UniversityNewcastle upon Tyne · United Kingdom18 works · 619 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 works

Cite

Cite as Xiaoyang Liu, Dawei Wang, Tian Li, Huizhi Liang, Gary Ushaw and Richard Davison. 2026. Tracing the Light of Thought: A Probabilistic Self- and Cross-Consistency Verification Mechanism Improving Mathematical Reasoning in LLMs. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1126

@inproceedings{liu2026tracing,
  title = {{Tracing the Light of Thought: A Probabilistic Self- and Cross-Consistency Verification Mechanism Improving Mathematical Reasoning in LLMs}},
  author = {Xiaoyang Liu, Dawei Wang, Tian Li, Huizhi Liang, Gary Ushaw and Richard Davison},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1126},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1126}
}

References

60 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AI-MO. 2024. AIMO Validation AIME Dataset. https://huggingface.co/datasets/AI-MO/ aimo-validation-aime. Accessed: 2025-12-22. ↗
  2. Arcee AI. 2025. Releasing five open-weights models: Supernova 70b, virtuoso-large 72b, caller 32b, glm-4-32b-base-32k, and homunculus 12b. https://www.arcee.ai/blog/ releasing-five-new-open-weights-models. [Online; accessed 2025-08-01]. ↗
  3. Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michał Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. 2024. Graph of thoughts: solving elaborate problems with large language models. In Proceedings of the Thirty-Eighth… find
  4. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  5. Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, and Benyou Wang. 2025a. Towards medical complex reasoning with LLMs through medical verifiable problems. In Findings of the Association for Computational Linguistics: ACL 2025, pages 14552–14573, Vienna, Austria. Associatio… find
  6. Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. 2025b. Do NOT think that much for 2+3=? on the overthinking of long reasoning models. In Forty-second International Conference… find
  7. Xinyun Chen, Ryan A. Chi, Xuezhi Wang, and Denny Zhou. 2024a. Premise order matters in reasoning with large language models. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org. find
  8. Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. 2024b. Self-play fine-tuning con-vertsweak language models to strong language models. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org. find
  9. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 81 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llm…
  10. Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. 2023. Complexity-based prompting for multi-step reasoning. In The Eleventh International Conference on Learning Representations.
  11. Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Pal: program-aided language models. In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org. find
  12. Olga Golovneva, Sean O’Brien, Ramakanth Pasunuru, Tianlu Wang, Luke Zettlemoyer, Maryam Fazel-Zarandi, and Asli Celikyilmaz. 2023. Pathfinder: Guided search over multi-step reasoning paths. arXiv preprint arXiv:2312.05180. find
  13. Google. 2024. Gemini 2.0. https://blog.google/technology/google-deepmind/ google-gemini-ai-update-december-2024/ ?utm_source=deepmind.google&utm_medium= referral&utm_campaign=gdm&utm_content=. ↗
  14. Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. 2025. rstar-math: Small LLMs can master math reasoning with self-evolved deep thinking. In Forty-second International Conference on Machine Learning.
  15. Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. 2023. Reasoning with language model is planning with world model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8154–8173, Singapore. Association for Computational Lingu… find
  16. W. K. Hastings. 1970. Monte carlo sampling methods using markov chains and their applications. Biometrika, 57(1):97–109. find
  17. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024a. Large language models cannot self-correct reasoning yet. In The Twelfth International Conference on Learning Representations.
  18. Zhen Huang, Haoyang Zou, Xuefeng Li, Yixiu Liu, Yuxiang Zheng, Ethan Chern, Shijie Xia, Yiwei Qin, Weizhe Yuan, and Pengfei Liu. 2024b. O1 replication journey - part 2: Surpassing o1-preview through simple distillation, big progress or bitter lesson? CoRR, abs/2411.16489. find
  19. Zhongzhen Huang, Gui Geng, Shengyi Hua, Zhen Huang, Haoyang Zou, Shaoting Zhang, Pengfei Liu, and Xiaofan Zhang. 2025. O1 replication journey -part 3: Inference-time scaling for medical reasoning. CoRR, abs/2501.06458. — [arXiv:2501.06458](https://arxiv.org/abs/2501.06458) find
  20. Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with self-feedback. In Thirty-seventh Conference on Neural Information Processing Systems. find
  21. Lingjie Jiang, Shaohan Huang, Xun Wu, and Furu Wei. 2024. Textual aesthetics in large language models. arXiv preprint arXiv:2411.02930. find
  22. Nicholas Metropolis, Arianna W Rosenbluth, Marshall N Rosenbluth, Augusta H Teller, and Edward Teller. 1953. Equation of state calculations by fast computing machines. The journal of chemical physics, 21(6):1087–1092. find
  23. Daniel Kahneman. 2011. macmillan. find
  24. Csaba Kelemen, László Szirmay-Kalos, György Antal, and Ferenc Csonka. 2002. A simple and robust mutation strategy for the metropolis light transport algorithm. In Computer Graphics Forum, volume 21, pages 531–540. Wiley Online Library. find
  25. Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, and 1 others. 2025. From system 1 to system 2: A survey of reasoning large language models. arXiv preprint arXiv:2502.17419.
  26. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2024. Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17…
  27. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s verify step by step. In The Twelfth International Conference on Learning Representations.
  28. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 158–167, Vancouver… find
  29. Zichen Liu, Changyu Chen, Wenjun Li, Tianyu Pang, Chao Du, and Min Lin. 2025. There may not be aha moment in r1-zero-like training — a pilot study. https://oatllm.notion.site/oat-zero. Notion Blog. ↗
  30. Jianqiao Lu, Zhiyang Dou, Hongru WANG, Zeyu Cao, Jianbo Dai, Yunlong Feng, and Zhijiang Guo. 2024. AutoPSV: Automated process-supervised verifier. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
  31. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Ning Miao, Yee Whye Teh, and Tom Rainforth. 2024. Selfcheck: Using LLMs to zero-shot check their own step-by-step… find
  32. Mistral AI. 2025. Mistral-Small-24B-Instruct-2501. https://huggingface.co/mistralai/ Mistral-Small-24B-Instruct-2501. Accessed: 2025-12-22. ↗
  33. Mistral team. 2025. Mistral small 3. https://mistral.ai/news/mistral-small-3. [Online; accessed 2025-08-01]. ↗
  34. OpenAI. 2024. Gpt-4o. index/hello-gpt-4o/. https://openai.com/ ↗
  35. OpenAI. 2024. Gpt-4o mini: Advancing cost-efficient intelligence. https://openai.com/index/ gpt-4o-mini-advancing-cost-efficient-intelligence/. Accessed: 2025-08-01. ↗
  36. OpenAI. 2024a. Learning to reason with LLMs. find
  37. OpenAI. 2024b. Reinforcement fine-tuning. find
  38. OpenAI. 2025a. Gpt-4.1. https://openai.com/ index/gpt-4-1/. Accessed: 2025-08-01. ↗
  39. OpenAI. 2025b. GPT-OSS-120B & GPT-OSS-20B model card. Model card.
  40. OpenAI. 2025. OpenAI o3-mini. find
  41. Zhenting Qi, Mingyuan MA, Jiahang Xu, Li Lyna Zhang, Fan Yang, and Mao Yang. 2025. Mutual reasoning makes smaller LLMs stronger problem-solver. In The Thirteenth International Conference on Learning Representations. find
  42. Yiwei Qin, Xuefeng Li, Haoyang Zou, Yixiu Liu, Shijie Xia, Zhen Huang, Yixin Ye, Weizhe Yuan, Hector Liu, Yuanzhi Li, and Pengfei Liu. 2024. O1 replication journey: A strategic progress report - part 1. CoRR, abs/2410.18982. find
  43. Zeyu Tang, Zhenhao Chen, Xiangchen Song, Loka Li, Yunlong Deng, Yifan Shen, Guangyi Chen, Peter Spirtes, and Kun Zhang. 2025. Reflection-window decoding: Text generation with selective refinement. In Forty-second International Conference on Machine Learning. find
  44. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, and 1 others. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118. find
  45. Fengwei Teng, Zhaoyang Yu, Quan Shi, Jiayi Zhang, Chenglin Wu, and Yuyu Luo. 2025. Atom of thoughts for markov llm test-time scaling. CoRR, abs/2502.12018. — [arXiv:2502.12018](https://arxiv.org/abs/2502.12018) find
  46. Together AI. 2025. Together AI Playground. https://api.together.ai/playground/chat. Accessed: 2025-12-22. ↗
  47. Robert Vacareanu, Anurag Pratik, Evangelia Spiliopoulou, Zheng Qi, Giovanni Paolini, Neha Anna John, Jie Ma, Yassine Benajiba, and Miguel Ballesteros. 2024. General purpose verification for chain of thought prompting. arXiv preprint arXiv:2405.00204. — [arXiv:2405.00204](https://arxiv.org/abs/2405.00204) find
  48. Eric Veach and Leonidas J. Guibas. 1997. Metropolis light transport. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’97, page 65–76, USA. ACM Press/Addison-Wesley Publishing Co. find
  49. Junlin Wang, Siddhartha Jain, Dejiao Zhang, Baishakhi Ray, Varun Kumar, and Ben Athiwaratkun. 2024. Reasoning in token economies: Budget-aware evaluation of LLM reasoning strategies. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 19916–19939, Miami,… find
  50. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
  51. Ling Yang, Zhaochen Yu, Bin Cui, and Mengdi Wang. 2025. Reasonflux: Hierarchical llm reasoning via scaling thought templates. arXiv preprint arXiv:2502.06772. find
  52. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik R Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. In Thirty-seventh Conference on Neural Information Processing Systems. find
  53. Yuxuan YAO, Han Wu, Zhijiang Guo, Zhou Biyan, Jiahui Gao, Sichun Luo, Hanxu Hou, Xiaojin Fu, and Linqi Song. 2024. Learning from correctness without prompting makes LLM efficient reasoner. In First Conference on Language Modeling. find
  54. Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Junqi Dai, Qinyuan Cheng, Xuanjing Huang, and Xipeng Qiu. 2024. Reasoning in flux: Enhancing large language models reasoning through uncertainty-aware adaptive guidance. In Proceedings of the 62nd Annual Meeting of the Association for C… find
  55. Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. 2024. ReST-MCTS*: LLM self-training via process reward guided tree search. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
  56. Zhi Zhou, Tan Yuhao, Zenan Li, Yuan Yao, Lan-Zhe Guo, Xiaoxing Ma, and Yu-Feng Li. 2025. Bridging internal probability and self-consistency for effective and efficient llm reasoning. arXiv preprint arXiv:2502.00511. — [arXiv:2502.00511](https://arxiv.org/abs/2502.00511) find
  57. P. C. Wason and J. S. T. B.. T. Evans. 1974. Dual processes in reasoning? Cognition, 3(2):141–154. find
  58. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems.
  59. Jinyang Wu, Mingkuan Feng, Shuai Zhang, Feihu Che, Zengqi Wen, and Jianhua Tao. 2024. Beyond examples: High-level automated reasoning paradigm in in-context learning via mcts. CoRR, abs/2411.18478. — [arXiv:2411.18478](https://arxiv.org/abs/2411.18478) find
  60. Violet Xiang, Charlie Snell, Kanishk Gandhi, Alon Albalak, Anikait Singh, Chase Blagden, Duy Phung, Rafael Rafailov, Nathan Lile, Dakota Mahan, and 1 others. 2025. Towards system 2 reasoning in llms: Learning how to think with meta chain-of-thought. arXiv preprint arXiv:2501.04682. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-tracing-light-thought-probabilistic-self/