Elamur Research⌕
☰
home / papers / 2026 / Self-Consistency from Only Two Samples: CoT–PoT Ensembling for Efficient LLM Reasoning

Self-Consistency from Only Two Samples: CoT–PoT Ensembling for Efficient LLM Reasoning

Paper · 2026 · Raman Saparkhan, Majd Hawasly, Md Rizwan Parvez et al.

Summary

Self-consistency (SC) improves LLM reasoning by majority voting over many sampled solutions, but it typically needs dozens of inference calls. Early-stopping variants reduce this cost only modestly and often lose accuracy. The authors argue that temperature sampling within a single modality produces mostly cosmetic variation. They instead pair two distinct reasoning styles: natural-language Chain-of-Thought (CoT) and executable Program-of-Thought (PoT). Because these modes tend to fail in different ways (CoT through arithmetic slips, PoT through wrong symbolic formulations), agreement between them is a strong correctness signal.

The paper proposes two families of methods. Full-budget aggregation rules combine CoT and PoT votes. Early-stopping strategies come from a Bayesian model of cross-modal agreement and can be fit on held-out data or set to extreme, data-free parameters. Across five benchmarks, five mainstream LLMs and one small model, CoT–PoT ensembling beats prior SC methods in accuracy while using far fewer samples. For most problems, a single sample from each of CoT and PoT is enough to settle the answer, which makes inference-time scaling much cheaper.

Key points

  • Full-sampling aggregation rules:
    • CP Maj: a joint majority vote over both modalities.
    • CP Max: the answer from whichever modality is more confident.
    • CP Agr: answers found in both modalities are preferred, with frequency used as a tie-breaker.
  • Early stopping: CoT and PoT samples alternate, and parallel Bayesian tests track agreement with each candidate answer. The cheapest variant halts as soon as any CoT answer matches any PoT answer.
  • Efficiency: sample usage drops by 9.3× compared with the usual 40-sample budget. On average, 78.6% of tasks need only two samples, and the share exceeds 90% for some model/benchmark pairs. Prior early-stopping methods such as adaptive consistency need at least four samples.
  • Model effects: reasoning-heavy models like DeepSeek R1 gain most from two-sample agreement. For the weak Llama 3B, the data-driven stopping rules are more robust.
  • Limitations: the method works best on math and algorithmic tasks. Early experiments on HotpotQA still show gains over CoT-only SC.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓Carnegie Mellon University QatarDoha · Qatar47 works · 2,773 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 works

Cite

Cite as Raman Saparkhan, Majd Hawasly, Md Rizwan Parvez and Mohammad Raza. 2026. Self-Consistency from Only Two Samples: CoT–PoT Ensembling for Efficient LLM Reasoning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1642

@inproceedings{saparkhan2026self,
  title = {{Self-Consistency from Only Two Samples: CoT–PoT Ensembling for Efficient LLM Reasoning}},
  author = {Raman Saparkhan, Majd Hawasly, Md Rizwan Parvez and Mohammad Raza},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1642},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1642}
}

References

30 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Pranjal Aggarwal, Aman Madaan, Yiming Yang, and Mausam. 2023. Let‘s sample step by step: Adaptive-consistency for efficient reasoning and coding with LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12375–12396, Singapore. Association for Computa… find
  2. Michael Hassid, Tal Remez, Jonas Gehring, Roy Schwartz, and Yossi Adi. 2024. The larger the better? improved llm code-generation via budget reallocation. arXiv preprint arXiv:2404.00725. — [arXiv:2404.00725](https://arxiv.org/abs/2404.00725) find
  3. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. In NeurIPS 2021.
  4. Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. 2024. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787.
  5. Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. 2022. Large language models can self-improve. arXiv preprint arXiv:2210.11610.
  6. Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. 2023. Program of thoughts Shima Imani, Liang Du, and Harsh Shrivastava. 2023. Mathprompter: Mathematical reasoning using large language models. In ACL (industry), pages 37–42. Association for Computational Linguistics.
  7. Arkil Patel, S. Bhattamishra, and Navin Goyal. 2021. Are nlp models really able to solve simple math word problems? In NAACL 2021.
  8. Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2024. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper…
  9. Robert Irvine, Douglas Boubert, Vyas Raina, Adian Liusie, Ziyi Zhu, Vineet Mudupalli, Aliaksei Korshuk, Zongyi Liu, Fritz Cremer, Valentin Assassi, and 1 others. 2023. Rewarding chatbots for real-world engagement with millions of users. arXiv preprint arXiv:2303.06135.
  10. Xinglin Wang, Shaoxiong Feng, Yiwei Li, Peiwen Yuan, Yueqi Zhang, Boyuan Pan, Heda Wang, Yao Hu, and Kan Li. 2025. Make every penny count: Difficulty-adaptive self-consistency for cost-efficient reasoning. In NAACL. find
  11. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974.
  12. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
  13. Kuang-Huei Lee, Ian Fischer, Yueh-Hua Wu, Dave Marwood, Shumeet Baluja, Dale Schuurmans, and Xinyun Chen. 2025. Evolving deeper llm thinking. arXiv preprint arXiv:2501.09891. find
  14. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, R…
  15. Yiwei Li, Peiwen Yuan, Shaoxiong Feng, Boyuan Pan, Xinglin Wang, Bin Sun, Heda Wang, and Kan Li. 2024. Escape sky-high cost: Early-stopping self-consistency for multi-step reasoning. In The Twelfth International Conference on Learning Representations. find
  16. Tengxiao Liu, Qipeng Guo, Yuqing Yang, Xiangkun Hu, Yue Zhang, Xipeng Qiu, and Zheng Zhang. 2023. Plan, verify and switch: Integrated reasoning with diverse x-of-thoughts. In EMNLP, pages 2807–2822. Association for Computational Linguistics. find
  17. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 40 others. 2025. Qwen3 technical report. CoRR, abs/2505.09388.
  18. Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, and Ashwin Kalyan. 2022. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning. Preprint, arXiv:2209.14610.
  19. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369…
  20. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, and 1 others. 2024. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36. find
  21. Murong Yue, Jie Zhao, Min Zhang, Liang Du, and Ziyu Yao. 2024a. Large language model cascades with mixture of thought representations for cost-efficient reasoning. In ICLR. OpenReview.net. find
  22. Mistral AI. 2024. Mistral large: Flagship model launch. Accessed 2025-05-13. find
  23. OpenAI. 2022. Gpt-3.5-turbo model card. Accessed 2025-05-13. find
  24. Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2024b. MAmmoTH: Building math generalist models through hybrid instruction tuning. In International Conference on Learning Representations.
  25. OpenAI. 2024. Hello gpt-4o. Accessed: 2025-05-13. find
  26. OpenAI. 2024. Learning to reason with llms. https://openai.com/index/ learning-to-reason-with-llms/. Accessed: 2024-11-20. ↗
  27. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. 2022. Star: Bootstrapping reasoning with reasoning. Preprint, arXiv:2203.14465.
  28. OpenAI. 2025. Introducing gpt-5. https://openai.com/index/introducing-gpt-5/. Accessed: 2025-08-07. ↗
  29. James Xu Zhao, Yuxi Xie, Kenji Kawaguchi, Junxian He, and Michael Qizhe Xie. 2023. Automatic model selection with large language models for reasoning. find
  30. In EMNLP (Findings), pages 758–783. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/saparkhan-2026-self-consistency-only-two-samples/