Elamur Research⌕
☰
home / papers / 2026 / SELENE: Selective and Evidence-Weighted LLM Debating for Efficient and Reliable Reasoning

SELENE: Selective and Evidence-Weighted LLM Debating for Efficient and Reliable Reasoning

Paper · 2026 · Akshay Verma, Swapnil Gupta, Deepak Gupta et al.

Summary

Multi-agent debate (MAD) lets several LLMs argue before a judge picks an answer, which can curb hallucinations. However, debating every question is expensive, and a lone judge is easily swayed by wording or verbosity. The Amazon authors propose SELENE, a training-free extension of CFMAD with two parts.

  • Selective Debate Initiation (SDI) inspects the agents' first answers. It measures semantic disagreement through embedding cosine distances, and the gap between each agent's stated confidence and its token log-probability. Only queries scoring high on either signal enter a bounded debate. That debate stops early once disagreement stops changing.
  • Evidence-Weighted Self-Consistency (EWSC) replaces the single judge with several judgments over paraphrased or subsampled evidence. Each candidate's score is weighted toward low-variance (stable) judgments.

On BoolQ, CosmosQA and a proprietary e-commerce QnA set, SELENE cuts token use by roughly half. Skipped queries lose only slightly in accuracy, and verdicts become clearly more consistent. This makes debate-style reasoning more realistic under production compute budgets.

Key points

  • SDI skip rates and token cost: SDI skips debate for 58% of BoolQ, 43% of CosmosQA and 27% of Internal-QnA queries. Relative token cost drops from 3.7× to 1.4× (BoolQ), 3.7× to 1.8× (CosmosQA) and 3.9× to 2.1× (Internal-QnA).
  • Accuracy on skipped queries: the loss versus full CFMAD debate is small, at about 0.8–1.5 points (e.g., BoolQ 83.6% → 82.1%).
  • Long debates (more than two rounds): EWSC outperforms CFMAD by +4.9 on BoolQ (86.1), +5.7 on CosmosQA (80.2) and +7.7 on Internal-QnA.
  • Judge Stability across K=3 paraphrased evidence variants: this metric is 1 minus the variance of judge scores. It rises from 0.84 to 0.93 (BoolQ), 0.79 to 0.89 (CosmosQA) and 0.72 to 0.88 (Internal-QnA).
  • Internal-QnA dataset: a 20K-sample proprietary e-commerce catalog dataset.
  • Limitation: the gating thresholds rely on logit-based signals tested on GPT-4-class models, so they may drift on smaller or instruction-tuned LLMs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksReasoning · 1,685 works

Cite

Cite as Akshay Verma, Swapnil Gupta, Deepak Gupta, Prateek Sircar and Siddharth Pillai. 2026. SELENE: Selective and Evidence-Weighted LLM Debating for Efficient and Reliable Reasoning. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-industry.7

@inproceedings{verma2026selene,
  title = {{SELENE: Selective and Evidence-Weighted LLM Debating for Efficient and Reliable Reasoning}},
  author = {Akshay Verma, Swapnil Gupta, Deepak Gupta, Prateek Sircar and Siddharth Pillai},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-industry.7},
  url = {https://doi.org/10.18653/v1/2026.eacl-industry.7}
}

References

30 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jie Chen, Bowen Zhao, Dian Yu, and Bill Yuchen Lin. 2023. Faithful chain-of-verification improves reasoning in large language models. arXiv preprint arXiv:2310.04383. — [arXiv:2310.04383](https://arxiv.org/abs/2310.04383) find
  2. Zhenyu Cui, Hao Wang, Cheng Zhang, and Jie Zhou. 2025. Free-mad: Bias-reduced multi-agent debate via aggregated trajectory voting. arXiv preprint arXiv:2502.02134. — [arXiv:2502.02134](https://arxiv.org/abs/2502.02134) find
  3. Yilun Du, Jiayuan Li, and Christopher D. Manning. 2023. Improving factuality and reasoning in language models through multi-agent debate. arXiv preprint arXiv:2305.13269.
  4. Wei Fang, Lin Chen, Rui Zhang, Ming Li, and Xiaodong Liu. 2025. Counterfactual multi-agent debate improves factuality and diversity in llm reasoning. arXiv preprint arXiv:2501.04210. — [arXiv:2501.04210](https://arxiv.org/abs/2501.04210) find
  5. Jack Hills and Sam Anadkat. 2023. Using logprobs. https://cookbook.openai.com/examples/ using_logprobs. Accessed: 2025-11-03. ↗
  6. Zeqiu Ji, Yujin Lee, Jason Fries, Danqi Chen, et al. 2023. Survey on hallucination in large language models. arXiv preprint arXiv:2309.05922.
  7. Saurav Kadavath, Andy Lin, Nicholas Schiefer, Jacob Hilton, Owain Evans, Samuel R. Bowman, and Andreas Stuhlmüller. 2022. Language models (mostly) know what they know. In Advances in Neural Information Processing Systems (NeurIPS). find
  8. Anant Khandelwal, Happy Mittal, Shreyas Sunil Kulkarni, and Deepak Gupta. 2023. Large scale generative multimodal attribute extraction for e-commerce attributes. In ACL (industry), pages 305–312. find
  9. Jie Li, Jian Zhou, Tao Lin, Han Wang, and Fang Liu. 2024. Calibrating large language models with log-probability guidance for factual reliability. arXiv preprint arXiv:2402.08032. — [arXiv:2402.08032](https://arxiv.org/abs/2402.08032) find
  10. Weizhe Liang, Yanzhe Zhang, Yixin Kwon, Tianyi Ye, Yizhong Zheng, Jason Weston, Luke Zettlemoyer, and Mark Yatskar. 2023. Encouraging divergent thinking in large language models through multi-agent debate. arXiv preprint arXiv:2305.14325.
  11. Stephanie Lin, Jacob Hilton, and Owain Evans. 2023. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL).
  12. Potsawee Manakul, Alham Liusie, and Mark J. F. Gales. 2023. Selfcheckgpt: Detecting llm hallucinations via token-level sampling. arXiv preprint arXiv:2303.08896. find
  13. Noah Shinn, Francesco Cassano, Bradley Labash, Dinesh Gopinath, Matthew Finlayson, Anca Dragan, Dorsa Sadigh, and Noah Goodman. 2023. Reflexion: Language agents with verbal reinforcement learning. arXiv preprint arXiv:2303.11366.
  14. Chenglei Si, Hongxin Xie, Fangyuan Xu, Yue Zhang, Jing Ma, and Rui Zhang. 2023. Measuring uncertainty in large language models for hallucination detection. arXiv preprint arXiv:2305.13669. find
  15. Prateek Sircar, Aniket Chakrabarti, Deepak Gupta, and Anirban Majumdar. 2022. Distantly supervised aspect clustering and naming for e-commerce reviews. In NAACL-HLT (Industry Papers), pages 94–102. find
  16. Vinay Kumar Verma, Shreyas Sunil Kulkarni, Happy Mittal, and Deepak Gupta. 2025. Moemoe: Question guided dense and scalable sparse mixture-of-expert for multi-source multi-modal answering. arXiv preprint arXiv:2503.06296. find
  17. Hao Wang, Ming Zhou, Yue Zhang, and Zhiyuan Li. 2024. Cross-examination: Improving judge reliability in multi-agent llm debates. arXiv preprint arXiv:2403.04127. — [arXiv:2403.04127](https://arxiv.org/abs/2403.04127) find
  18. Xuezhi Wang, Jason Wei, Dale Schuurmans, Ed Chi, Quoc Le, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations (ICLR).
  19. Yizhong Wang and et al. 2024. Large language models are inconsistent and biased evaluators. arXiv preprint arXiv:2405.01724.
  20. Model BoolQ CosmosQA Internal-QnA GPT-4o-mini CFMAD SELENE ∆ (vs CFMAD) 84.1 85.4 +1.3 74.0 76.1 +2.1 – – +8.6 find
  21. Yizhong Wang, Han Zhou, Tianyi Zhang, and Zhiyuan Liu. 2023. Self-contrast: Aligning large language models via contrastive self-refinement. In Empirical Methods in Natural Language Processing (EMNLP). find
  22. CFMAD Claude 3 Haiku SELENE ∆ (vs CFMAD) 83.2 84.6 +1.4 73.5 75.7 +2.2 – – +8.2 find
  23. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NeurIPS).
  24. Yuyang Wu, Yifei Wang, Tianqi Du, Stefanie Jegelka, and Yisen Wang. 2025. When more is less: Understanding chain-of-thought length in llms. ArXiv, abs/2502.07266.
  25. Rui Yang, Wei Lin, Hao Chen, and Lei Zhou. 2024. Judge summarization: Enhancing verdict quality in multi-agent llm debates. arXiv preprint arXiv:2402.01987. — [arXiv:2402.01987](https://arxiv.org/abs/2402.01987) find
  26. Wenqi Zhang, Yongliang Shen, Linjuan Wu, Qiuying Peng, Jun Wang, Yue Ting Zhuang, and Weiming Lu. 2024a. Self-contrast: Better reflection through inconsistent solving perspectives. In Annual Meeting of the Association for Computational Linguistics.
  27. Yixin Zhang, Yuchen Chen, Hao Li, Qian Liu, and Hongtao Xu. 2024b. Lm-detect: Likelihood-based hallucination detection in large language models. arXiv preprint arXiv:2404.06112. *Table 6: Cross-LLM comparison of accuracy (%) on BoolQ, CosmosQA, and Internal-QnA. Both GPT-4omini and Claude 3 Haiku ex… — [arXiv:2404.06112](https://arxiv.org/abs/2404.06112) find
  28. B All prompts are executed using GPT-4-Turbo-2025-04-09 with temperature = 0.3 and top-p = 0.9. Each box below shows the actual prompt used in SE-LENE at various stages. find
  29. Ke Zhou, Yuan Zhao, Pengfei Wang, and Chenguang Wang. 2024a. Factuality assessment of large language models via token probability entropy. arXiv preprint arXiv:2401.03455. — [arXiv:2401.03455](https://arxiv.org/abs/2401.03455) find
  30. Liang Zhou, Ying Zhu, and et al. 2024b. Larger and more instructable large language models become less robust to prompt variations. Nature, 633:679–687. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/verma-2026-selene-selective-evidence-weighted-llm/