Elamur Research⌕
☰
home / papers / 2026 / Evolutionary Search for Automated Design of Uncertainty Quantification Methods

Evolutionary Search for Automated Design of Uncertainty Quantification Methods

Paper · 2026 · Mikhail Seleznyov, Daniil Korbut, Viktor Moskvoretskii et al.

Summary

Detecting hallucinations in large language models often relies on uncertainty quantification (UQ) scores. These scores are usually hand-crafted from researchers' intuitions, which limits how far they scale. The authors instead let an LLM invent such scores automatically. They use the EvoTune framework for LLM-driven evolutionary search. Starting from the Sequence Probability baseline, an evolution LLM repeatedly writes new unsupervised UQ methods as Python programs. These programs operate on precomputed internal states of Llama-3.1-8B-Instruct, and each candidate is scored on the PopQA training split.

On atomic claim verification across 9 datasets, the discovered methods beat strong manual baselines and transfer well to out-of-distribution data. Transfer to selective prediction is weaker, but running evolution directly on that task gives competitive results. The study also compares the search behaviour of different evolution LLMs. Claude models tend to build many-feature linear scorers, while Gpt-oss-120B favours compact schemes based on token position. The work matters for two reasons: it suggests hallucination detectors can be designed automatically while remaining readable, and it shows that frontier LLMs explore open-ended search spaces in noticeably different ways.

Key points

  • Accuracy gains: Evolved methods reach up to 6.7% relative ROC-AUC improvement over manual baselines on 9 atomic claim verification datasets. The gains are statistically significant on 6 of those 9 datasets.
  • Search setup:
    • Typically 500 evolution rounds, with 2 candidates per round, starting from Sequence Probability.
    • Evolution LLMs tested: Gpt-oss-120B and Claude Sonnet 4.0, Haiku 4.5, Sonnet 4.5, Opus 4.5 and Opus 4.6.
    • Evaluation covers 17 datasets in total: 9 for claim verification and 8 for selective prediction (e.g., GSM8K, SamSum, WMT14/19, CoQA).
  • Complexity vs. performance: Only Sonnet 4.5 and Opus 4.5 turn added method complexity into better scores. Opus 4.6 regresses compared with its predecessor.
  • Claude-designed estimators: These use 10–30 features yet still generalize well. Their weights are largely uncorrelated with the weights a logistic regression would learn.
  • Diversity of methods: Similarity to Sequence Probability was measured with Spearman correlation on 5,536 claims drawn from PopQA, X-Fact and UHead. Several evolved methods have near-zero correlation with the baseline while performing comparably, so the search finds genuinely different approaches.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓École Polytechnique Fédérale de LausanneLausanne · Switzerland44 works · 7,050 citations🎓Moscow Institute of Physics and TechnologyDolgoprudny · Russia8 works · 2,761 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksDecision Sciences · 209 worksStatistics, Probability and Uncertainty · 61 works

Cite

Cite as Mikhail Seleznyov, Daniil Korbut, Viktor Moskvoretskii, Oleg Somov, Alexander Panchenko and Elena Tutubalina. 2026. Evolutionary Search for Automated Design of Uncertainty Quantification Methods. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-srw.115

@inproceedings{seleznyov2026evolutionary,
  title = {{Evolutionary Search for Automated Design of Uncertainty Quantification Methods}},
  author = {Mikhail Seleznyov, Daniil Korbut, Viktor Moskvoretskii, Oleg Somov, Alexander Panchenko and Elena Tutubalina},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-srw.115},
  url = {https://doi.org/10.18653/v1/2026.acl-srw.115}
}

References

32 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 2025. Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers),…
  2. Anthropic. 2025a. Introducing claude 4.0. find
  3. Anthropic. 2025b. Introducing claude haiku 4.5. find
  4. Anthropic. 2025c. Introducing claude opus 4.5. find
  5. Anthropic. 2025d. Introducing claude sonnet 4.5. find
  6. Anthropic. 2026. Introducing claude opus 4.6. find
  7. Henrique Assumpção, Diego Ferreira, Leandro Campos, and Fabricio Murai. 2026. Codeevolve: an open source evolutionary coding agent for algorithmic discovery and optimization. Preprint, arXiv:2510.14150. — [arXiv:2510.14150](https://arxiv.org/abs/2510.14150) find
  8. Ping-yeh Chiang, Renkun Ni, David Yu Miller, Arpit Bansal, Jonas Geiping, Micah Goldblum, and Tom Goldstein. 2023. Loss landscapes are all you need: Neural network generalization can be explained without the implicit bias of gradient descent. In The Eleventh International Conference on Learning Repr… find
  9. Ekaterina Fadeeva, Roman Vashurin, Akim Tsvigun, Artem Vazhentsev, Sergey Petrakov, Kirill Fedyanin, Daniil Vasilev, Elizaveta Goncharova, Alexander Panchenko, Maxim Panov, Timothy Baldwin, and Artem Shelmanov. 2023. Lm-polygraph: Uncertainty estimation for language models. In Proceedings of the 202… find
  10. Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, and Lucia Specia. 2020. Unsupervised quality estimation for neural machine translation. Transactions of the Association for Computational Linguistics, 8:539–555.
  11. Tomer Galanti and Tomaso A. Poggio. 2022. SGD noise and implicit low-rank bias in deep neural networks. CoRR, abs/2206.05794. — [arXiv:2206.05794](https://arxiv.org/abs/2206.05794) find
  12. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst., 43(2):42:1– 42:55…
  13. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, and 17 others. 2022.… find
  14. Valentin Khrulkov, Andrey Galichin, Denis Bashkirov, Dmitry Vinichenko, Oleg Travkin, Roman Alferov, Andrey Kuznetsov, and Ivan Oseledets. 2025. Gigaevo: An open source optimization framework powered by llms and evolution algorithms. Preprint, arXiv:2511.17592. — [arXiv:2511.17592](https://arxiv.org/abs/2511.17592) find
  15. Robert Tjarko Lange, Yuki Imajuku, and Edoardo Cetin. 2025. Shinkaevolve: Towards open-ended and sample-efficient program evolution. CoRR, abs/2509.19349. find
  16. Kuang-Huei Lee, Ian Fischer, Yueh-Hua Wu, Dave Marwood, Shumeet Baluja, Dale Schuurmans, and Xinyun Chen. 2025. Evolving deeper llm thinking. Preprint, arXiv:2501.09891. find
  17. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  18. Jinlong Liu, Yunzhi Bai, Guoqing Jiang, Ting Chen, and Huayan Wang. 2020. Understanding why neural networks generalize well through GSNR of parameters. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. find
  19. Yixiu Liu, Yang Nan, Weixian Xu, Xiangkun Hu, Lyumanshan Ye, Zhen Qin, and Pengfei Liu. 2025. Alphago moment for model architecture discovery. CoRR, abs/2507.18074. find
  20. Viktor Moskvoretskii, Maria Marina, Mikhail Salnikov, Nikolay Ivanov, Sergey Pletenev, Daria Galimzianova, Nikita Krayko, Vasily Konovalov, Irina Nikishina, and Alexander Panchenko. 2025. Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home. In Proceedings of the 63rd Annual Mee…
  21. Alexander Novikov, Ngân Vu, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Bal… — AlphaEvolve: A coding agent for scientific and algorithmic discovery
  22. OpenAI, :, Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Haiming Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sebastien Bubeck, and 108 others. 2025. gpt-oss-120b and gpt-oss-20b mod…
  23. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  24. Ori Press, Brandon Amos, Haoyu Zhao, Yikai Wu, Samuel K. Ainsworth, Dominik Krupke, Patrick Kidger, Touqir Sajed, Bartolomeo Stellato, Jisun Park, Nathanael Bosch, Eli Meril, Albert Steppi, Arman Zharmagambetov, Fangzhao Zhang, David Perez-Pineiro, Alberto Mercurio, Ni Zhan, Talor Abramovich, and 5… — [arXiv:2507.15887](https://arxiv.org/abs/2507.15887) find
  25. Asankhaya Sharma. 2025. Openevolve: an open-source evolutionary coding agent. find
  26. Gaurang Sriramanan, Siddhant Bharti, Vinu Sankar Sadasivan, Shoumik Saha, Priyatham Kattakinda, and Soheil Feizi. 2024. LLM-Check: Investigating Detection of Hallucinations in Large Language Models. In Advances in Neural Information Processing Systems, volume 37, pages 34188–34216. Curran Associates… find
  27. Anja Surina, Amin Mansouri, Lars Quaedvlieg, Amal Seddas, Maryna Viazovska, Emmanuel Abbe, and Caglar Gulcehre. 2025. Algorithm discovery with llms: Evolutionary search meets reinforcement learning. CoRR, abs/2504.05108. find
  28. Richard S. Sutton. 2019. The bitter lesson. http://www.incompleteideas.net/IncIdeas/ BitterLesson.html. Accessed: 2026-03-15. ↗
  29. Llama Team. 2024. The llama 3 herd of models. CoRR, abs/2407.21783.
  30. Vladimir N. Vapnik. 1998. Statistical Learning Theory. Wiley-Interscience. find
  31. Artem Vazhentsev, Maria Marina, Daniil Moskovskiy, Sergey Pletenev, Mikhail Seleznyov, Mikhail Salnikov, Elena Tutubalina, Vasily Konovalov, Irina Nikishina, Alexander Panchenko, and Viktor Moskvoretskii. 2026. Leveraging llm parametric knowledge for fact checking without retrieval. Preprint, arXiv:… — [arXiv:2603.05471](https://arxiv.org/abs/2603.05471) find
  32. Artem Vazhentsev, Lyudmila Rvanova, Gleb Kuzmin, Ekaterina Fadeeva, Ivan Lazichny, Alexander Panchenko, Maxim Panov, Timothy Baldwin, Mrinmaya Sachan, Preslav Nakov, and Artem Shelmanov. 2025. Uncertainty-Aware Attention Heads: Efficient B find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/seleznyov-2026-evolutionary-search-automated-design-uncertainty/