Elamur Research⌕
☰
home / papers / 2026 / BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents

BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents

Paper · 2026 · Litu Ou, Kuan Li, Huifeng Yin et al.

Summary

The paper asks whether LLM-based web search agents can meaningfully report how sure they are after long chains of browsing and reasoning. It also asks whether such self-reports can cut the cost of test-time scaling (TTS). The authors prompted open-source agentic models, gpt-oss-120b and DeepSeek-V3.1, to attach a 0–100 verbalized confidence score to their final answers on BrowseComp. The scores were overconfident in absolute terms, but they ranked answer quality well: low-confidence answers were almost never correct, and very confident ones were much more accurate than average.

Building on this, the authors propose BrowseConf. The agent makes another attempt only when its confidence falls below a threshold, which is tuned on a SailorFog-QA validation subset. There are three variants:

  • BrowseConf-Zero restarts from scratch.
  • BrowseConf-Summary passes a summary of the previous attempt to the next one.
  • BrowseConf-Neg forbids repeating earlier low-confidence answers.

On BrowseComp and BrowseComp-zh, BrowseConf matches or beats fixed-budget Self-Consistency and CISC while using far fewer rollouts. This makes adaptive compute allocation practical for agent deployments.

Key points

  • Confidence signal: accuracy is near zero below 70% confidence. Above 95% it is more than twice the overall average. Even in the 90–94% bin, the best accuracy was only 53.33% (DeepSeek-V3.1), which shows strong overconfidence.
  • Threshold choice: τ is the smallest threshold at which answers above it give at least k% relative accuracy gain over overall validation accuracy. Going from k = 5 to 10 helps much more than going from 10 to 20.
  • gpt-oss-120b results: BrowseConf-Neg reaches 54.5% on BC with 3.87 attempts on average. This compares with 52.2% for CISC and 47.5% for Self-Consistency, both using 10 rollouts. On BC-ZH, BrowseConf-Neg reaches 54.0% with 2.43 attempts.
  • DeepSeek-V3.1 results: BrowseConf-Neg reaches 41.8% on BC, against 38.7% for CISC. On BC-ZH, BrowseConf-Zero reaches 59.2% with 3.43 attempts, close to the 59.9% of both baselines.
  • Variant trade-offs: across all settings, BrowseConf needs only 2.06–5.72 attempts on average. The Summary variant uses the fewest rollouts. The two variants that carry information forward (Summary and Neg) also sharply reduce the number of interactions after the first attempt.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢Alibaba GroupHangzhou · China201 works · 17,172 citations🏢Alibaba Group (Cayman Islands)George Town · Cayman Islands36 works · 3,645 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksInformation Systems · 1,730 works

Cite

Cite as Litu Ou, Kuan Li, Huifeng Yin, Liwen Zhang, Zhongwang Zhang, Xixi Wu, Rui Ye, Zile Qiao, Yong Jiang, Pengjun Xie, Fei Huang and Jingren Zhou. 2026. BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.21

@inproceedings{ou2026browseconf,
  title = {{BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents}},
  author = {Litu Ou, Kuan Li, Huifeng Yin, Liwen Zhang, Zhongwang Zhang, Xixi Wu, Rui Ye, Zile Qiao, Yong Jiang, Pengjun Xie, Fei Huang and Jingren Zhou},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.21},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.21}
}

References

49 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Taubenfeld Amir, Sheffer Tom, Ofek Eran, Feder Amir, Goldstein Ariel, Gekhman Zorik, and Yona Gal. 2025. Confidence improves self-consistency in llms. In Findings of the Association for Computational Linguistics: ACL 2025, page 20090–20111. Association for Computational Linguistics. find
  2. Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. 2025. τ 2 -bench: Evaluating conversational agents in a dual-control environment. Preprint, arXiv:2506.07982.
  3. Feng Chen, Allan Raventos, Nan Cheng, Surya Ganguli, and Shaul Druckmann. 2025a. Rethinking fine-tuning when scaling test-time compute: Limiting confidence improves mathematical reasoning. Preprint, arXiv:2502.07154. find
  4. Haotian Chen, Zijun Song, Boye Niu, Ke Zhang, Litu Ou, Yaxi Lu, Zhong Zhang, Xin Cong, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2025b. Toleap: Rethinking development of tool learning with large language models. Preprint, arXiv:2505.11833. — [arXiv:2505.11833](https://arxiv.org/abs/2505.11833) find
  5. Prateek Chhikara. 2025. Mind the confidence gap: Overconfidence, calibration, and distractor effects in large language models. Preprint, arXiv:2502.11028. — [arXiv:2502.11028](https://arxiv.org/abs/2502.11028) find
  6. DeepSeek-AI. 2024. Deepseek-v3 technical report. Preprint, arXiv:2412.19437. find
  7. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. Preprint, arXiv:1706.04599. find
  8. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. Preprint, arXiv:1503.02531.
  9. Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, and Jiaxin Huang. 2025. Efficient test-time scaling via self-calibration. Preprint, arXiv:2503.00031. — [arXiv:2503.00031](https://arxiv.org/abs/2503.00031) find
  10. Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations.
  11. Platt John. 2000. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv. Large Margin Classif., 10. find
  12. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, and 17 others. 2022.… find
  13. Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang. 2025. Why language models hallucinate. Preprint, arXiv:2509.04664. — [arXiv:2509.04664](https://arxiv.org/abs/2509.04664) find
  14. Sanyam Kapoor, Nate Gruver, Manley Roberts, Katherine Collins, Arka Pal, Umang Bhatt, Adrian Weller, Samuel Dooley, Micah Goldblum, and Andrew Gordon Wilson. 2025. Large language models must be taught to know what they don’t know. Preprint, arXiv:2406.08391. find
  15. Sanyam Kapoor, Nate Gruver, Manley Roberts, Arka Pal, Samuel Dooley, Micah Goldblum, and Andrew Wilson. 2024. Calibration-tuning: Teaching large language models to know what they don’t know. In Proceedings of the 1st Workshop on Uncertainty-Aware NLP (UncertaiNLP 2024), pages 1–14, St Julians, Malta… find
  16. Jannik Kossen, Jiatong Han, Muhammed Razzak, Lisa Schut, Shreshth Malik, and Yarin Gal. 2024. Semantic entropy probes: Robust and cheap hallucination detection in llms. Preprint, arXiv:2406.15927. — [arXiv:2406.15927](https://arxiv.org/abs/2406.15927) find
  17. Maia Kotelanski, Robert Gallo, Ashwin Nayak, and Thomas Savage. 2023. Methods to estimate large language model confidence. Preprint, arXiv:2312.03733. — [arXiv:2312.03733](https://arxiv.org/abs/2312.03733) find
  18. Mihir Prabhudesai, Lili Chen, Alex Ippoliti, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak. 2025. Maximizing confidence alone improves reasoning. Preprint, arXiv:2505.22660. — [arXiv:2505.22660](https://arxiv.org/abs/2505.22660) find
  19. Jixuan Leng, Chengsong Huang, Banghua Zhu, and Jiaxin Huang. 2025. Taming overconfidence in llms: Reward calibration in rlhf. Preprint, arXiv:2410.09724. — [arXiv:2410.09724](https://arxiv.org/abs/2410.09724) find
  20. Shuchang Tao, Liuyi Yao, Hanxing Ding, Yuexiang Xie, Qi Cao, Fei Sun, Jinyang Gao, Huawei Shen, and Bolin Ding. 2024. When to trust llms: Aligning confidence with response quality. Preprint, arXiv:2404.17287. find
  21. Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baixuan Li, Zhengwei Tao, Xinyu Wang, Weizhou Shen, Junkai Zhang, Dingchu Zhang, Xixi Wu, Yong Jiang, Ming Yan, Pengjun Xie, Fei Huang, and Jingren Zhou. 2025a. Websailor: Navigating super-human reasoning for web a… find
  22. Yubo Li, Xiaobin Shen, Xinyu Yao, Xueying Ding, Yidi Miao, Ramayya Krishnan, and Rema Padman. 2025b. Beyond single-turn: A survey on multi-turn interactions with large language models. Preprint, arXiv:2504.04717. — [arXiv:2504.04717](https://arxiv.org/abs/2504.04717) find
  23. Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D. Manning. 2023. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. Preprint, arXiv:2305.14975. find
  24. Xiaoyu Tian, Sitong Zhao, Haotian Wang, Shuaiting Chen, Yunjie Ji, Yiping Peng, Han Zhao, and Xiangang Li. 2025. Think twice: Enhancing llm reasoning by scaling multi-round test-time thinking. Preprint, arXiv:2503.19855. — [arXiv:2503.19855](https://arxiv.org/abs/2503.19855) find
  25. Yukun Li, Sijia Wang, Lifu Huang, and Li-Ping Liu. 2025c. Graph-based confidence calibration for large language models. Preprint, arXiv:2411.02454. — [arXiv:2411.02454](https://arxiv.org/abs/2411.02454) find
  26. Cheng Wang, Gyuri Szarvas, Georges Balazs, Pavel Danchenko, and Patrick Ernst. 2024. Calibrating verbalized probabilities for large language models. Preprint, arXiv:2410.06707. — [arXiv:2410.06707](https://arxiv.org/abs/2410.06707) find
  27. Hao Liu, Zi-Yi Dou, Yixin Wang, Nanyun Peng, and Yisong Yue. 2024. Uncertainty calibration for tool-using language agents. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16781–16805, Miami, Florida, USA. Association for Computational Linguistics. find
  28. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. Preprint, arXiv:2203.11171.
  29. Pakdaman Naeini Mahdi, Cooper Gregory, and Hauskrecht Milos. 2015. Obtaining well calibrated probabilities using bayesian binning. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1). find
  30. Jeremy Nixon, Mike Dusenberry, Ghassen Jerfel, Timothy Nguyen, Jeremiah Liu, Linchuan Zhang, and Dustin Tran. 2020. Measuring calibration in deep learning. Preprint, arXiv:1904.01685. find
  31. OpenAI. 2025. gpt-oss-120b gpt-oss-20b model card. Preprint, arXiv:2508.10925.
  32. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others.… — GPT-4 Technical Report
  33. Shishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning. find
  34. Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. 2025. Browsecomp: A simple yet challenging benchmark for browsing agents. Preprint, arXiv:2504.12516.
  35. Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He, Deyu Zhou, Pengjun Xie, and Fei Huang. 2025. Webwalker: Benchmarking llms in web traversal. Preprint, arXiv:2501.07572.
  36. Shijie Xia, Yiwei Qin, Xuefeng Li, Yan Ma, Run-Ze Fan, Steffi Chern, Haoyang Zou, Fan Zhou, Xiangkun Hu, Jiahe Jin, Yanheng He, Yixin Ye, Yixiu Liu, and Pengfei Liu. 2025. Generative ai act ii: Test time scaling drives cognition engineering. Preprint, arXiv:2504.13828. — [arXiv:2504.13828](https://arxiv.org/abs/2504.13828) find
  37. Daniel Yang, Yao-Hung Hubert Tsai, and Makoto Yamada. 2024. On verbalized confidence scores for llms. Preprint, arXiv:2412.14737. — [arXiv:2412.14737](https://arxiv.org/abs/2412.14737) find
  38. Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. 2024. τ -bench: A benchmark for tool-agent-user interaction in real-world domains. Preprint, arXiv:2406.12045.
  39. Long Phan and 1 others. 2025. Humanity’s last exam. Preprint, arXiv:2501.14249.
  40. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR).
  41. Bianca Zadrozny and Charles Elkan. 2002. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’02, page 694–699, New York, NY, USA. Association for Computing Machine… find
  42. Mozhi Zhang, Mianqiu Huang, Rundong Shi, Linsen Guo, Chong Peng, Peng Yan, Yaqian Zhou, and Xipeng Qiu. 2024. Calibrating the confidence of large language models by eliciting fidelity. Preprint, arXiv:2404.02655. find
  43. Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, Irwin King, Xue Liu, and Chen Ma. 2025a. A survey on test-time scaling in large language models: What, how, where, and how well? Preprint, arXiv:2503.24235.
  44. Tunyu Zhang, Haizhou Shi, Yibin Wang, Hengyi Wang, Xiaoxiao He, Zhuowei Li, Haoxian Chen, Ligong Han, Kai Xu, Huan Zhang, Dimitris Metaxas, and Hao Wang. 2025b. Tokur: Token-level uncertainty estimation for large language model reasoning. Preprint, arXiv:2505.11737. — [arXiv:2505.11737](https://arxiv.org/abs/2505.11737) find
  45. Zhaohan Zhang, Ziquan Liu, and Ioannis Patras. 2025c. Grace: A generative approach to better confidence elicitation in large language models. Preprint, arXiv:2509.09438. — [arXiv:2509.09438](https://arxiv.org/abs/2509.09438) find
  46. Qiwei Zhao, Xujiang Zhao, Yanchi Liu, Wei Cheng, Yiyou Sun, Mika Oishi, Takao Osaki, Katsushi Matsuda, Huaxiu Yao, and Haifeng Chen. 2024. Saup: Situation awareness uncertainty propagation on llm agent. Preprint, arXiv:2412.01033. find
  47. Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, Yuxin Gu, Sixin Hong, Jing Ren, Jian Chen, Chao Liu, and Yining Hua. 2025a. Browsecomp-zh: Benchmarking web browsing ability of large language models in chinese. Preprint, arXiv:… — [arXiv:2504.19314](https://arxiv.org/abs/2504.19314) find
  48. Ziang Zhou, Tianyuan Jin, Jieming Shi, and Qing Li. 2025b. Steerconf: Steering llms for confidence elicitation. Preprint, arXiv:2503.02863. — [arXiv:2503.02863](https://arxiv.org/abs/2503.02863) find
  49. Algorithm 1 Restart from Scratch TTS (BrowseConf-Zero) 1: Input: Query Q, confidence threshold τ , rollout budget N 2: Output: The final answer A f inal 3: Initialize: i ← 1, A best ← null, C max ← −1 4: while i ≤ N do 5: Generate answer A i and confidence score C i for query Q. 6: if C i ≥ τ then 7… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ou-2026-browseconf-confidence-guided-test-time/