Elamur Research⌕
☰
home / papers / 2026 / The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents

The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents

Paper · 2026 · Weihao Xuan, Qingcheng Zeng, Heli Qi et al.

Summary

The paper asks whether tool use always worsens how well LLM agents' stated confidence matches their real accuracy, or whether the effect depends on the tool. The authors compare plain prompting, prompted tool use and RL-trained tool use with Qwen2.5-3B-Instruct. They find a split by tool type. Retrieval-style "evidence" tools such as web search push agents toward strong overconfidence. Execution-style "verification" tools such as code interpreters give deterministic feedback that helps keep confidence realistic. Their explanation is that search almost always returns something, so agents read the mere presence of passages, often sharing words with the query, as support for their answer. To fix this, they propose Calibration Agentic RL (CAR), which rewards correctness and verbalized calibration together. They also benchmark several reward designs, including a new Margin-Separated Calibration Reward (MSCR). CAR agents lower calibration error while keeping competitive accuracy. They also generalize from a local retriever to noisy API-based web search and to tool-integrated math. This matters for deploying agents that must signal uncertainty honestly.

Key points

  • The pilot study uses NQ and HotpotQA for search, with an RL variant following Search-R1 on the VeRL framework. For the code interpreter it uses AIME2024/2025 and MATH-500, with an RL variant following SimpleTIR.
  • The main pilot metric is Mean Confidence on Incorrect Predictions (MCIP), computed on questions that all settings answer wrongly and checked with t-tests. The overconfidence effect persists under both prompting and standard tool-use RL.
  • MSCR builds on RLCR (Damani et al., 2025) by keeping reward regions for correct and incorrect trajectories strictly separate. This preserves a correctness margin and prevents confidence from collapsing into an uninformative signal.
  • Full RL sweeps cover 3B–7B models. The appendix extends this by testing CAR on a 14B model and running a pilot study at 72B, both pointing to robustness across model scale.
  • Stated limitations include evaluation only on short-answer QA and math, where correctness is clear. Open-ended tasks such as long-form report writing or multi-step planning remain untested.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓The University of TokyoTokyo · Japan90 works · 9,156 citations🎓Northwestern UniversityEvanston · United States41 works · 2,194 citations🔬RIKEN Center for Advanced Intelligence ProjectTokyo · Japan15 works · 957 citations🎓Waseda UniversityTokyo · Japan20 works · 584 citations🔬RIKENWako · Japan26 works · 526 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Weihao Xuan, Qingcheng Zeng, Heli Qi, Yunze Xiao, Junjue Wang and Naoto Yokoya. 2026. The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.520

@inproceedings{xuan2026confidence,
  title = {{The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents}},
  author = {Weihao Xuan, Qingcheng Zeng, Heli Qi, Yunze Xiao, Junjue Wang and Naoto Yokoya},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.520},
  url = {https://doi.org/10.18653/v1/2026.acl-long.520}
}

References

53 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Art of Problem Solving. 2025. Aime problems and solutions. https://artofproblemsolving.com/wiki/index.php/AIME_Problems_and_ Solutions. ↗
  2. Glenn W. Brier. 1950. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1–3. find
  3. Yifei Chen, Guanting Dong, and Zhicheng Dou. 2026. Et-agent: Incentivizing effective tool-integrated reasoning agent via behavior calibration. Preprint, arXiv:2601.06860. find
  4. Polina Kirichenko, Mark Ibrahim, Kamalika Chaudhuri, and Samuel J. Bell. 2025. Abstentionbench: Reasoning llms fail on unanswerable questions. Preprint, arXiv:2506.09038. find
  5. Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, and 1 others. 2025. Browsecomp-plus: A more fair and transparent evaluation benchmark of deep-research agent. arXiv preprint arXiv:2508.06600. — [arXiv:2508.06600](https://arxiv.org/abs/2508.06600) find
  6. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural…
  7. Mehul Damani, Isha Puri, Stewart Slocum, Idan Shenfeld, Leshem Choshen, Yoon Kim, and Jacob Andreas. 2025. Beyond binary rewards: Training lms to reason about their uncertainty. Preprint, arXiv:2507.16806. find
  8. Mustafa Omer Gul, Claire Cardie, and Tanya Goyal. 2026. Mash: Modeling abstention via selective help-seeking. Preprint, arXiv:2510.01152. — [arXiv:2510.01152](https://arxiv.org/abs/2510.01152) find
  9. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 1321–1330. JMLR.org. find
  10. Lukas Haas, Gal Yona, Giovanni D’Antonio, Sasha Goldshtein, and Dipanjan Das. 2025. Simpleqa verified: A reliable factuality benchmark to measure parametric knowledge. Preprint, arXiv:2509.07968.
  11. Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. 2024. Webvoyager: Building an end-to-end web agent with large multimodal models. arXiv preprint arXiv:2401.13919.
  12. Wenyue Hua, Xianjun Yang, Mingyu Jin, Zelong Li, Wei Cheng, Ruixiang Tang, and Yongfeng Zhang. 2024. TrustAgent: Towards safe and trustworthy LLM-based agents. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 10000–10016, Miami, Florida, USA. Association for Computatio… find
  13. Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2024. Swe-bench: Can language models resolve real-world github issues? Preprint, arXiv:2310.06770.
  14. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. Preprint, arXiv:2503.09516.
  15. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In EMNLP (1), pages 6769–6781.
  16. Kuan Li, Zhongwang Zhang, Huifeng Yin, Rui Ye, Yida Zhao, Liwen Zhang, Litu Ou, Dingchu Zhang, Xixi Wu, Jialong Wu, Xinyu Wang, Zile Qiao, Zhen Zhang, Yong Jiang, Pengjun Xie, Fei Huang, and Jingren Zhou. 2025a. Websailor-v2: Bridging the chasm to proprietary agents via synthetic data and scalable r… — [arXiv:2509.13305](https://arxiv.org/abs/2509.13305) find
  17. Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baixuan Li, Zhengwei Tao, Xinyu Wang, Weizhou Shen, Junkai Zhang, Dingchu Zhang, Xixi Wu, Yong Jiang, Ming Yan, Pengjun Xie, Fei Huang, and Jingren Zhou. 2025b. Websailor: Navigating super-human reasoning for web a… find
  18. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. Preprint, arXiv:2305.20050.
  19. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Teaching models to express their uncertainty in words. Preprint, arXiv:2205.14334. find
  20. Hao Liu, Zi-Yi Dou, Yixin Wang, Nanyun Peng, and Yisong Yue. 2024. Uncertainty calibration for tool-using language agents. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16781–16805, Miami, Florida, USA. Association for Computational Linguistics. find
  21. Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, and 1 others. 2025. Deepscaler: Surpassing o1-preview with a 1.5 b model by scaling rl. Notion Blog, 3(5). find
  22. OpenAI. 2025. GPT-5 mini Model \| OpenAI API. find
  23. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  24. Yijia Shao, Humishka Zope, Yucheng Jiang, Jiaxin Pei, David Nguyen, Erik Brynjolfsson, and Diyi Yang. 2025. Future of work with ai agents: Auditing automation and augmentation potential across the u.s. workforce. Preprint, arXiv:2506.06576. find
  25. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Preprint, arXiv:2402.03300.
  26. Yuanhao Shen, Xiaodan Zhu, and Lei Chen. 2024. SMARTCAL: An approach to self-aware tool-use evaluation and calibration. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 774– 789, Miami, Florida, US. Association for Computational Linguis… find
  27. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256.
  28. Yucheng Shi, Wenhao Yu, Jingyuan Huang, Wenlin Yao, Wenhu Chen, and Ninghao Liu. 2026. Towards trustworthy gui agents: A survey. Preprint, arXiv:2503.23434. — [arXiv:2503.23434](https://arxiv.org/abs/2503.23434) find
  29. Linxin Song, Taiwei Shi, and Jieyu Zhao. 2025. The hallucination tax of reinforcement finetuning. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 2105–2120, Suzhou, China. Association for Computational Linguistics. find
  30. STUDENT. 1908. The probable error of a mean. Biometrika, 6(1):1–25. find
  31. Nishant Subramani, Jason Eisner, Justin Svegliato, Benjamin Van Durme, Yu Su, and Sam Thomson. 2025. MICE for CATs: Model-internal confidence estimation for calibrating agents with tools. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computationa… find
  32. Tongyi DeepResearch Team, Baixuan Li, Bo Zhang, Dingchu Zhang, Fei Huang, Guangyu Li, Guoxin Chen, Huifeng Yin, Jialong Wu, Jingren Zhou, Kuan Li, Liangcai Su, Litu Ou, Liwen Zhang, Pengjun Xie, Rui Ye, Wenbiao Yin, Xinmiao Yu, Xinyu Wang, and 38 others. 2025. Tongyi deepresearch technical report. P… — [arXiv:2510.24701](https://arxiv.org/abs/2510.24701) find
  33. Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher Manning. 2023. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. In Proceedings of the 2023 Conferen… find
  34. Trae Research Team, Pengfei Gao, Zhao Tian, Xiangxin Meng, Xinchen Wang, Ruida Hu, Yuanan Xiao, Yizhou Liu, Zhao Zhang, Junjie Chen, Cuiyun Gao, Yun Lin, Yingfei Xiong, Chao Peng, and Xia Liu. 2025. Trae agent: An llm-based agent for software engineering with test-time scaling. Preprint, arXiv:2507.… — [arXiv:2507.23370](https://arxiv.org/abs/2507.23370) find
  35. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2024. Text embeddings by weakly-supervised contrastive pre-training. Preprint, arXiv:2212.03533.
  36. Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus. 2024. Measuring short-form factuality in large language models. Preprint, arXiv:2411.04368.
  37. Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. 2025a. Browsecomp: A simple yet challenging benchmark for browsing agents. Preprint, arXiv:2504.12516. find
  38. Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, Hyokun Yun, and Lihong Li. 2025b. Webagent-r1: Training web agents via end-to-end multi-turn reinforcement learning. Preprint, arXiv:2505.16421.
  39. Jialong Wu, Baixuan Li, Runnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingchu Zhang, Zekun Xi, Gang Fu, Yong Jiang, Pengjun Xie, Fei Huang, and Jingren Zhou. 2025a. Webdancer: Towards autonomous information seeking agency. Preprint, arXiv:2505.22648.
  40. Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He, Deyu Zhou, Pengjun Xie, and Fei Huang. 2025b. WebWalker: Benchmarking LLMs in web traversal. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long P…
  41. Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. 2024. Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms. Preprint, arXiv:2306.13063. find
  42. Weihao Xuan, Qingcheng Zeng, Heli Qi, Junjue Wang, and Naoto Yokoya. 2025. Seeing is believing, but how much? a comprehensive analysis of verbalized calibration in vision-language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 1408–1450, Suzh…
  43. Zhenghai Xue, Longtao Zheng, Qian Liu, Yingru Li, Xiaosen Zheng, Zejun Ma, and Bo An. 2025. Simpletir: End-to-end reinforcement learning for multi-turn tool-integrated reasoning. Preprint, arXiv:2509.02479. — [arXiv:2509.02479](https://arxiv.org/abs/2509.02479) find
  44. John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik R Narasimhan, and Ofir Press. 2024. SWE-agent: Agent-computer interfaces enable automated software engineering. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
  45. John Yang, Kilian Lieret, Carlos E. Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang. 2025. Swe-smith: Scaling data for software engineering agents. Preprint, arXiv:2504.21798. find
  46. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369…
  47. Dongkeun Yoon, Seungone Kim, Sohee Yang, Sunkyoung Kim, Soyeon Kim, Yongil Kim, Eunbi Choi, Yireun Kim, and Minjoon Seo. 2025. Reasoning models better express their confidence. Preprint, arXiv:2505.14489. — [arXiv:2505.14489](https://arxiv.org/abs/2505.14489) find
  48. Miao Yu, Fanci Meng, Xinyun Zhou, Shilong Wang, Junyuan Mao, Linsey Pang, Tianlong Chen, Kun Wang, Xinfeng Li, Yongfeng Zhang, Bo An, and Qingsong Wen. 2025. A survey on trustworthy llm agents: Threats and countermeasures. Preprint, arXiv:2503.09648. — [arXiv:2503.09648](https://arxiv.org/abs/2503.09648) find
  49. Qingcheng Zeng, Weihao Xuan, Leyang Cui, and Rob Voigt. 2025. Thinking out loud: Do reasoning models know when they’re right? In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 1394–1407, Suzhou, China. Association for Computational Linguistics.
  50. Jiaxin Zhang, Caiming Xiong, and Chien-Sheng Wu. 2026. Agentic confidence calibration. Preprint, arXiv:2601.15778. — [arXiv:2601.15778](https://arxiv.org/abs/2601.15778) find
  51. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, and 1 others. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176.
  52. Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, Yuxin Gu, Sixin Hong, Jing Ren, Jian Chen, Chao Liu, and Yining Hua. 2025. find
  53. Browsecomp-zh: Benchmarking web browsing ability of large language models in chinese. Preprint, arXiv:2504.19314. — [arXiv:2504.19314](https://arxiv.org/abs/2504.19314) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xuan-2026-confidence-dichotomy-analyzing-mitigating-miscalibration/