Elamur Research⌕
☰
home / papers / 2026 / Beyond “I Don't Know”: Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty

Beyond “I Don't Know”: Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty

Paper · 2026 · Jingyi Ren, Ante Wang, Yunghwei Lai et al.

Summary

Most work on LLM abstention treats every refusal the same way. This paper argues that a reliable model also needs to know why it is unsure. It separates two sources of uncertainty. Data uncertainty means the question is ambiguous or missing information, which calls for asking the user to clarify. Model uncertainty means the question has a unique answer that the model cannot reach alone, which calls for using a tool.

To measure this, the authors built UA-Bench, which mixes answerable and unanswerable items from six existing datasets. Models must either answer or emit an explicit uncertainty label. Model uncertainty is not annotated in advance; it is decided per model, based on which answerable questions that model fails. Testing 18 frontier LLMs showed that even the strongest systems often confuse the two categories, and that answer accuracy does not predict good attribution.

The authors also propose RL-UA, a reinforcement learning method trained only on synthetic rewrites of math problems. It improves attribution on all UA-Bench settings without lowering accuracy. This makes uncertainty signals more useful for deciding what an LLM agent should do next.

Key points

  • UA-Bench has 3,505 questions in two groups:
    • Knowledge-intensive (2,197): GAIA (165 answerable), MuSiQue (1,000 answerable) and SelfAware (1,032 unanswerable).
    • Reasoning-intensive (1,308): OlympiadBench-math (674 answerable), GSM8K-MiP (582 unanswerable) and MATH-MiP (52 unanswerable).
  • Each model must give an answer, <DATA_UNCERTAIN> or <MODEL_UNCERTAIN>, after reasoning step by step.
  • Among the 18 evaluated LLMs, larger closed-source models generally score higher on uncertainty F1. Thinking-enabled models often attribute uncertainty worse despite stronger reasoning.
  • RL-UA was applied to Qwen3-4B-Instruct-2507 and to Qwen3-8B in thinking mode. On reasoning-intensive tasks, MU-F1 reached 53.5% and 60.8% respectively, still far from saturation.
  • Stated limitations: the two categories are treated as mutually exclusive, and the automatically synthesized training data carries label noise. Code and data are available at github.com/ren258/UA-Bench.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Jingyi Ren, Ante Wang, Yunghwei Lai, Xiaolong Wang, Linlu Gong, Weitao Li, Weizhi Ma and Yang Liu. 2026. Beyond “I Don't Know”: Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.547

@inproceedings{ren2026beyond,
  title = {{Beyond “I Don't Know”: Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty}},
  author = {Jingyi Ren, Ante Wang, Yunghwei Lai, Xiaolong Wang, Linlu Gong, Weitao Li, Weizhi Ma and Yang Liu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.547},
  url = {https://doi.org/10.18653/v1/2026.acl-long.547}
}

References

53 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gustaf Ahdritz, Tian Qin, Nikhil Vyas, Boaz Barak, and Benjamin L. Edelman. 2024. Distinguishing the knowable from the unknowable with language models. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net. find
  2. Alfonso Amayuelas, Kyle Wong, Liangming Pan, Wenhu Chen, and William Yang Wang. 2024. Knowledge of knowledge: Exploring known-unknowns uncertainty with large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 6416–6432, Bangkok, Thailand. Association for C…
  3. Anthropic. 2025. The claude 4 model family: Opus, sonnet, and haiku. https://www.anthropic.com/ research/claude-4-technical-report. ↗
  4. AI at Meta. 2025. Llama 4 model card. find
  5. Youssef Benchekroun, Megi Dervishi, Mark Ibrahim, Jean-Baptiste Gaya, Xavier Martinet, Grégoire Mialon, Thomas Scialom, Emmanuel Dupoux, Dieuwke Hupkes, and Pascal Vincent. 2023. Worldsense: A synthetic benchmark for grounded reasoning in large language models. ArXiv preprint, abs/2311.15930.
  6. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. Training verifiers to solve math word problems. ArXiv preprint, abs/2110.14168.
  7. Amin Dada, Osman Alperen Koraş, Marie Bauer, Jean-Philippe Corbeil, Amanda Butler Contreras, Constantin Marc Seibold, Kaleb E Smith, Julian Friedrich, and Jens Kleesiek. 2025. Does biomedical training lead to better medical performance? In Proceedings of the Fourth Workshop on Generation, Evaluatio… find
  8. Mehul Damani, Isha Puri, Stewart Slocum, Idan Shenfeld, Leshem Choshen, Yoon Kim, and Jacob Andreas. 2025. Beyond binary rewards: Training lms to reason about their uncertainty. ArXiv preprint, abs/2507.16806. find
  9. Yang Deng, Wenqiang Lei, Wai Lam, and Tat-Seng Chua. 2023. A survey on proactive dialogue systems: Problems, methods, and prospects. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI 2023, 19th-25th August 2023, Macao, SAR, China, pages 6583–6591. i… find
  10. Yang Deng, Yong Zhao, Moxin Li, See-Kiong Ng, and Tat-Seng Chua. 2024. Don’t just say “I don’t know”! self-aligning large language models for responding to unknown questions with explanations. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13652–1367… find
  11. Yihong Dong, Xue Jiang, Yongding Tao, Huanyu Liu, Kechi Zhang, Lili Mou, Rongyu Cao, Yingwei Ma, Jue Chen, Binhua Li, and 1 others. 2025. Countering capability boundary collapse of llms in reinforcement learning with hybrid-policy optimization. ArXiv preprint, abs/2508.00222. — [arXiv:2508.00222](https://arxiv.org/abs/2508.00222) find
  12. Ahmed Elhady, Eneko Agirre, and Mikel Artetxe. 2025. WiCkeD: A simple method to make multiple choice benchmarks more challenging. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1183–1192, Vienna, Austria. Association for Com…
  13. Chenrui Fan, Ming Li, Lichao Sun, and Tianyi Zhou. 2025. Missing premise exacerbates overthinking: Are reasoning models losing critical thinking skill? ArXiv preprint, abs/2504.06514. — [arXiv:2504.06514](https://arxiv.org/abs/2504.06514) find
  14. Ruth Garner and Patricia A Alexander. 1989. Metacognition: Answered and unanswered questions. Educational psychologist, 24(2):143–158. find
  15. Jiahui Geng, Fengyu Cai, Yuxia Wang, Heinz Koeppl, Preslav Nakov, and Iryna Gurevych. 2024. A survey of confidence estimation and calibration in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language… find
  16. Linlu Gong, Ante Wang, Yunghwei Lai, Weizhi Ma, and Yang Liu. 2025. The dialogue that heals: A comprehensive evaluation of doctor agents’ inquiry capability. ArXiv preprint, abs/2509.24958. — [arXiv:2509.24958](https://arxiv.org/abs/2509.24958) find
  17. Google DeepMind. 2025. Gemini 3 flash. https://deepmind.google/models/gemini/flash/. Model description page. ↗
  18. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2024. CRITIC: large language models can self-correct with tool-interactive critiquing. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. Open-Revie…
  19. Melody Y Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Helyar, Rachel Dias, Andrea Vallone, Hongyu Ren, Jason Wei, and 1 others. 2024. Deliberative alignment: Reasoning enables safer language models. ArXiv preprint, abs/2412.16339.
  20. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. 2024. OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems. In Proce…
  21. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. ArXiv preprint, abs/2103.03874.
  22. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. ArXiv preprint, abs/2503.09516.
  23. Sahil Kale and Devendra Singh Dhami. 2025. Knowrl: Teaching language models to know what they know. ArXiv preprint, abs/2510.11407. — [arXiv:2510.11407](https://arxiv.org/abs/2510.11407) find
  24. Sanyam Kapoor, Nate Gruver, Manley Roberts, Katie Collins, Arka Pal, Umang Bhatt, Adrian Weller, Samuel Dooley, Micah Goldblum, and Andrew Gordon Wilson. 2024. Large language models must be taught to know what they don’t know. In Advances in Neural Information Processing Systems 38: Annual Conferenc… find
  25. Polina Kirichenko, Mark Ibrahim, Kamalika Chaudhuri, and Samuel J Bell. 2025. Abstentionbench: Reasoning llms fail on unanswerable questions. ArXiv preprint, abs/2506.09038. find
  26. Yunghwei Lai, Kaiming Liu, Ziyue Wang, Weizhi Ma, and Yang Liu. 2025. Doctor-r1: Mastering clinical inquiry with experiential agentic reinforcement learning. ArXiv preprint, abs/2510.04284. — [arXiv:2510.04284](https://arxiv.org/abs/2510.04284) find
  27. Dongryeol Lee, Segwang Kim, Minwoo Lee, Hwanhee Lee, Joonsuk Park, Sang-Woo Lee, and Kyomin Jung. 2023. Asking clarification questions to handle ambiguity in open-domain QA. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11526–11544, Singapore. Association for Comput… find
  28. Weitao Li, Boran Xiang, Xiaolong Wang, Zhinan Gou, Weizhi Ma, and Yang Liu. 2025a. Ur 2 : Unify rag and reasoning through reinforcement learning. ArXiv preprint, abs/2508.06165. find
  29. Wenjun Li, Dexun Li, Kuicai Dong, Cong Zhang, Hao Zhang, Weiwen Liu, Yasheng Wang, Ruiming Tang, and Yong Liu. 2025b. Adaptive tool use in large language models with meta-cognition trigger. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap…
  30. Xiaoou Liu, Tiejin Chen, Longchao Da, Chacha Chen, Zhen Lin, and Hua Wei. 2025. Uncertainty quantification and confidence calibration in large language models: A survey. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 6107–6117.
  31. Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. 2024. GAIA: a benchmark for general AI assistants. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net.
  32. OpenAI. 2024. Gpt-4o system card. https://openai.com/index/gpt-4o-system-card. ↗
  33. OpenAI. 2025a. Gpt-5 system card. https://openai.com/index/gpt-5-system-card. ↗
  34. OpenAI. 2025b. Introducing gpt-oss: Open weights for advanced reasoning. https://openai.com/index/ introducing-gpt-oss/. ↗
  35. Baochang Ren, Shuofei Qiao, Da Zheng, Huajun Chen, and Ningyu Zhang. 2025a. Knowrl: Exploring knowledgeable reinforcement learning for factuality. ArXiv preprint, abs/2506.19807.
  36. Jingyi Ren, Yekun Xu, Xiaolong Wang, Weitao Li, Weizhi Ma, and Yang Liu. 2025b. Transparent and robust rag: Adaptive-reward reinforcement learning for decision traceability. ArXiv preprint, abs/2505.13258. — [arXiv:2505.13258](https://arxiv.org/abs/2505.13258) find
  37. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. ArXiv preprint, abs/2402.03300.
  38. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297.
  39. Aviv Slobodkin, Omer Goldman, Avi Caciularu, Ido Dagan, and Shauli Ravfogel. 2023. The curious case of hallucinatory (un)answerability: Finding truths in the hidden states of over-confident large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processi… find
  40. Ionut Teodor Sorodoc, Leonardo F. R. Ribeiro, Rexhina Blloshmi, Christopher Davis, and Adrià de Gispert. 2025. GaRAGe: A benchmark with grounding annotations for RAG evaluation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 17030–17049, Vienna, Austria. Association fo… find
  41. YuHong Sun, Zhangyue Yin, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Hui Zhao. 2024. Benchmarking hallucination in large language models based on unanswerable math word problem. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LRE… find
  42. Zhi Rui Tam, Cheng-Kuang Wu, Chieh-Yen Lin, and Yun-Nung Chen. 2025. None of the above, less of the right parallel patterns in human and LLM performance on multi-choice questions answering. In Findings of the Association for Computational Linguistics: ACL 2025, pages 20112–20134, Vienna, Austria. As… find
  43. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. MuSiQue: Multi-hop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539–554. find
  44. Roman Vashurin, Ekaterina Fadeeva, Artem Vazhentsev, Lyudmila Rvanova, Daniil Vasilev, Akim Tsvigun, Sergey Petrakov, Rui Xing, Abdelrahman Sadallah, Kirill Grishchenkov, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, Maxim Panov, and Artem Shelmanov. 2025. Benchmarking uncertainty quantificat…
  45. Ante Wang, Weizhi Ma, and Yang Liu. 2025. Let the model distribute its doubt: Confidence estimation through verbalized probability distribution. ArXiv preprint, abs/2511.14275. — [arXiv:2511.14275](https://arxiv.org/abs/2511.14275) find
  46. Tianyang Xu, Shujin Wu, Shizhe Diao, Xiaoze Liu, Xingyao Wang, Yangyi Chen, and Jing Gao. 2024. SaySelf: Teaching LLMs to express confidence with self-reflective rationales. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5985–5998, Miami, Florida, US… find
  47. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. ArXiv preprint, abs/2505.09388.
  48. Fanghua Ye, Mingming Yang, Jianhui Pang, Longyue Wang, Derek F. Wong, Emine Yilmaz, Shuming Shi, and Zhaopeng Tu. 2024. Benchmarking llms via uncertainty quantification. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS… find
  49. Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. 2023. Do large language models know what they don’t know? In Findings of the Association for Computational Linguistics: ACL 2023, pages 8653–8665, Toronto, Canada. Association for Computational Linguistics. find
  50. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, and 1 others. 2025. Dapo: An open-source llm reinforcement learning system at scale. ArXiv preprint, abs/2503.14476.
  51. Tong Zhang, Peixin Qin, Yang Deng, Chen Huang, Wenqiang Lei, Junhong Liu, Dingnan Jin, Hongru Liang, and Tat-Seng Chua. 2024. CLAMBER: A benchmark of identifying and clarifying ambiguous information needs in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Comp…
  52. Zhaohan Zhang, Ziquan Liu, and Ioannis Patras. 2025. Grace: A generative approach to better confidence elicitation in large language models. ArXiv preprint, abs/2509.09438. — [arXiv:2509.09438](https://arxiv.org/abs/2509.09438) find
  53. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. In Advances in Neural Information Processing Systems 36: A…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ren-2026-beyond-i-don-t-know/