Elamur Research⌕
☰
home / papers / 2026 / The Digital Dunning-Kruger Effect: Decoupling Hallucinations via Geometric Hidden-state Observation for Semantic Truthfulness

The Digital Dunning-Kruger Effect: Decoupling Hallucinations via Geometric Hidden-state Observation for Semantic Truthfulness

Paper · 2026 · Yueheng Mao, Min Yu, Gengwang Li et al.

Summary

The paper tackles hallucination detection in LLMs. Black-box detectors that rely on repeated sampling or external verification are accurate but slow. Cheap white-box detectors miss errors that the model states with high confidence. The authors compare this to the Dunning-Kruger effect and split hallucinations into two kinds. "Confused" hallucinations show unstable hidden-state trajectories across layers. "Stubborn" hallucinations settle on a wrong answer too early in the layer stack.

Their framework, GHOST, works within the model's single forward pass and needs no changes to the LLM. For each generated token it computes two geometric signals, then averages them over the response:

  • inter-layer turbulence, the cosine change between adjacent layers;
  • stubbornness, how similar intermediate layers already are to the final layer.

These are combined with predictive entropy and semantic divergence of the output distribution. A non-linear classifier then labels the answer as truthful or not.

On FinanceBench, RAGTruth, HaluEval and PopQA, GHOST beats white-box baselines and comes close to black-box methods at a small fraction of their cost. This makes real-time hallucination monitoring practical when model internals are accessible.

Key points

  • The feature vector has four parts: V_turb, V_stub, V_ent and V_div. They cover turbulence, stubbornness, predictive entropy and semantic divergence.
  • Hidden-state analysis suggests that confused hallucinations start drifting from truthful trajectories at about 50% of network depth. Stubborn ones remain close to truthful paths.
  • The tested base models are Qwen2.5-1.5B, DeepSeek-R1-7B, Gemma-3-4B and Mistral-7B-Instruct, run on four RTX 3090 GPUs.
  • GHOST adds only 0.08–0.18 s per response, about 4.3–4.6% overhead. SelfCheckGPT with N = 5 samples adds roughly 400%, and the paper reports an overall compute reduction above 90%.
  • The authors list several limitations:
    • only decoder-only models were evaluated;
    • the method needs access to hidden states, so API-only models are excluded;
    • stable-but-wrong outputs remain harder to catch;
    • no dataset yet targets premature-convergence hallucinations.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️Chinese Academy of SciencesBeijing · China194 works · 23,906 citations🎓University of Chinese Academy of SciencesBeijing · China147 works · 8,480 citations🔬Institute of Information EngineeringBeijing · China30 works · 1,411 citations🎓Deakin UniversityBurwood · Australia4 works · 977 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

Cite

Cite as Yueheng Mao, Min Yu, Gengwang Li, Jianguo Jiang, Gang Li, Meng Zhang, Zhen Xu, Weiqing Huang and Ming Liu. 2026. The Digital Dunning-Kruger Effect: Decoupling Hallucinations via Geometric Hidden-state Observation for Semantic Truthfulness. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.993

@inproceedings{mao2026digital,
  title = {{The Digital Dunning-Kruger Effect: Decoupling Hallucinations via Geometric Hidden-state Observation for Semantic Truthfulness}},
  author = {Yueheng Mao, Min Yu, Gengwang Li, Jianguo Jiang, Gang Li, Meng Zhang, Zhen Xu, Weiqing Huang and Ming Liu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.993},
  url = {https://doi.org/10.18653/v1/2026.acl-long.993}
}

References

38 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mistral AI. 2024. Mistral 7b v0.3 model card. https://huggingface.co/mistralai/ Mistral-7B-Instruct-v0.3. Accessed: 2025-12-25. ↗
  2. Amos Azaria and Tom Mitchell. 2023. The internal state of an LLM knows when it’s lying. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 967–976, Singapore. Association for Computational Linguistics.
  3. Yuzhuo Bai, Shuzheng Si, Kangyang Luo, Qingyi Wang, Wenhao Li, Gang Chen, Fanchao Qi, and Maosong Sun. 2026. Infi-check: Interpretable and fine-grained fact-checking of llms. Preprint, arXiv:2601.06666. — [arXiv:2601.06666](https://arxiv.org/abs/2601.06666) find
  4. Alexandra Bazarova, Aleksandr Yugay, Andrey Shulga, Alina Ermilova, Andrei Volodichev, Konstantin Polev, Julia Belikova, Rauf Parchiev, Dmitry Simakov, Maxim Savchenko, Andrey Savchenko, Serguei Barannikov, and Alexey Zaytsev. 2025. Hallucination detection in llms with topological divergence on atte… find
  5. Jakub Binkowski, Denis Janiak, Albert Sawczyn, Bogdan Gabrys, and Tomasz Jan Kajdanowicz. 2025. Hallucination detection in LLMs using spectral features of attention maps. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 24354–24385, Suzhou, China. Asso…
  6. Anwoy Chatterjee, Yash Goel, and Tanmoy Chakraborty. 2025. Hide and seek: Detecting hallucinations in language models via decoupled representations. Preprint, arXiv:2506.17748. — [arXiv:2506.17748](https://arxiv.org/abs/2506.17748) find
  7. Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. 2024. INSIDE: LLMs’ internal states retain the power of hallucination detection. In The Twelfth International Conference on Learning Representations. find
  8. Yung-Sung Chuang, Linlu Qiu, Cheng-Yu Hsieh, Ranjay Krishna, Yoon Kim, and James R. Glass. 2024. Lookback lens: Detecting and mitigating contextual hallucinations in large language models using only attention maps. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Proces…
  9. Roi Cohen, May Hamri, Mor Geva, and Amir Globerson. 2023. LM vs LM: Detecting factual errors via cross examination. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12621–12640, Singapore. Association for Computational Linguistics.
  10. Ekaterina Fadeeva, Roman Vashurin, Akim Tsvigun, Artem Vazhentsev, Sergey Petrakov, Kirill Fedyanin, Daniil Vasilev, Elizaveta Goncharova, Alexander Panchenko, Maxim Panov, Timothy Baldwin, and Artem Shelmanov. 2023. LM-polygraph: Uncertainty estimation for language models. In Proceedings of the 202… find
  11. Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, Yarin Chen, and Yarin Gal. 2024. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017):625–630. find
  12. Leon Festinger. 1957. A theory of cognitive dissonance. Stanford university press. find
  13. Gemma Team and Google DeepMind. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. find
  14. Aman Goel, Daniel Schwartz, and Yanjun Qi. 2025. Zero-knowledge LLM hallucination detection and mitigation through fine-grained cross-model consistency. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1982–1999, Suzhou (China). Associa… find
  15. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruipu Xu, Qi Zhu, Shirong Ma, Pei Wang, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  16. Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen. 2023. Financebench: A new benchmark for financial question answering. Preprint, arXiv:2311.11944. — [arXiv:2311.11944](https://arxiv.org/abs/2311.11944) find
  17. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Comput. Surv., 55(12).
  18. Hazel Kim, Tom A. Lamb, Adel Bibi, Philip Torr, and Yarin Gal. 2025. Detecting LLM hallucination through layer-wise information deficiency: Analysis of ambiguous prompts and unanswerable questions. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 32298… find
  19. Justin Kruger and David Dunning. 1999. Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6):1121–1134. find
  20. Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. HaluEval: A large-scale hallucination evaluation benchmark for large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6449–6464, Singapore. Association for Computa…
  21. Junteng Liu, Shiqi Chen, Yu Cheng, and Junxian He. 2024. On the universal truthfulness hyperplane inside LLMs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18199–18224, Miami, Florida, USA. Association for Computational Linguistics. find
  22. Andrey Malinin and Mark Gales. 2021. Uncertainty estimation in autoregressive structured prediction. In Proceedings of the International Conference on Learning Representations (ICLR). find
  23. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V… find
  24. Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9004–9017, Singapore. Association for Computation… find
  25. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural L…
  26. Oscar Obeso, Andy Arditi, Javier Ferrando, Joshua Freeman, Cameron Holmes, and Neel Nanda. 2026. Real-time detection of hallucinated entities in long-form generation. Preprint, arXiv:2509.03531. — [arXiv:2509.03531](https://arxiv.org/abs/2509.03531) find
  27. OpenAI. 2025. gpt-oss-120b & gpt-oss-20b model card. Preprint, arXiv:2508.10925.
  28. Shuzheng Si, Haozhe Zhao, Gang Chen, Cheng Gao, Yuzhuo Bai, Zhitong Wang, Kaikai An, Kangyang Luo, Chen Qian, Fanchao Qi, Baobao Chang, and Maosong Sun. 2025. Aligning large language models to follow instructions and hallucinate less via effective data filtering. In Proceedings of the 63rd Annual Me…
  29. Shuzheng Si, Haozhe Zhao, Cheng Gao, Yuzhuo Bai, Zhitong Wang, Bofei Gao, Kangyang Luo, Wenhao Li, Yufei Huang, Gang Chen, Fanchao Qi, Minjia Zhang, Baobao Chang, and Maosong Sun. 2026. Teaching large language models to maintain contextual faithfulness via synthetic tasks and reinforcement learning.… find
  30. The statistical distribution and partitioning of these datasets are summarized in Table 6. • HaluEval (General): This large-scale benchmark provides 10,000 samples covering general knowledge. It is instrumental for training our baseline classifiers and assessing the model’s fundamental factuality al… find
  31. Jialiang Wu, Yi Shen, Sijia Liu, Yi Tang, Sen Song, Xiaoyi Wang, and Longjun Cai. 2025. Improve decoding factuality by token-wise cross layer entropy of large language models. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 3912–3921, Albuquerque, New Mexico. Associat… find
  32. PopQA (Long-tail Knowledge): Focused on low-popularity entities, PopQA challenges the model’s ability to distinguish between internal knowledge and parasitic memory. This dataset is crucial for validating the "Digital Dunning-Kruger Effect," as models often exhibit high confidence in these long-tail… find
  33. Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum, Cheng Niu, Randy Zhong, Juntong Song, and Tong Zhang. 2023. Ragtruth: A hallucination corpus for developing trustworthy retrieval-augmented language models. Preprint, arXiv:2401.00396.
  34. Yilun Xu, Shengjia Zhao, Jiaming Song, Russell Stewart, and Stefano Ermon. 2020. A theory of usable information under computational constraints. In International Conference on Learning Representations. find
  35. FinanceBench (Domain-Specific): Representing high-stakes professional reasoning, FinanceBench requires precise numerical and conceptual accuracy. It serves to test GHOST’s efficacy in detecting "Stubborn Hallucinations" where models generate plausible but incorrect financial deductions. An Yang, Bao… find
  36. RAGTruth (Retrieval-Augmented): This dataset evaluates hallucinations in the context of external documents. It allows us to analyze how geometric trajectories shift when a model is constrained by provided context versus relying solely on internal parameters, providing insights into "Confused Halluci… — Prompt-Guided Internal States for Hallucination Detection of Large Language Models
  37. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. 2025b. Siren’s song in the ai ocean: A survey on hallucination in large language models. Computational Linguistics, pages…
  38. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, and 3 others. 2025. A survey of large language models. Prepri…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mao-2026-digital-dunning-kruger-effect-decoupling/