Elamur Research⌕
☰
home / papers / 2026 / LLM Beliefs Are in Their Heads

LLM Beliefs Are in Their Heads

Paper · 2026 · Alessandro Corona Mendozza and Anders Søgaard

Summary

The paper asks whether decoder-only LLMs have internal states that could count as beliefs. It tests them against four standards from Herrmann and Levinstein (2025): Accuracy, Use, Coherence and Uniformity. The authors study six model families: GPT-2, Llama-3.1-8B, Gemma-2-9B, Yi-6B, Pythia-6.9B and GPT-J-6B, using base and instruction-tuned versions where available. They inspect every layer at two levels, the residual stream and individual attention heads. Linear probes (logistic regression and mass-mean, checked against control tasks) are trained on the True/False dataset of Marks and Tegmark (2024). For coherence, the dataset is extended with negations, conjunctions and disjunctions. The methods include:

  • isotonic calibration of the probes;
  • steering activations along probe directions;
  • leave-one-domain-out evaluation.

Larger and instruction-tuned models separate true from false statements well, and steering reliably changes their outputs. Coherence is moderate, and training on diverse data gives truth directions that hold across domains. Attention heads turn out to be a particularly good place to find truth signals. The authors argue this supports belief attribution under minimal functionalism, though not under richer realist accounts. They also see practical value for safety, hallucination diagnosis and uncertainty estimation.

Key points

  • Probe accuracy (top-3 layers, residual stream): GPT-2-Large 0.567, Yi 0.630, Pythia 0.721, Llama 0.889, Gemma 0.920. At the head level, average accuracy reaches 0.931 on Gemma Instruct, while control probes stay at chance.

  • Where truth appears: truth-sensitivity emerges in the middle layers (roughly 10–20). In base models, truth-sensitive heads fade in later layers. This pattern is absent in IFT models, which suggests finer truth tracking deeper in the network.

  • Restricted experiments: Use, Coherence and Uniformity were tested only on models with residual accuracy above 80% (the Llama and Gemma families, plus GPT-J-6B), using the best layer or head.

  • Coherence: these middling coherence results run counter to prior studies, Bürger et al. (2024) and Levinstein and Herrmann (2025), which concluded that coherence breaks down.

  • Limitations:

    • models only up to about 9B parameters;
    • linear probes only;
    • a deliberately simple dataset whose only non-English content is Spanish.

    Code is available at github.com/Supersheep7/beliefs_llms.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 8The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSafety Research · 543 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Alessandro Corona Mendozza and Anders Søgaard. 2026. LLM Beliefs Are in Their Heads. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1905

@inproceedings{mendozza2026llm,
  title = {{LLM Beliefs Are in Their Heads}},
  author = {Alessandro Corona Mendozza and Anders Søgaard},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1905},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1905}
}

References

70 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 0.1AI. 2025. Yi: Open foundation models by 01.ai.
  2. Mostafa Abdou, Artur Kulmizev, Daniel Hershcovich, Stella Frank, Ellie Pavlick, and Anders Søgaard. 2021. Can language models encode perceptual structure without grounding? a case study in color. In Proceedings of the 25th Conference on Computational Natural Language Learning, page 109–132, Online.…
  3. Ekin Akyurek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, and Kelvin Guu. 2022. Towards tracing knowledge in language models back to the training data. In Findings of the Association for Computational Linguistics: EMNLP 2022, page 2429–2446, Abu Dhabi, United Arab Emirat… find
  4. Guillaume Alain and Yoshua Bengio. 2018. Understanding intermediate layers using linear classifier probes. find
  5. David M. Armstrong. 1968. A Materialist Theory of the Mind. Routledge Kegan Paul. find
  6. Amos Azaria and Tom Mitchell. 2023. The internal state of an llm knows when it‘s lying. In Findings of the Association for Computational Linguistics: EMNLP 2023, page 967–976, Singapore. Association for Computational Linguistics.
  7. Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207–219.
  8. Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar Van Der Wal. 2023. Pythia: a suite for analyzing large language models across trainin… find
  9. Ned Block. 1978. Troubles with functionalism. Minnesota Studies in the Philosophy of Science, 9:261–325. find
  10. David Bourget and David J. Chalmers. 2023. Philosophers on philosophy: The 2020 philpapers survey. Philosophers’ Imprint, 23(11). find
  11. Luc Bovens and Stephan Hartmann. 2003. Bayesian Epistemology. Oxford University Press, Oxford. find
  12. Michael Bratman. 1987. Intention, Plans, and Practical Reason. Harvard University Press. find
  13. Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. 2023. Discovering latent knowledge in language models without supervision. In International Conference on Learning Representations (ICLR). find
  14. Lennart Bürger, Fred A Hamprecht, and Boaz Nadler. 2024. Truth is universal: Robust detection of lies in llms. In Advances in Neural Information Processing Systems, volume 37, page 138393–138431. Curran Associates, Inc. find
  15. Herman Cappelen and Josh Dever. forthcoming-a. A Hyper-Externalist Manifesto for LLMs. Oxford University Press. Herman Cappelen and Josh Dever. forthcoming-b. Going Whole Hog: A Philosophical Defense of AI Cognition. Oxford University Press. find
  16. David J. Chalmers. 2025. Propositional interpretability in artificial intelligence. find
  17. Paul Christiano, Ajeya Cotra, and Mark Xu. 2021. Eliciting latent knowledge: How to tell if your eyes deceive you (report). Technical report, Alignment Research Center. find
  18. Paul M. Churchland. 1981. Eliminative materialism and the propositional attitudes. Journal of Philosophy, 78(2):67–90. find
  19. Alessandro Corona Mendozza. 2025. Don’t believe the belief hype! In Mechanistic Interpretability Workshop at NeurIPS 2025. find
  20. Donald Davidson. 1973. Radical interpretation. Dialectica, 27(1):313–328. find
  21. Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, page 6491–6506, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  22. Daniel C. Dennett. 1971. Intentional systems. Journal of Philosophy, 68(February):87–106. find
  23. Daniel C. Dennett. 1981. The Intentional Stance. MIT Press. find
  24. Ophelia Deroy. 2023. The ethics of terminology: Can we use human terms to describe ai? Topoi, 42(3):881–889. find
  25. Jerry Fodor. 1975. The Language of Thought. Harvard University Press. find
  26. Simon Goldstein and Benjamin A. Levinstein. 2024. Does chatgpt have a mind? find
  27. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — The Llama 3 Herd of Models
  28. Wes Gurnee and Max Tegmark. 2024. Language models represent space and time. In International Conference on Representation Learning, volume 2024, page 2483–2503. find
  29. Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, and Srinivasan Iyer. 2023. Methods for measuring, updating, and visualizing factual beliefs in language models. In Proceedings of the 17th Conference of the European Chapter of the Association for Co…
  30. Peter Hase, Thomas Hofweber, Xiang Zhou, Elias Stengel-Eskin, and Mohit Bansal. 2024. Fundamental problems with model editing: How should rational belief revision work in llms? Transactions on Machine Learning Research. find
  31. Daniel A. Herrmann and Benjamin A. Levinstein. 2025. Standards for belief representations in llms. Minds and Machines, 35(1):5. find
  32. John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), page 2733–2743, Hong Kong, Chin… find
  33. Hanti Lin. 2024. Bayesian Epistemology, summer 2024 edition. Metaphysics Research Lab, Stanford University. find
  34. Daniel Hutto and Ian Ravenscroft. 2021. Folk Psychology as a Theory, fall 2021 edition. Metaphysics Research Lab, Stanford University. find
  35. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page 3214–3252, Dublin, Ireland. Association for Computational Linguistics.
  36. Richard C. Jeffrey. 1965. The Logic of Decision. University of Chicago Press, New York, NY, USA. find
  37. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page 1601–1611, Vancouver,…
  38. James M. Joyce. 1998. A nonpragmatic vindication of probabilism. Philosophy of Science, 65(4):575–603. find
  39. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, and 17 others. 2022.… find
  40. Lea Krause, Wondimagegnhue Tufa, Selene Baez Santamaria, Angel Daza, Urja Khurana, and Piek Vossen. 2023. Confidently wrong: Exploring the calibration and expression of (un)certainty of large language models in a multilingual setting. In Proceedings of the Workshop on Multimodal, Multilingual Natura… find
  41. Benjamin A. Levinstein and Daniel A. Herrmann. 2025. Still no lie detector for language models: probing empirical and conceptual roadblocks. Philosophical Studies, 182:1539–1565. find
  42. David K. Lewis. 1972. Psychophysical and theoretical identifications. Australasian Journal of Philosophy, 50(3):249–258. find
  43. Kenneth Li, Aspen K. Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2023a. Emergent world representations: Exploring a sequence model trained on a synthetic task. In The Eleventh International Conference on Learning Representations. find
  44. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2023b. Inference-time intervention: eliciting truthful answers from a language model. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Curran As…
  45. Chen Ling, Xujiang Zhao, Xuchao Zhang, Wei Cheng, Yanchi Liu, Yiyou Sun, Mika Oishi, Takao Osaki, Katsushi Matsuda, Jie Ji, Guangji Bai, Liang Zhao, and Haifeng Chen. 2024. Uncertainty quantification for in-context learning of large language models. In Proceedings of the 2024 Conference of the North… find
  46. Samuel Marks and Max Tegmark. 2024. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets.
  47. Ruth G. Millikan. 1984. Language, Thought, and Other Biological Categories: New Foundations for Realism. MIT Press. find
  48. Neel Nanda, Andrew Lee, and Martin Wattenberg. 2023. Emergent linear representations in world models of self-supervised sequence models. In Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, page 16–30, Singapore. Association for Computational Linguistic… find
  49. Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. 2020. Zoom in: An introduction to circuits. Distill. find
  50. OpenAI. 2025. Introducing gpt-4.1 in the api. find
  51. Fabio Petroni, Patrick Lewis, Aleksandra Piktus, Tim Rocktäschel, Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel. 2020. How context affects language models’ factual predictions.
  52. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural…
  53. Nina Poth and Annika Schuster. 2026. Mental, scientific, and artificial representations. Philosophy and the Mind Sciences, 7(1). find
  54. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI. find
  55. Frank Ramsey. 1931. Truth and Probability, page 156–98. Routledge Kegan Paul. find
  56. Fabien Roger. 2023. What discovering latent knowledge did and did not find. find
  57. Eric Schwitzgebel. 2002. A phenomenal, dispositional account of belief. Noûs, 36(2):249–275. find
  58. Eric Schwitzgebel. 2024. Belief , spring 2024 edition. Metaphysics Research Lab, Stanford University. find
  59. John R. Searle. 1990. Is the brain a digital computer? Proceedings and Addresses of the American Philosophical Association, 64(3):21–37. American Philosophical Association. find
  60. Ola Shorinwa, Zhiting Mei, Justin Lidard, Allen Z. Ren, and Anirudha Majumdar. 2025. A survey on uncertainty quantification of large language models: Taxonomy, open research challenges, and future directions. ACM Computing Surveys, 58(3). find
  61. Stephen P. Stich. 1983. From folk psychology to cognitive science: The case against belief. MIT Press, Cambridge, MA, US. find
  62. Zhisheng Tang, Ke Shen, and Mayank Kejriwal. 2024. An evaluation of estimative uncertainty in large language models. Preprint, arXiv:2405.15185. — [arXiv:2405.15185](https://arxiv.org/abs/2405.15185) find
  63. Gemma Team. 2024. Gemma 2: Improving open language models at a practical size. find
  64. Adam Toon. 2023. Mind as Metaphor: A Defence of Mental Fictionalism, 1 edition. Oxford University PressOxford. find
  65. Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. 2024. Steering language models with activation engineering.
  66. Ben Wang and Aran Komatsuzaki. 2021. Gpt-j-6b: A 6 billion parameter autoregressive language model. find
  67. Jonathan Weisberg. 2021. Formal Epistemology, spring 2021 edition. Metaphysics Research Lab, Stanford University. find
  68. Bianca Zadrozny and Charles Elkan. 2002. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’02, page 694–699, New York, NY, USA. Association for Computing Machine… find
  69. Aaron Z. Zimmerman. 2018. Belief: A Pragmatic Picture. Oxford University Press, Oxford. find
  70. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, and 2 others. 2025. Representa…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mendozza-2026-llm-beliefs-are-their-heads/