Elamur Research⌕
☰
home / papers / 2026 / Toward Cognitive Alignment in Large Language Models: Integrating Linguistic Theory and Human Data

Toward Cognitive Alignment in Large Language Models: Integrating Linguistic Theory and Human Data

Summary

This position paper (CMCL 2026 at LREC) addresses a conceptual gap. LLMs predict human reading times and brain responses well, but that fit alone does not show they process language as people do. The author releases no new dataset or model. Instead, he proposes a framework of "cognitive alignment," adapted from Marr's levels of analysis, which requires models to match humans on three fronts at once:

  • Behavioral: fit to surprisal, eye-tracking and EEG/fMRI data, plus controlled tests such as agreement and garden-path sentences.
  • Representational: whether internal states correspond to formal accounts like Construction Grammar, AMR or FrameNet, and whether the model causally relies on them.
  • Developmental: whether models can learn from child-scale input.

A review of recent evidence shows that larger or instruction-tuned models are not automatically more human-like, and that probing results need causal backing. The paper closes with a four-part research agenda aimed at systems that are easier to interpret, more data-efficient and better grounded in cognition. It casts the cognitive modeling community as the natural leader of this work.

Key points

  • Four initiatives are put forward: interdisciplinary training programs, infrastructure for simulating language at the population level, modeling challenges that combine neural and symbolic approaches, and a Cognitive Alignment Benchmark Suite.
  • Scaling evidence is mixed:
    • Instruction-tuned LLMs predict reading behavior worse than base models (Kuribayashi et al., 2024).
    • Larger models' surprisal fits reading times worse, the "inverse scaling" effect (Oh et al., 2025).
    • Internal layers of large LMs match or beat smaller models on self-paced reading, gaze, maze and N400 data (Kuribayashi et al., 2025).
  • An RSA study of 23 LLMs (Ren et al., 2025) found that model-brain similarity rises with pre-training data size and scale, and that alignment training improves it.
  • On the developmental gap, children hear roughly 10–15 million words by age 10 (CHILDES), far less than LLM corpora. BLiMP and the BabyLM Challenge are cited as tools for studying this gap.
  • The paper builds on earlier CMCL eye-tracking shared tasks (Hollenstein et al., 2021, 2022) and the MECO multilingual eye-movement corpus.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5d9kn8b26y4q

@inproceedings{zaghouani2026toward,
  title = {{Toward Cognitive Alignment in Large Language Models: Integrating Linguistic Theory and Human Data}},
  author = {Zaghouani, Wajdi},
  booktitle = {Proceedings of the 15th Workshop on Cognitive Modeling and Computational Linguistics},
  year = {2026},
  doi = {10.63317/5d9kn8b26y4q},
  url = {https://doi.org/10.63317/5d9kn8b26y4q}
}

References

96 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004). An integrated theory of the mind. Psychological Review, 111(4), 1036–1060. find
  2. Asher, N., & Lascarides, A. (2003). Logics of Conversation. Cambridge University Press. find
  3. Baker, C. F., Fillmore, C. J., & Lowe, J. B. (1998). The Berkeley FrameNet Project. In Proceedings of COLING-ACL, 86–90. find our card
  4. Banarescu, L., Bonial, C., Cai, S., Georgescu, M., Griffitt, K., Hermjakob, U., Knight, K., Koehn, P., Palmer, M., & Schneider, N. (2013). Abstract meaning representation for sembanking. In Proceedings of the 7th Linguistic Annotation Workshop, 178–186. find our card
  5. Baroni, M. (2022). On the proper role of linguistically oriented deep net analysis in linguistic theorising. In Algebraic Structures in Natural Language, 1–28. CRC Press. find
  6. Bender, E. M., & Koller, A. (2020). Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of ACL, 5185– 5198. find our card
  7. Bender, E. M. (2011). On achieving and evaluating language-independence in NLP. Linguistic Issues in Language Technology, 6(3), 1–26. find
  8. Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009). Curriculum learning. In Proceedings of ICML, 41–48. find our card
  9. Bird, S. (2020). Decolonising speech and language technology. In Proceedings of COLING, 3504– 3519. find our card
  10. Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020). Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of ACL, 5454–5476. find our card
  11. Boyce, V., & Levy, R. (2023). A-maze of Natural Stories: Comprehension and surprisal in the Maze task. Glossa Psycholinguistics, 2(1). find
  12. Caliskan, A., Bryson, J. J., & Narayanan, A. (2017). Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334), 183–186. find
  13. Caucheteux, C., & King, J. R. (2022). Brains and algorithms partially converge in natural language processing. Communications Biology, 5(1), 134. find
  14. Clark, K., Khandelwal, U., Levy, O., & Manning, C. D. (2019). What does BERT look at? An analysis of BERT’s attention. In Proceedings of the BlackboxNLP Workshop, 276–286. find
  15. Cop, U., Dirix, N., Drieghe, D., & Duyck, W. (2017). Presenting GECO: An eyetracking corpus of monolingual and bilingual sentence reading. Behavior Research Methods, 49(2), 602–615. find
  16. Doerig, A., Kietzmann, T. C., Allen, E., Wu, Y., Naselaris, T., Kay, K., & Charest, I. (2025). High-level visual representations in the human brain are aligned with large language models. Nature Machine Intelligence, 7(8), 1220–1234. find
  17. Futrell, R., Wilcox, E., Mober, T., Qian, P., Ballesteros, M., Dyer, C., Gibson, E., & Levy, R. (2019). Neural language models as psycholinguistic subjects: Representations of syntactic state. In Proceedings of NAACL-HLT, 32–42. find our card
  18. Futrell, R., Gibson, E., Tily, H. J., Blank, I., Vishnevetsky, A., Piantadosi, S. T., & Fedorenko, E. (2021). The Natural Stories corpus: A reading-time corpus of English texts containing rare syntactic constructions. Language Resources and Evaluation, 55(1), 63–77. find
  19. Garcez, A. d’A., & Lamb, L. C. (2020). Neurosymbolic AI: The 3rd wave. arXiv preprint arXiv:2012.05876. find
  20. Geiger, A., Lu, H., Icard, T., & Potts, C. (2021). Causal abstractions of neural networks. In Advances in Neural Information Processing Systems, 34, 9574–9586. find
  21. Gibson, E. (1998). Linguistic complexity: Locality of syntactic dependencies. Cognition, 68(1), 1–76. find
  22. Goldberg, A. E. (1995). Constructions: A Construction Grammar Approach to Argument Structure. University of Chicago Press. find
  23. Dryer, M. S., & Haspelmath, M. (Eds.). (2013). WALS Online. Max Planck Institute for Evolutionary Anthropology. find
  24. Goldberg, A. E. (2006). Constructions at Work: The Nature of Generalization in Language. Oxford University Press. find
  25. Elazar, Y., Ravfogel, S., Jacovi, A., & Goldberg, Y. (2021). Amnesic probing: Behavioral explanation with amnesic counterfactuals. Transactions of the ACL, 9, 160–175. find our card
  26. Goldstein, A., et al. (2022). Shared computational principles for language processing in humans and deep language models. Nature Neuroscience, 25(3), 369–380. find
  27. Fedorenko, E., Scott, T. L., Brunner, P., Coon, W. G., Pritchett, B., Schalk, G., & Kanwisher, N. (2016). Neural correlate of the construction of sentence meaning. Proceedings of the National Academy of Sciences, 113(41), E6256–E6262. find
  28. Goodhart, C. A. (1984). Problems of monetary management: The UK experience. In Monetary Theory and Practice, 91–121. Springer. find
  29. Finn, C., Abbeel, P., & Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of ICML, 1126– 1135. find
  30. Fodor, J. A., & Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1-2), 3–71. find
  31. Frank, S. L., & Bod, R. (2011). Insensitivity of the human sentence-processing system to hierarchical structure. Psychological Science, 22(6), 829–834. find
  32. Frank, S. L., Otten, L. J., Galli, G., & Vigliocco, G. (2015). The ERP response to the amount of information conveyed by words in sentences. Brain and Language, 140, 1–11. find
  33. Gottlieb, J., Oudeyer, P. Y., Lopes, M., & Baranes, A. (2013). Information-seeking, curiosity, and attention: Computational and neural mechanisms. Trends in Cognitive Sciences, 17(11), 585–593. find
  34. Grice, H. P. (1975). Logic and conversation. In Syntax and Semantics 3: Speech Acts, 41–58. Academic Press. find
  35. Gries, S. T. (2006). Corpus-based methods and cognitive semantics: The many senses of to run. In Corpora in Cognitive Linguistics, 57–99. Mouton de Gruyter. find
  36. Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic model. In Proceedings of NAACL, 1–8. find our card
  37. Harris, Z. S. (1954). Distributional structure. Word, 10(2-3), 146–162. find
  38. Hempel, C. G. (1965). Aspects of Scientific Explanation. Free Press. find
  39. Hewitt, J., & Manning, C. D. (2019). A structural probe for finding syntax in word representations. In Proceedings of NAACL-HLT, 4129–4138. find our card
  40. Hollenstein, N., Chersoni, E., Jacobs, C. L., Oseki, Y., Prévot, L., & Santus, E. (2021). find
  41. CMCL 2021 shared task on eye-tracking prediction. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, 72–78. find
  42. Hollenstein, N., Chersoni, E., Jacobs, C. L., Oseki, Y., Prévot, L., & Santus, E. (2022). find
  43. CMCL 2022 shared task on multilingual and crosslingual prediction of human reading behavior. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics. find
  44. Jakobi, D. N., Stegenwallner-Schütz, M., Hollenstein, N., et al. (2025). MultiplEYE: Creating a multilingual eye-tracking-while-reading corpus. In Proceedings of the 2025 Symposium on Eye Tracking Research and Applications, 1–11. ACM. find
  45. Kennedy, A., Hill, R., & Pynte, J. (2003). The Dundee corpus. In Proceedings of the 12th European Conference on Eye Movement. find
  46. Kim, N., & Linzen, T. (2020). COGS: A compositional generalization challenge based on semantic interpretation. In Proceedings of EMNLP, 9087–9105. find
  47. Kirby, S., Griffiths, T., & Smith, K. (2014). Iterated learning and the evolution of language. Current Opinion in Neurobiology, 28, 108–114. find
  48. Krifka, M. (2007). Basic notions of information structure. In Interdisciplinary Studies on Information Structure, 6, 13–55. find
  49. Kuribayashi, T., Oseki, Y., & Baldwin, T. (2024). Psychometric predictive power of large language models. In Findings of the Association for Computational Linguistics: NAACL 2024, 1983–2005. find
  50. Kuribayashi, T., Oseki, Y., Ben Taieb, S., Inui, K., & Baldwin, T. (2025). Large language models are human-like internally. Transactions of the Association for Computational Linguistics, 13, 1743– 1766. find
  51. Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207(4427), 203–205. find
  52. Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017). Building machines that learn and think like people. Behavioral and Brain Sciences, 40, e253. find
  53. Lake, B. M., & Baroni, M. (2018). Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In Proceedings of ICML, 2873–2882. find
  54. Lakretz, Y., Desbordes, T., King, J. R., Crabbé, B., Oquab, M., & Dehaene, S. (2021). Can RNNs learn recursive nested subject-verb agreements? In Proceedings of NAACL-HLT, 3558–3569. find
  55. Lazaridou, A., Potapenko, A., & Tieleman, O. (2020). Multi-agent communication meets natural language: Synergies between functional and structural language learning. In Proceedings of ACL, 7663–7674. find
  56. Levy, R. (2008). Expectation-based syntactic comprehension. Cognition, 106(3), 1126–1177. find
  57. Lewis, R. L., & Vasishth, S. (2005). An activation-based model of sentence processing as skilled memory retrieval. Cognitive Science, 29(3), 375– 419. find
  58. Linzen, T., Dupoux, E., & Goldberg, Y. (2016). Assessing the ability of LSTMs to learn syntax-sensitive dependencies. Transactions of the ACL, 4, 521–535. find our card
  59. Luke, S. G., & Christianson, K. (2018). The Provo Corpus: A large eye-tracking corpus with predictability norms. Behavior Research Methods, 50(2), 826–833. find
  60. MacWhinney, B. (2000). The CHILDES Project: Tools for Analyzing Talk (3rd ed.). Lawrence Erlbaum Associates. find
  61. Manning, C. D., Clark, K., Hewitt, J., Khandelwal, U., & Levy, O. (2020). Emergent linguistic structure in artificial neural networks trained by self-supervision. Proceedings of the National Academy of Sciences, 117(48), 30046–30054. find
  62. Marr, D. (1982). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. W.H. Freeman. find
  63. McCoy, R. T., Pavlick, E., & Linzen, T. (2019). Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of ACL, 3428–3448. find our card
  64. Mel’čuk, I. (1988). Dependency Syntax: Theory and Practice. SUNY Press. find
  65. Montague, R. (1973). The proper treatment of quantification in ordinary English. In Approaches to Natural Language, 221–242. Springer. find
  66. Nivre, J., et al. (2016). Universal Dependencies v1: A multilingual treebank collection. In Proceedings of LREC, 1659–1666. find our card
  67. Oh, B.-D., Zhu, H., & Schuler, W. (2025). The inverse scaling effect of pre-trained language model surprisal is not due to data leakage. In Findings of the Association for Computational Linguistics: ACL 2025, 1820–1827. find
  68. Osterhout, L., & Holcomb, P. J. (1992). Event-related brain potentials elicited by syntactic anomaly. Journal of Memory and Language, 31(6), 785–806. find
  69. Oudeyer, P. Y., Kaplan, F., & Hafner, V. V. (2007). Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation, 11(2), 265–286. find
  70. Piantadosi, S. T., & Hill, F. (2022). Meaning without reference in large language models. arXiv preprint arXiv:2208.02957. — [arXiv:2208.02957](https://arxiv.org/abs/2208.02957) find
  71. Ravfogel, S., Elazar, Y., Gonen, H., Twiton, M., & Goldberg, Y. (2020). Null it out: Guarding protected attributes by iterative nullspace projection. In Proceedings of ACL, 7237–7256. find our card
  72. Rayner, K. (2009). Eye movements and attention in reading, scene perception, and visual search. The Quarterly Journal of Experimental Psychology, 62(8), 1457–1506. find
  73. Ren, Y., Jin, R., Zhang, T., & Xiong, D. (2025). Do large language models mirror cognitive language processing? In Proceedings of the 31st International Conference on Computational Linguistics (COLING), 2988–3001. find
  74. Schaeffer, R., Miranda, B., & Koyejo, S. (2023). Are emergent abilities of large language models a mirage? In Advances in Neural Information Processing Systems, 36. find our card
  75. Schrimpf, M., et al. (2021). The neural architecture of language: Integrative modeling converges on predictive processing. Proceedings of the National Academy of Sciences, 118(45), e2105646118. find
  76. Shanahan, M. (2024). Talking about large language models. Communications of the ACM, 67(2), 68– 79. find
  77. Smith, L. B., Suanda, S. H., & Yu, C. (2014). The unrealized promise of infant statistical word-referent learning. Trends in Cognitive Sciences, 18(5), 251–258. find
  78. Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46(1-2), 159–216. find
  79. Soderstrom, M. (2007). Beyond babytalk: Re-evaluating the nature and content of speech input to preverbal infants. Developmental Review, 27(4), 501–532. find
  80. Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217), 1632–1634. find
  81. Tenenbaum, J. B., Kemp, C., Griffiths, T. L., & Goodman, N. D. (2011). How to grow a mind: Statistics, structure, and abstraction. Science, 331(6022), 1279–1285. find
  82. Tesnière, L. (1959). Éléments de syntaxe structurale. Klincksieck. find
  83. Tomasello, M. (2003). Constructing a Language: A Usage-Based Theory of Language Acquisition. Harvard University Press. find
  84. Vaswani, A., et al. (2017). Attention is all you need. In Advances in Neural Information Processing Systems, 30, 5998–6008. find
  85. Vig, J., et al. (2020). Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems, 33, 12388–12401. find
  86. Voita, E., Talbot, D., Moiseev, F., Sennrich, R., & Titov, I. (2019). Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of ACL, 5797–5808. find our card
  87. Warstadt, A., et al. (2020). BLiMP: The benchmark of linguistic minimal pairs for English. Transactions of the ACL, 8, 377–392. find
  88. Siegelman, N., Schroeder, S., Acartürk, C., et al. (2022). Expanding horizons of cross-linguistic research on reading: The Multilingual Eye-movement Corpus (MECO). Behavior Research Methods, 54(6), 2843–2863. find
  89. Warstadt, A., et al. (2023). Findings of the BabyLM Challenge: Sample-efficient pretraining on developmentally plausible corpora. In Proceedings of the BabyLM Challenge at CoNLL, 1–34. find
  90. Smith, N. J., & Levy, R. (2013). The effect of word predictability on reading time is logarithmic. Cognition, 128(3), 302–319. find
  91. Wei, J., et al. (2022). Emergent abilities of large language models. Transactions on Machine Learning Research. find our card
  92. Wilcox, E. G., Gauthier, J., Hu, J., Qian, P., & Levy, R. (2020). On the predictive power of neural language models for human real-time comprehension behavior. In Proceedings of CogSci, 1707– 1713. find
  93. Wilcox, E. G., Pimentel, T., Meister, C., Cotterell, R., & Levy, R. P. (2023). Testing the predictions of surprisal theory in 11 languages. Transactions of the Association for Computational Linguistics, 11, 1451–1470. find our card
  94. Wu, S., & Dredze, M. (2020). Are all languages created equal in multilingual BERT? In Proceedings of the 5th Workshop on Representation Learning for NLP, 120–130. find our card
  95. Xu, F., & Tenenbaum, J. B. (2007). Word learning as Bayesian inference. Psychological Review, 114(2), 245–272. find
  96. Zaghouani, W., & Charfi, A. (2018). AraP-Tweet: A large multi-dialect twitter corpus for gender, age and language variety identification. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zaghouani-2026-cognitive-alignment-large-language-models/