Elamur Research⌕
☰
home / papers / 2026 / Modeling semantic association in self-paced reading with language model embeddings

Modeling semantic association in self-paced reading with language model embeddings

Summary

How strongly a word relates in meaning to its context seems to affect reading even after accounting for how predictable the word is. Studies that measure this relatedness with language model embeddings differ widely in their choices, though. They use different embedding models, different ways of building the context vector, and different context sizes. The authors compared these choices systematically on naturalistic Dutch data from the TiNT corpus, which combines EEG and self-paced reading recordings. They built ten versions of semantic association. These combined a word2vec model (averaging either all words or only content words) or a Dutch sentence embedding model with several context definitions, from a single preceding word up to the whole preceding text. Bayesian hierarchical models and Bayes factors then tested whether each version predicted N400 amplitude and reading times beyond word predictability. The choice of embedding model changed the estimated effects. Only the sentence-embedding versions gave reliable evidence on both the neural and the behavioral measure. The study shows that operationalization choices can change the conclusions researchers draw about semantic association.

Key points

  • Data: TiNT corpus, with 71 participants recorded and 56 analyzed.
    • Each participant read eight Dutch texts of about 600 words.
    • Only the seven texts read in self-paced mode were used.
    • The N400 was taken as mean centroparietal amplitude at 300–500 ms.
  • Models: The word-level model was wikipedia2vec_nlwiki_20180420_300d, with context averaged over all words (WE) or content words (CWE). The sentence-level model was e5-large-trm-nl (SE), chosen for its strong MTEB(nld, v1) results.
  • POS tagging: spaCy nl_core_news_sm identified content words, defined as nouns, verbs, adjectives and adverbs.
  • Coverage: Sentence embeddings left far fewer words without a score.
    • SE lost 0.17% of words.
    • The CWE single-word window lost up to 3.91%.
  • Validation: All implementations were checked on the Federmeier and Kutas (1999) stimuli, extended with longer contexts of about 100 words generated by GPT-5.2. Every implementation ranked the target words correctly, and the sentence embedding models separated the target types more cleanly.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5hz52qgp2gdn

@inproceedings{ostergaard2026modeling,
  title = {{Modeling semantic association in self-paced reading with language model embeddings}},
  author = {Østergaard, Sara Møller and Enevoldsen, Kenneth and Alishahi, Afra and Nicenboim, Bruno},
  booktitle = {Proceedings of the 15th Workshop on Cognitive Modeling and Computational Linguistics},
  year = {2026},
  doi = {10.63317/5hz52qgp2gdn},
  url = {https://doi.org/10.63317/5hz52qgp2gdn}
}

References

48 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Christoph Aurnhammer, Francesca Delogu, Harm Brouwer, and Matthew W. Crocker. 2023. The P600 as a continuous index of integration effort. Psychophysiology, 60(9):e14302. find
  2. James M. Dickey and B. P. Lientz. 1970. The Weighted Likelihood Ratio, Sharp Hypotheses about Chances, the Order of a Markov Chain. The Annals of Mathematical Statistics, 41(1):214–226. find
  3. Nikolay Banar, Ehsan Lotfi, Jens Van Nooten, Cristina Arhiliuc, Marija Kliocaite, and Walter Daelemans. 2025. Mteb-nl and e5-nl: Embedding benchmark and models for dutch. find
  4. Susan F. Ehrlich and Keith Rayner. 1981. Contextual effects on word perception and eye movements during reading. Journal of Verbal Learning and Verbal Behavior, 20(6):641–655. find
  5. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics, 5:135–146. find
  6. Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemiński, Genta Indra Winata, Saba Sturua, Saiteja Utpala, Mathieu Ciancone, Marion Schaeffer, Gabriel Sequeira, Diganta Misra, Shreeya Dhakal, Jonathan Rystrøm, Roman Soloma… find
  7. Stefan L. Frank and Roel M. Willems. 2017. Word predictability and semantic similarity show distinct patterns of brain activity during language comprehension. Language, Cognition and Neuroscience, 32(9):1192–1203. find
  8. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple Contrastive Learning of Sentence Embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistic… find our card
  9. John Hale. 2001. A Probabilistic Earley Parser as a Psycholinguistic Model. In Second Meeting of the North American Chapter of the Association for Computational Linguistics. find our card
  10. Matthew Honnibal, Ines Montani, Sofie Van Lan-deghem, and Adriane Boyd. 2020. spaCy: Industrial-strength Natural Language Processing in Python. find
  11. Benedict Krieger, Harm Brouwer, Christoph Aurnhammer, and Matthew W. Crocker. 2024. On the limits of LLM surprisal as functional Explanation of ERPs. Proceedings of the Annual Meeting of the Cognitive Science Society, 46(0). find
  12. Gina R. Kuperberg, Tatiana Sitnikova, David Caplan, and Phillip J. Holcomb. 2003. Electrophysiological distinctions in processing conceptual relationships within simple sentences. Brain Research. Cognitive Brain Research, 17(1):117– 129. find
  13. Allyson Ettinger, Naomi Feldman, Philip Resnik, and Colin Phillips. 2016. Modeling N400 amplitude using vector space models of word representation. Proceedings of the Annual Meeting of the Cognitive Science Society, 38(0). find
  14. Marta Kutas and Kara D. Federmeier. 2011. Thirty Years and Counting: Finding Meaning in the N400 Component of the Event-Related Brain Potential (ERP). Annual Review of Psychology, 62(1):621–647. find
  15. Kara D. Federmeier and Marta Kutas. 1999. A Rose by Any Other Name: Long-Term Memory Structure and Sentence Processing. Journal of Memory and Language, 41(4):469–495. find
  16. I. Fischler, P. A. Bloom, D. G. Childers, S. E. Roucos, and N. W. Perry. 1983. Brain potentials related to stages of sentence verification. Psychophysiology, 20(4):400–409. find
  17. Litao Li, Leo Song, Steven Ding, Benjamin C. M. Fung, and Philippe Charland. 2025. Transforming Generic Coder LLMs to Effective Binary Code Embedding Models for Similarity Detection. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  18. Stefan L. Frank. 2017. Word Embedding Distance Does not Predict Word Reading Time. Proceedings of the Annual Meeting of the Cognitive Science Society, 39(0). find
  19. Steven G. Luke and Kiel Christianson. 2018. The Provo Corpus: A large eye-tracking corpus with predictability norms. Behavior Research Methods, 50(2):826–833. find
  20. Stefan L. Frank and Anna Aumeistere. 2024. An eye-tracking-with-EEG coregistration corpus of narrative sentences. Language Resources and Evaluation, 58(2):641–657. find
  21. Hannah Mechtenberg, James Reilly, Emily B Myers, and Jonathan E Peelle. 2025. Measuring brain sensitivity to semantic distance in spoken narrative comprehension. find
  22. Stefan L. Frank, Leun J. Otten, Giulia Galli, and Gabriella Vigliocco. 2015. The ERP response to the amount of information conveyed by words in sentences. Brain and Language, 140:1–11. find
  23. James A. Michaelov, Megan D. Bardolph, Cyma K. Van Petten, Benjamin K. Bergen, and Seana Coulson. 2024. Strong Prediction: Language Model Surprisal Explains Multiple N400 Effects. Neurobiology of Language, 5(1):107–135. find
  24. R Core Team. 2024. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. find
  25. James A. Michaelov and Benjamin K. Bergen. 2024. On the Mathematical Relationship Between Contextual Probability and N400 Amplitude. Open Mind, 8:859–897. find
  26. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving Language Understanding by Generative Pre-Training. OpenAI Technical Report. find
  27. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. find
  28. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. find
  29. Jeff Mitchell, Mirella Lapata, Vera Demberg, and Frank Keller. 2010. Syntactic and Semantic Factors in Processing Difficulty: An Integrated Measure. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 196–206, Uppsala, Sweden. Association for Computation… find
  30. Lavinia Salicchi and Yu-Yin Hsu. 2025. Different Reading Processing Stages or Different Brain Areas? A Computational Cognitive Investigation on N400, P600, and PNP. In Computational Psycholinguistics Meeting 2025. Conference presentation abstract. find
  31. Cory Shain. 2024. Word Frequency and Predictability Dissociate in Naturalistic Reading. Open Mind, 8:177–201. find
  32. Bruno Nicenboim, Daniel J. Schad, and Shravan Vasishth. 2025. The influence of priors: sensitivity analysis. In Introduction to Bayesian Data Analysis for Cognitive Science, 1st edn edition, chapter 3.4, page 634. Chapman & Hall. find
  33. Stan Development Team. 2023. Stan Reference Manual. find
  34. Bruno Nicenboim and Shravan Vasishth. 2016. Statistical methods for linguistic research: Foundational Ideas—Part II. Language and Linguistics Compass, 10(11):591–613. find
  35. Kate Stone and Milena Rabovsky. 2025. The Role of Syntactic and Semantic Cues in Preventing Temporary Illusions of Plausibility. Journal of Cognitive Neuroscience, 37(9):1535–1561. find
  36. Mante S. Nieuwland and Jos J. A. Van Berkum. 2005. Testing the limits of the semantic illusion phenomenon: ERPs reveal temporary semantic change deafness in discourse comprehension. Cognitive Brain Research, 24(3):691–701. find
  37. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. LLaMA: Open and Efficient Foundation Language Models. find
  38. OpenAI. 2025. GPT-5.2 via OpenAI API. find
  39. Mehdi Parviz, Mark Johnson, Blake Johnson, and Jon Brock. 2011. Using Language Models and Latent Semantic Analysis to Characterise the N400m Neural Response. In Proceedings of the Australasian Language Technology Association Workshop 2011, pages 38–46, Canberra, Australia. find
  40. Matthew J. Traxler, Donald J. Foss, Rachel E. Seely, Barbara Kaup, and Robin K. Morris. 2000. Priming in Sentence Processing: Intralexical Spreading Activation, Schemas, and Situation Models. Journal of Psycholinguistic Research, 29(6):581– 595. find
  41. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. Multilingual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672. find our card
  42. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics. find
  43. Roslyn Wong, Erik D. Reichle, and Aaron Veldre. 2024. Prediction in reading: A review of predictability effects, their theoretical implications, and beyond. Psychonomic Bulletin & Review. find
  44. Tiago Pimentel, Clara Meister, Ethan G. Wilcox, Roger Levy, and Ryan Cotterell. 2023. On the Effect of Anticipation on Reading Times. find
  45. Haoyin Xu, Masaki Nakanishi, and Seana Coulson. 2024. Revisiting Joke Comprehension with Surprisal and Contextual Similarity: Implication from N400 and P600 Components. Proceedings of the Annual Meeting of the Cognitive Science Society, 46(0). find
  46. Joel Pynte, Boris New, and Alan Kennedy. 2008. On-line contextual influences during reading normal text: A multiple-regression analysis. Vision Research, 48(21):2172–2183. find
  47. Ikuya Yamada, Akari Asai, Jin Sakuma, Hiroyuki Shindo, Hideaki Takeda, Yoshiyasu Takefuji, and Yuji Matsumoto. 2020. Wikipedia2Vec: An efficient toolkit for learning and visualizing the embeddings of words and entities from Wikipedia. In Proceedings of the 2020 Conference on Empirical Methods in Nat… find
  48. Sara Møller Østergaard, Lenneke Lichtenberg, Laura Boon, and Bruno Nicenboim. 2025. A Corpus of Joint EEG and Self-Paced Reading of Natural Dutch Texts. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ostergaard-2026-modeling-semantic-association-self-paced/