Elamur Research⌕
☰
home / papers / 2026 / Non-literal Meaning Representation in the Brain during Naturalistic Listening

Non-literal Meaning Representation in the Brain during Naturalistic Listening

Summary

The paper asks how the brain represents implied meaning during natural story listening. In continuous narratives, surface content and implied content are hard to separate computationally. The authors used a public fMRI dataset (Li et al., 2022) in which 35 native Mandarin speakers listened to the full Chinese audiobook of The Little Prince. Two linguistically trained annotators marked every sentence that carries non-literal meaning, such as metaphorical, symbolic or inferential readings, and wrote a paraphrase of what each one implies.

For these sentence pairs, the authors took final-token embeddings from LLaMA3.1-8B for the original sentence and for its interpretation. They then fit these features, alone and together, in whole-brain encoding models using banded ridge regression. Literal features best predicted activity in left frontotemporal language areas. Interpretation features reached further into the right hemisphere. Combining both improved prediction in bilateral temporal and dorsal frontal cortex.

Literal embeddings could not be linearly mapped onto non-literal ones. This suggests that LLM encodings of surface text miss interpretive meaning, and that model-brain alignment work should model that meaning explicitly.

Key points

  • Stimulus: the Chinese The Little Prince audiobook has 15,603 words in 1,289 sentences and lasts about 99 minutes.
    • It was presented in nine runs of roughly 10 minutes each.
    • fMRI data were projected to the fsaverage4 surface.
  • Annotations: 429 sentences (33.3%) were labeled non-literal and given human-written interpretations.
    • The two annotators resolved disagreements by discussion.
    • The sentence-aligned annotations are offered as a resource for future work.
  • Participants: 35 native Mandarin speakers took part (15 female, mean age 19.3).
    • Low-level acoustic and rate regressors (sentence rate, pitch, intensity) were handled with GLMs.
  • Linear probing: layer-wise ridge mappings from literal to non-literal embeddings were tested with 5-fold cross-validated R² against a shuffled-pair control.
    • The mappings did not reliably recover non-literal embeddings.
    • The two embedding types were separable in embedding space.
  • Limitations:
    • All non-literal subtypes were pooled into one category.
    • Interpretations come from only two annotators.
    • Only the Chinese portion of the dataset was used, although English and French data are also available.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.scil-main.22

@inproceedings{ma2026non,
  title = {{Non-literal Meaning Representation in the Brain during Naturalistic Listening}},
  author = {Ma, Zhengwu and Huang, Yuhan and Wang, Chengcheng and Li, Jixing},
  booktitle = {Proceedings of the Society for Computation in Linguistics 2026},
  year = {2026},
  doi = {10.18653/v1/2026.scil-main.22},
  url = {https://doi.org/10.18653/v1/2026.scil-main.22}
}

References

42 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Francesca M. M. Citron and Adele E. Goldberg. 2014. Metaphorical Sentences Are More Emotionally Engaging than Their Literal Counterparts. Journal of Cognitive Neuroscience, 26(11):2585–2595. find
  2. Jingmin An, Yilong Song, Ruolin Yang, Nai Ding, Lingxi Lu, Yuxuan Wang, Wei Wang, Chu Zhuang, Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North A… find
  3. Tom Dupré la Tour, Michael Eickenberg, Anwar O. Nunez-Elizalde, and Jack L. Gallant. 2022. Feature-space selection with banded ridge regression. NeuroImage, 264:119728. find
  4. Oscar Esteban, Rastko Ciric, Karolina Finc, Ross W. Blair, Christopher J. Markiewicz, Craig A. Moodie, James D. Kent, Mathias Goncalves, Elizabeth DuPre, Daniel E. P. Gomez, Zhifang Ye, Taylor Salo, Romain Valabregue, Inge K. Amlien, Franziskus Liem, Nir Jacoby, Hrvoje Stojić, Matthew Cieslak, Seba… find
  5. Zohar Eviatar and Marcel Adam Just. 2006. Brain correlates of discourse processing: An fMRI investigation of irony and conventional metaphor comprehension. Neuropsychologia, 44(12):2348–2359. find
  6. Evelina Fedorenko, Anna A. Ivanova, and Tamar I. Regev. 2024. The language network as a natural kind within the broader landscape of the human brain. Nature Reviews Neuroscience, 25(5):289–312. find
  7. Changjiang Gao, Jixing Li, Jiajun Chen, and Shujian Huang. 2024. Measuring meaning composition in the human brain with composition scores from large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11295– 11308… find
  8. Changjiang Gao, Zhengwu Ma, Jiajun Chen, Ping Li, Shujian Huang, and Jixing Li. 2025. Increasing alignment of large language models with language processing in the human brain. Nature Computational Science, 5(11):1080–1090. find
  9. Ariel Goldstein, Avigail Grinstein-Dabush, Mariano Schain, Haocheng Wang, Zhuoqiao Hong, Bobbi Aubrey, Samuel A. Nastase, Zaid Zada, Eric Ham, Amir Feder, Harshvardhan Gazula, Eliav Buchnik, Werner Doyle, Sasha Devore, Patricia Dugan, Roi Reichart, Daniel Friedman, Michael Brenner, Avinatan Hassidim… find
  10. Ariel Goldstein, Eric Ham, Mariano Schain, Samuel A. Nastase, Bobbi Aubrey, Zaid Zada, Avigail Grinstein-Dabush, Harshvardhan Gazula, Amir Feder, Werner Doyle, Sasha Devore, Patricia Dugan, Daniel Friedman, Michael Brenner, Avinatan Hassidim, Yossi Matias, Orrin Devinsky, Noam Siegelman, Adeen Flink… find
  11. Ariel Goldstein, Haocheng Wang, Leonard Niekerken, Mariano Schain, Zaid Zada, Bobbi Aubrey, Tom Sheffer, Samuel A. Nastase, Harshvardhan Gazula, Aditi Singh, Aditi Rao, Gina Choe, Catherine Kim, Werner Doyle, Daniel Friedman, Sasha Devore, Patricia Dugan, Avinatan Hassidim, Michael Brenner, Yossi Ma… find
  12. Ariel Goldstein, Zaid Zada, Eliav Buchnik, Mariano Schain, Amy Price, Bobbi Aubrey, Samuel A. Nastase, Amir Feder, Dotan Emanuel, Alon Cohen, Aren Jansen, Harshvardhan Gazula, Gina Choe, Aditi Rao, Catherine Kim, Colton Casto, Lora Fanda, Werner Doyle, Daniel Friedman, Patricia Dugan, Lucia Melloni,… find
  13. Alexandre Gramfort, Martin Luessi, Eric Larson, Denis A. Engemann, Daniel Strohmeier, Christian Brodbeck, Lauri Parkkonen, and Matti S. Hämäläinen. 2014. MNE software for processing MEG and EEG data. NeuroImage, 86:446–460. find
  14. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao,… find
  15. H. P. Grice. 1975. Logic and Conversation. In Speech Acts, pages 41–58. Brill. find
  16. Peter Hagoort. 2013. MUC (Memory, Unification, Control) and beyond. Frontiers in Psychology, 4. find
  17. Uri Hasson, Eunice Yang, Ignacio Vallines, David J. Heeger, and Nava Rubin. 2008. A Hierarchy of Temporal Receptive Windows in Human Cortex. Journal of Neuroscience, 28(10):2539–2550. find
  18. Gregory Hickok and David Poeppel. 2007. The cortical organization of speech processing. Nature Reviews Neuroscience, 8(5):393–402. find
  19. Alexander G. Huth, Wendy A. de Heer, Thomas L. Griffiths, Frédéric E. Theunissen, and Jack L. Gallant. 2016. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532(7600):453– 458. find
  20. Mark Jung-Beeman. 2005. Bilateral brain processes for comprehending natural language. Trends in Cognitive Sciences, 9(11):512–518. find
  21. Sreejan Kumar, Theodore R. Sumers, Takateru Yamakoshi, Ariel Goldstein, Uri Hasson, Kenneth A. Norman, Thomas L. Griffiths, Robert D. Hawkins, and Samuel A. Nastase. 2024. Shared functional specialization in transformer-based language models and the human brain. Nature Communications, 15(1):5523. find
  22. Ellen F. Lau, Colin Phillips, and David Poeppel. 2008. A cortical network for semantics: (de)constructing the N400. Nature Reviews Neuroscience, 9(12):920– 933. find
  23. Yulia Lerner, Christopher J. Honey, Lauren J. Silbert, and Uri Hasson. 2011. Topographic Mapping of a Hierarchy of Temporal Receptive Windows Using a Narrated Story. Journal of Neuroscience, 31(8):2906– 2915. find
  24. Jixing Li, Shohini Bhattasali, Shulin Zhang, Berta Franzluebbers, Wen-Ming Luh, R. Nathan Spreng, Jonathan R. Brennan, Yiming Yang, Christophe Pallier, and John Hale. 2022. Le petit prince multilingual naturalistic fmri corpus. Scientific Data, 9(1):530. find
  25. Eric Maris and Robert Oostenveld. 2007. Nonparametric statistical testing of EEG- and MEG-data. Journal of Neuroscience Methods, 164(1):177–190. find
  26. William Matchin and Gregory Hickok. 2020. The Cortical Organization of Syntax. Cerebral Cortex, 30(3):1481–1498. find
  27. Mohammad Momenian, Zhengwu Ma, Shuyi Wu, Chengcheng Wang, Jonathan Brennan, John Hale, Lars Meyer, and Jixing Li. 2024. Le petit prince hong kong (lpphk): Naturalistic fmri and eeg data from older cantonese speakers. Scientific data, 11(1):992. find
  28. Christophe Pallier, Anne-Dominique Devauchelle, and Stanislas Dehaene. 2011. Cortical representation of the constituent structure of sentences. Proceedings of the National Academy of Sciences, 108(6):2522– 2527. find
  29. Karalyn Patterson, Peter J. Nestor, and Timothy T. Rogers. 2007. Where do you know what you know? The representation of semantic knowledge in the human brain. Nature Reviews Neuroscience, 8(12):976– 987. find
  30. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. find
  31. Matthew A. Lambon Ralph, Elizabeth Jefferies, Karalyn Patterson, and Timothy T. Rogers. 2017. The neural and computational bases of semantic cognition. Nature Reviews Neuroscience, 18(1):42–55. find
  32. Alexander M Rapp, Dirk T Leube, Michael Erb, Wolfgang Grodd, and Tilo T. J Kircher. 2004. Neural correlates of metaphor processing. Cognitive Brain Research, 20(3):395–402. find
  33. Alexander M. Rapp, Dorothee E. Mutschler, and Michael Erb. 2012. Where in the brain is nonliteral language? A coordinate-based meta-analysis of functional magnetic resonance imaging studies. NeuroImage, 63(1):600–610. find
  34. Martin Schrimpf, Idan Asher Blank, Greta Tuckute, Carina Kauf, Eghbal A. Hosseini, Nancy Kanwisher, Joshua B. Tenenbaum, and Evelina Fedorenko. 2021. The neural architecture of language: Integrative modeling converges on predictive processing. Proceedings of the National Academy of Sciences, 118(45)… find
  35. William W. Seeley, Vinod Menon, Alan F. Schatzberg, Jennifer Keller, Gary H. Glover, Heather Kenna, Allan L. Reiss, and Michael D. Greicius. 2007. Dissociable intrinsic connectivity networks for salience processing and executive control. Journal of Neuroscience, 27(9). find
  36. Mariya Toneva, Tom M. Mitchell, and Leila Wehbe. 2022. Combining computational controls with natural text reveals aspects of meaning composition. Nature Computational Science, 2(11):745–757. find
  37. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc. find
  38. Pauli Virtanen, Ralf Gommers, Travis E Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, et al. 2020. Scipy 1.0: fundamental algorithms for scientific computing in python. Nature Methods, 17(3):261–272. find our card
  39. Qixuan Wang, Qian Zhou, Zhengwu Ma, Nan Wang, Tianyu Zhang, Yaoyao Fu, and Jixing Li. 2025. Le petit prince (lpp) multi-talker: Naturalistic 7 t fmri and eeg dataset. Scientific data, 12(1):829. find
  40. Shaoyun Yu, Chanyuan Gu, Kexin Huang, and Ping Li. 2024. Predicting the next sentence (not word) in large language models: What model-brain alignment tells us about discourse comprehension. Science Advances, 10(21):eadn7744. find
  41. Monika-Zita Zempleni, Remco Renken, John C. J. Hoeks, Johannes M. Hoogduin, and Laurie A. Stowe. 2007. Semantic ambiguity processing in sentence context: Evidence from event-related fMRI. NeuroImage, 34(3):1270–1279. find
  42. Rolf A. Zwaan and Gabriel A. Radvansky. 1998. Situation models in language comprehension and memory. Psychological Bulletin, 123(2):162–185. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/ma-2026-non-literal-meaning-representation-brain/