Elamur Research⌕
☰
home / papers / 2026 / Grounded Concreteness: Human-Like Concreteness Sensitivity in Vision–Language Models

Grounded Concreteness: Human-Like Concreteness Sensitivity in Vision–Language Models

Paper · 2026 · Aryan Roy, Zekun Wang and Christopher J. MacLellan

Summary

The paper asks whether multimodal training gives language models a more human-like sense of how concrete or abstract words are, even when the models only receive text. The authors compare Llama 3.1 text-only models with their Llama Vision 3.2 counterparts at two scales (8B and 70B). Because the language backbone stays as comparable as possible, the vision–language training acts as an ablation of perceptual grounding. Images are never used at inference.

Concreteness is probed at three levels:

  • Behavior: how QA accuracy changes with question concreteness.
  • Representations: whether token embeddings cluster by concreteness bin.
  • Processing: whether attention entropy, a measure of reliance on context, differs between abstract and concrete tokens.

The authors also elicit token-level concreteness ratings from the models and compare them with human norms.

The VLMs consistently look more human-like. They gain more on concrete questions, form tighter concreteness clusters, separate abstract from concrete tokens more sharply in attention, and give ratings closer to the human ones. This positions concreteness as an interpretable axis for diagnosing grounding in foundation models.

Key points

  • Models compared: Llama 3.1 (text) versus Llama Vision 3.2 at 8B and 70B. The pattern of effects is qualitatively the same at both sizes.
  • Human reference: the concreteness norms of Brysbaert et al. (2014), covering about 40K English lemmas. Model rating distributions are compared with them using symmetric KL divergence. The VLM shows lower divergence, and its divergence drops faster as words get more concrete.
  • Embedding analysis: t-SNE projections plus intra-cluster dispersion across concreteness bins. VLM tokens cluster more tightly, especially for highly concrete words.
  • Attention analysis: concreteness correlates more negatively with attention entropy in VLMs. This fits context-availability theory, which holds that abstract words lean more on context. The attention result was replicated on a Qwen2 / Qwen2-VL pair.
  • Limitations: the full pipeline was run only on one model family, and token frequency may be a confound. Only final checkpoints were studied, with no analysis of how the effects develop during training.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Georgia Institute of TechnologyAtlanta · United States133 works · 24,497 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksVision & language · 657 works

Cite

Cite as Aryan Roy, Zekun Wang and Christopher J. MacLellan. 2026. Grounded Concreteness: Human-Like Concreteness Sensitivity in Vision–Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.2081

@inproceedings{roy2026grounded,
  title = {{Grounded Concreteness: Human-Like Concreteness Sensitivity in Vision–Language Models}},
  author = {Aryan Roy, Zekun Wang and Christopher J. MacLellan},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.2081},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.2081}
}

References

56 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jean-Baptiste Alayrac, Jeffrey Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, and 1 others. 2022. Flamingo: a visual language model for few-shot learning. In Advances in Neural Information Processing Systems.
  2. Morris Alper, Michael Fiman, and Hadar Averbuch-Elor. 2023. Is bert blind? exploring the effect of vision-and-language pretraining on visual language understanding. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6778–6788. find
  3. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. Preprint, arXiv:2308.12966.
  4. Lawrence W. Barsalou. 2008. Grounded cognition. Annual Review of Psychology, 59:617–645. find
  5. Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics.
  6. Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Chai, Mirella Lapata, Angeliki Lazaridou, Jonathan May, Aleksandr Nisnevich, Nicolas Pinto, and Joseph Turian. 2020a. Experience grounds language. In Proceedings of the 2020 Conference on Empirical Methods in Natural Lan…
  7. Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. 2020b. PIQA: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI Conference on Artificial Intelligence.
  8. Khyathi Chandu, Siva Reddy, Alan W. Black, Yulia Tsvetkov, and Eric Nyberg. 2021. Grounding “grounding” in NLP. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Association for Computational Linguistics. find
  9. Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. Uniter: Universal image-text representation learning. In Proceedings of the European Conference on Computer Vision (ECCV).
  10. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. BoolQ: Exploring the surprising difficulty of natural yes/no questions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:…
  11. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try ARC, the AI2 reasoning challenge. arXiv preprint arXiv:1803.05457.
  12. Max Coltheart. 1981. The MRC psycholinguistic database. Quarterly Journal of Experimental Psychology. find
  13. Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison-Burch, and 31 others. 2025. M… — Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models
  14. Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.
  15. Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, and 6 others. 2021. A mathe… find
  16. Taichi Iki and Akiko Aizawa. 2021. Effect of visual extensions on natural language understanding in vision-and-language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2189–2196, Online and Punta Cana, Dominican Republic. Association for Compu… find
  17. Richard Futrell, Ethan Wilcox, Takashi Morita, Peng Qian, Miguel Ballesteros, and Roger Levy. 2019. Neural language models as psycholinguistic subjects: Representations of syntactic state. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Lingui…
  18. Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the ability of LSTMs to learn syntax-sensitive dependencies. Transactions of the Association for Computational Linguistics, 4:521–535.
  19. Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Goldberg. 2022. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 30–45, Abu Dhabi, United Arab Emirates. Asso…
  20. Adam Goodkind and Klinton Bicknell. 2018. Predictive power of word surprisal for reading times is a linear function of language model quality. In Proceedings of the 8th Workshop on Cognitive Modeling and Computational Linguistics (CMCL 2018), pages 10–18, Salt Lake City, Utah. Association for Comput…
  21. Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2019. A survey of methods for explaining black box models. ACM Computing Surveys, 51(5):1–42. find
  22. Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. Colorless green recurrent networks dream hierarchically. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume…
  23. Stevan Harnad. 1990. The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1–3):335–346. find
  24. Jack Hessel, David Mimno, and Lillian Lee. 2018. Quantifying the visual concreteness of words and topics in multimodal datasets. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers),… find
  25. Felix Hill, Roi Reichart, and Anna Korhonen. 2014. Multi-modal models for concrete and abstract concept meaning. Transactions of the Association for Computational Linguistics, 2:285–296. find
  26. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. arXiv preprint arXiv:2304.08485.
  27. Ziqiao Ma, Jiayi Pan, and Joyce Chai. 2023. World-to-words: Grounded open vocabulary acquisition through fast mapping in vision-language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 524–544, Toronto, Canada. Associa… find
  28. Ziqiao Ma, Zekun Wang, and Joyce Chai. 2025. Babysit a language model from scratch: Interactive language learning by trials and demonstrations. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (… find
  29. Avinash Madasu and Vasudev Lal. 2023. Is multimodal vision supervision beneficial to language? 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 2637– 2642. find
  30. Meta AI. 2024a. The llama 3.1 model family. Technical report and model cards for Llama 3.1 text-only models. find
  31. Meta AI. 2024b. The llama 3.2 model family. Includes Llama 3.2 Vision and Vision-Instruct model cards. find
  32. Timothée Mickus, Elaine Zosa, and Denis Paperno. 2023. Grounded and well-rounded: A methodological approach to the study of cross-modal and cross-lingual grounding. arXiv preprint arXiv:2310.11938. find
  33. Allan Paivio. 1990. Mental Representations: A Dual Coding Approach. Oxford University Press, New York, NY. find
  34. Eva Portelance. 2022a. Neural Network Approaches to the Study of Word Learning. Ph.D. thesis, Stanford University. find
  35. Eva Portelance. 2022b. Neural network approaches to the study of word learning. Ph.D. thesis, McGill University. find
  36. Yulu Qin, Dheeraj Varghese, Adam Dahlgren Lindström, Lucia Donatelli, Kanishka Misra, and Najoung Kim. 2025. Vision-and-language training helps deploy taxonomic knowledge but does not fundamentally alter it. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  37. Ekta Sood, Simon Tannert, Philipp Müller, and Andreas Bulling. 2020b. Improving natural language processing tasks with human gaze-guided neural attention. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA. Curran Associates Inc. find
  38. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the 38th Internatio…
  39. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  40. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. WinoGrande: An adversarial winograd schema challenge at scale. In Proceedings of the AAAI Conference on Artificial Intelligence. find
  41. Hao Tan and Mohit Bansal. 2019. Lxmert: Learning cross-modality encoder representations from transformers. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5099–51…
  42. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. Social IQa: Commonsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processi… find
  43. Paula J. Schwanenflugel, Carolyn Akin, and Wei-Ming Luh. 1992. Context availability and the recall of abstract and concrete words. Memory & Cognition, 20(1):96–104. find
  44. Paula J. Schwanenflugel and Edward J. Shoben. 1983. Differential context effects in the comprehension of abstract and concrete verbal materials. Journal of Experimental Psychology: Learning, Memory, and Cognition, 9(1):82–102. find
  45. Haoyue Shi, Jiayuan Mao, Kevin Gimpel, and Karen Livescu. 2019. Visually grounded neural syntax acquisition. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1842–1861, Florence, Italy. Association for Computational Linguistics. find
  46. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. LLaMA: Open and efficient foundation language models. arXiv pr…
  47. Peter Turney, Yair Neuman, Dan Assaf, and Yohai Cohen. 2011. Literal and metaphorical sense identification through concrete and abstract context. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 680– 690, Edinburgh, Scotland, UK. Association for Comput…
  48. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems.
  49. Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guillaume Couairon, Wojciech Galuba, Marcus Rohrbach, and Douwe Kiela. 2022. Flava: A foundational language and vision alignment model. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15617–15629. find
  50. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024. Qwen2-vl: Enhancing vision-language model’s perception of the w…
  51. Ekta Sood, Simon Tannert, Diego Frassinelli, Andreas Bulling, and Ngoc Thang Vu. 2020a. Interpreting attention models with human visual attention in machine reading comprehension. In Proceedings of the 24th Conference on Computational Natural Language Learning, pages 12–25, Online. Association for C… find
  52. Alex Warstadt, Aaron Mueller, Leshem Choshen, Ethan Wilcox, Chengxu Zhuang, Juan Ciro, Rafael Mosquera, Bhargavi Paranjabe, Adina Williams, Tal Linzen, and 1 others. 2023. Findings of the BabyLM challenge: Sample-efficient pretraining on developmentally plausible corpora. In Proceedings of the BabyL… find
  53. Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. BLiMP: The benchmark of linguistic minimal pairs for English. Transactions of the Association for Computational Linguistics, 8:377– 392.
  54. Christian Wartena. 2024. Estimating word concreteness from contextualized embeddings. In Proceedings of the 20th Conference on Natural Language Processing (KONVENS 2024), pages 81–88, Vienna, Austria. Association for Computational Linguistics. find
  55. Moran Yanuka, Morris Alper, Hadar Averbuch-Elor, and Raja Giryes. 2024. ICC : Quantifying image caption concreteness for multimodal dataset curation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 11048–11064, Bangkok, Thailand. Association for Computational Linguistic… find
  56. Leran Zhang and Nora Hollenstein. 2024. Eye-tracking features masking transformer attention in question-answering tasks. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 7057–7070, Torino, Italia. ELRA… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/roy-2026-grounded-concreteness-human-like-concreteness/