Elamur Research⌕
☰
home / papers / 2024 / Humans Learn Language from Situated Communicative Interactions. What about Machines?

Humans Learn Language from Situated Communicative Interactions. What about Machines?

Paper · 2024 · Katrien Beuls and Paul Van Eecke

Summary

This position paper by Beuls and Van Eecke (Computational Linguistics, 2024) contrasts how children acquire language with how large language models (LLMs) are trained. Children learn through meaningful, goal-directed exchanges embedded in a shared physical context. LLMs instead derive meaning from the statistical distribution of tokens in text. The authors attribute three major LLM weaknesses to this design: heavy data requirements, weak human-like logical and pragmatic reasoning, and vulnerability to bias. As an alternative, they model language learning in artificial agents, building on usage-based and constructivist theory. In this framework, learning combines intention reading, which forms abductive guesses about what a speaker means, with pattern finding, which inductively generalizes those guesses into constructions. Two agent experiments illustrate the approach. The resulting linguistic knowledge is grounded in communicative function rather than collocation, is learned more data-efficiently, and avoids hallucination. The authors do not offer a practical replacement for LLMs. They argue that modeling situated interaction is a promising path toward more human-like machine language processing.

Key points

  • Theoretical basis: The paper adopts Tomasello's (2003) distinction between intention reading and pattern finding. It describes how these two processes reinforce each other in a two-way loop.
  • Construction types: Holophrastic constructions are whole utterances paired with a meaning (e.g., "bear-gone"). Holistic constructions are single words such as "bear" or "ball." Item-based constructions contain open slots (e.g., "X-gone," "gimme-X"). Constructions become stronger or weaker depending on whether they lead to successful communication.
  • Experiment 1 (grounded concept learning): A population of autonomous agents self-organizes a shared convention for referring to arbitrary objects in their environment.
  • Experiment 2 (tutor–learner): A learner agent acquires constructions for asking and answering English questions about its surroundings. These constructions map utterances to situationally grounded, executable meaning representations.
  • Publication details: The paper is published open access under CC BY-NC-ND 4.0 (DOI 10.1162/coli_a_00534). Funding came from EU Horizon 2020 (grant 951,846), the Flanders AI programme, ARIAC (DigitalWallonia4.ai), and the FNRS-FWO HERMES project. The provided excerpt omits the middle of the paper, so no quantitative results are reported here.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Vrije Universiteit BrusselBrussels · Belgium12 works · 737 citations🎓University of NamurNamur · Belgium6 works · 21 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 works

Cite

Cite as Katrien Beuls and Paul Van Eecke. 2024. Humans Learn Language from Situated Communicative Interactions. What about Machines?. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00534

@article{beuls2024humans,
  title = {{Humans Learn Language from Situated Communicative Interactions. What about Machines?}},
  author = {Katrien Beuls and Paul {Van Eecke}},
  journal = {Computational Linguistics},
  year = {2024},
  doi = {10.1162/coli\_a\_00534},
  url = {https://doi.org/10.1162/coli_a_00534}
}

References

52 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Apidianaki, Marianna. 2022. From word types to tokens and back: A survey of approaches to word meaning representation and interpretation. Computational Linguistics, 49(2):465–523.
  2. Aslin, Richard N. 2017. Statistical learning: A powerful mechanism that operates by mere exposure. Wiley Interdisciplinary Reviews: Cognitive Science, 8(1–2):e1373. https://doi.org/10.1002/wcs.1373, PubMed: 27906526 Austin, John L. 1962. How to Do Things with Words. — [doi:10.1002/wcs.1373](https://doi.org/10.1002/wcs.1373) ↗
  3. Oxford University Press, London. Baevski, Alexei Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli. 2022. Data2vec: A general framework for self-supervised learning in speech, vision and language. In International Conference on Machine Learning, pages 1298–1312. find
  4. Bai, Yuntao, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. Baroni, Marco, Georgiana D… find
  5. Bruner, Jerome. 1983. Learning to use language. find
  6. Oxford University Press, Oxford, United Kingdom. Bybee, Joan. 2010. Language, Usage and Cognition. Cambridge University Press, Cambridge, United Kingdom. https://doi.org/10.1017/CBO9780511750526 Chang, Tyler A. and Benjamin K. Bergen. 2024. Language model behavior: A comprehensive survey. Computati… — [doi:10.1017/cbo9780511750526](https://doi.org/10.1017/cbo9780511750526) ↗
  7. Chowdhery, Aakanksha, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. PaLM: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.
  8. Clark, Herbert H. 1996. Using Language. Cambridge University Press, Cambridge, United Kingdom. https://doi.org/10 .1017/CBO9780511620539 Coelho Mollo, Dimitri and Raphaël Millière. 2023. The vector grounding problem. arXiv preprint arXiv:2304.01481v1. Cortez, Paulo, Antonio Cerdeira, Fernando Alme…
  9. Croft, William. 2001. Radical construction grammar: Syntactic theory in typological perspective. Oxford University Press, Oxford, United Kingdom. https://doi.org/10.1093/acprof:oso/9780198299554 .001.0001 Dal Pozzolo, Andrea, Olivier Caelen, Yann-Aël Le Borgne, Serge Waterschoot, and Gianluca Bont… — [doi:10.1093/acprof:oso/9780198299554](https://doi.org/10.1093/acprof:oso/9780198299554) ↗
  10. Darwin, Charles R. 1871. The Descent of Man, and Selection in Relation to Sex, 1st edition, volume 1. John Murray, London, United Kingdom. https://doi.org/10.1037/12293-000 Dasgupta, Ishita, Andrew K Lampinen, Stephanie C. Y. Chan, Antonia Creswell, Dharshan Kumaran, James L. McClelland, and Felix… ↗
  11. Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies…
  12. Diessel, Holger. 2017. Usage-based linguistics. In Mark Aronoff, editor, Oxford Research Encyclopedia of Linguistics. Oxford University Press, Oxford, United Kingdom. https://doi.org/10.1093/acrefore/9780199384655.013.363 Doumen, Jonas, Katrien Beuls, and Paul Van Eecke. 2023. Modelling language ac… — [doi:10.18653/v1/2023.findings](https://doi.org/10.18653/v1/2023.findings) ↗
  13. Driess, Danny, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Ze… find
  14. Fan, Yue, Jing Gu, Kaizhi Zheng, and Xin Wang. 2023. R2H: Building multimodal navigation helpers that respond to help requests. In Proceedings of the 2023 find
  15. Conference on Empirical Methods in Natural Language Processing, pages 14803–14819. https://doi.org/10.18653/v1/2023 .emnlp-main.915 Firth, John R. 1957. A synopsis of linguistic theory, 1930–1955. In Studies in Linguistic Analysis. Basil Blackwell, Oxford, pages 1–31. — [doi:10.18653/v1/2023](https://doi.org/10.18653/v1/2023) ↗
  16. Frank, Michael C. 2023. Bridging the data gap between children and large language models. Trends in Cognitive Sciences, 27(11):990–992. https://doi.org/10 .1016/j.tics.2023.08.007, PubMed: 37659919 Gabrieli, Gianmarco, Michal Muszynski, Edouard Thomas, David Labbe, and Patrick W. Ruch. 2022. Acceler… find
  17. Oxford University Press, Oxford, United Kingdom. Goodhart, Charles. 1975. Problems of monetary management: The U.K. experience. In Papers in Monetary Economics. Reserve Bank of Australia, Sydney, pages 1–20. find
  18. Grice, Paul. 1967. Logic and conversation. In Paul Grice, editor, Studies in the Way of Words. Harvard University Press, Cambridge, MA, USA, pages 41–58. find
  19. Harris, Zellig S. 1954. Distributional structure. Word, 10(2–3):146–162. https://doi.org/10.1080/00437956 .1954.11659520 Hill, Felix, Roi Reichart, and Anna Korhonen. 2015. SimLex-999: Evaluating semantic models with (genuine) similarity estimation. Computational Linguistics, 41(4):665–695. https://… ↗
  20. Hong, Xudong, Margarita Ryzhova, Daniel Adrian Biondi, and Vera Demberg. 2023. Do large language models and humans have similar behaviors in causal inference with script knowledge? arXiv preprint arXiv.07311. find
  21. Hu, Jennifer, Sammy Floyd, Olessia Jouravlev, Evelina Fedorenko, and Edward Gibson. 2023. A fine-grained comparison of pragmatic language understanding in humans and language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p…
  22. Jiang, Nanjiang, and Marie-Catherine de Marneffe. 2021. He thinks he knows better than the doctors: BERT for event factuality fails on pragmatics. Transactions of the Association for Computational Linguistics, 9:1081–1097. https://doi.org/10.1162/tacl a 00414 Johnson, Justin, Bharath Hariharan, Lau… ↗
  23. Kravtchenko, Ekaterina and Vera Demberg. 2022. Informationally redundant utterances elicit pragmatic inferences. Cognition, 225:105159. https://doi.org/10.1016/j.cognition.2022.105159, PubMed: 35580451 Kuhl, Patricia K., Feng-Ming Tsao, and Huei-Mei Liu. 2003. Foreign-language experience in infancy… — [doi:10.1016/j.cognition.2022.105159](https://doi.org/10.1016/j.cognition.2022.105159) ↗
  24. Stanford University Press, Stanford CA, USA. Lauriola, Ivano, Alberto Lavelli, and Fabio Aiolli. 2022. An introduction to deep learning in natural language processing: Models, techniques, and tools. Neurocomputing, 470(C):443–456. https://doi.org/10.1016/j.neucom.2021.05.103 Lazaridou, Angeliki, an… — [doi:10.1016/j.neucom.2021.05.103](https://doi.org/10.1016/j.neucom.2021.05.103) ↗
  25. Li, KunChang, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. 2023. VideoChat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355.
  26. Lieven, Elena. 2014. First language learning from a usage-based approach. In Thomas Herbst, Hans-Jörg Schmid, and Susen Faulhaber, editors, Constructions Collocations Patterns. De Gruyter Mouton, Berlin, pages 9–32. https://doi.org/10.1515/9783110356854.9 Liu, Ruibo, Ge Zhang, Xinyu Feng, and Soro… ↗
  27. MacWhinney, Brian. 2014. Item-based patterns in early syntactic development. In Thomas Herbst, Hans-Jörg Schmid, and Susen Faulhaber editors, Constructions Collocations Patterns. De Gruyter Mouton, Berlin, pages 33–69. https://doi.org/10.1515/9783110356854.33 Mandelkern, Matthew and Tal Linzen. 20… — [doi:10.1515/9783110356854.33](https://doi.org/10.1515/9783110356854.33) ↗
  28. Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L. Griffiths. 2023. Embers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv preprint arXiv:2309.13638. find
  29. McKenna, Nick, Tianyi Li, Liang Cheng, Mohammad Javad Hosseini, Mark Johnson, and Mark Steedman. 2023. Sources of hallucination by large language models on inference tasks. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2758–2774. https://doi.org/10.18653/v1/2023 .fi… ↗
  30. Mikolov, Tomas, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26 (NIPS 2013), pages 1–9.
  31. Mitchell, Melanie and David C. Krakauer. 2023. The debate over understanding in AI’s large language models. Proceedings of the National Academy of Sciences, 120(13):e2215907120. https://doi.org/10.1073/pnas.2215907120, PubMed: 36943882 — [doi:10.1073/pnas.2215907120](https://doi.org/10.1073/pnas.2215907120) ↗
  32. Mitchell, Melanie, Alessandro B. Palmarini, and Arsenii Kirillovich Moskvichev. 2024. Comparing humans, GPT-4, and GPT-4v on abstraction and reasoning tasks. In AAAI 2024 Workshop on “Are Large Language Models Simply Causal Parrots?”, 9 pages. https://llmcp.cause-lab.net/pdf/LLMCP 4.pdf. ↗
  33. Navigli, Roberto, Simone Conia, and Bjorn Ross. 2023. Biases in large language models: Origins, inventory, and discussion. Journal of Data and Information Quality, 15(2):1–21. https://doi.org/10.1145/3597307 Nelson, Katherine. 1998. Language in Cognitive Development: The Emergence of the Mediated M… — [doi:10.1145/3597307](https://doi.org/10.1145/3597307) ↗
  34. Cambridge University Press, Cambridge, United Kingdom. Nevens, Jens, Jonas Doumen, Paul Van Eecke, and Katrien Beuls. 2022. Language acquisition through intention reading and pattern finding. In Proceedings of the 29th International Conference on Computational Linguistics, pages 15–25. find
  35. Nevens, Jens, Paul Van Eecke, and Katrien Beuls. 2020. From continuous observations to symbolic concepts: A discrimination-based strategy for grounded concept learning. Frontiers in Robotics and AI, 7(84):1–20. https://doi.org/10.3389/frobt.2020.00084, PubMed: 33501251 Ngo, Richard, Lawrence Chan,… — [doi:10.3389/frobt.2020.00084](https://doi.org/10.3389/frobt.2020.00084) ↗
  36. Nissim, Malvina, Rik van Noord, and Rob van der Goot. 2020. Fair is better than sensational: Man is to doctor as woman is to doctor. Computational Linguistics, 46(2):487–497. https://doi.org/10 .1162/coli a 00379 Oliphant, Michael. 1999. The learning barrier: Moving from innate to learned systems of…
  37. Ouyang, Long, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744.
  38. Park, Kyungseo, Hyunwoo Yuk, M. Yang, Junhwi Cho, Hyosang Lee, and Jung Kim. 2022. A biomimetic elastomeric robot skin using electrical impedance and acoustic tomography for tactile sensing. Science Robotics, 7(67):eabm7187. https://doi.org/10.1126/scirobotics.abm7187, PubMed: 35675452 Pavlick, Ell… ↗
  39. Piantadosi, Steven T. and Felix Hill. 2022. Meaning without reference in large language models. arXiv preprint arXiv:2208.02957v2. — [arXiv:2208.02957](https://arxiv.org/abs/2208.02957) find
  40. Plotkin, Gordon D. 1970. A note on inductive generalization. Machine Intelligence, 5(1):153–163. find
  41. Radford, Alec, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the 38th Internati…
  42. Rehder, Bob, Missy E. Schreiner, Michael B. W. Wolfe, Darrell Laham, Thomas K. Landauer, and Walter Kintsch. 1998. Using latent semantic analysis to assess knowledge: Some technical considerations. Discourse Processes, 25(2–3):337–354. https://doi.org/10.1080/01638539809545031 Rogers, Anna, Olga Ko… ↗
  43. Saffran, Jenny R., Richard N. Aslin, and Elissa L. Newport. 1996. Statistical learning by 8-month-old infants. Science, 274(5294):1926–1928. https://doi.org/10.1126/science.274.5294.1926, PubMed: 8943209 Schleicher, August. 1869. Darwinism Tested by the Science of Language. English Translation of S… ↗
  44. Shanahan, Murray. 2024. Talking about large language models. Communication of the ACM, 67(2):68–79. https://doi.org/10.1145/3624724 Shichman, Mollie, Claire Bonial, Austin Blodgett, Taylor Hudson, Francis Ferraro, and Rachel Rudinger. 2023. Use defines possibilities: Reasoning about object function… ↗
  45. Shiffrin, Richard and Melanie Mitchell. 2023. Probing the psychology of AI models. Proceedings of the National Academy of Sciences, 120(10):e2300963120. https://doi.org/10.1073/pnas.2300963120, PubMed: 36857344 Skalse, Joar, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. 2022. Defining a… ↗
  46. Sperber, Dan and Deirdre Wilson. 1986. Relevance: Communication and cognition. find
  47. Harvard University Press, Cambridge, MA, USA. Spranger, Michael, Simon Pauw, Martin Loetzsch, and Luc Steels. 2012. Open-ended procedural semantics. In Luc Steels and Manfred Hild, editors, Language Grounding in Robots. Springer, New York, NY, USA, 153–172. https://doi.org/10.1007/978-1-4614 -3064-… ↗
  48. Tan, Xiaoyu, Shaojie Shi, Xihe Qiu, Chao Qu, Zhenting Qi, Yinghui Xu, and Yuan Qi. 2023. Self-criticism: Aligning large language models with their understanding of helpfulness, honesty, and harmlessness. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Indus…
  49. Trott, Sean, Cameron Jones, Tyler Chang, James Michaelov, and Benjamin Bergen. 2023. Do large language models know what humans know? Cognitive Science, 47:e13309. https://doi.org/10 .1111/cogs.13309, PubMed: 37401923 Turney, Peter D. and Patrick Pantel. 2010. From frequency to meaning: Vector space…
  50. Vallor, Shannon. 2024. The AI Mirror: How to Reclaim our Humanity in an Age of Machine Thinking. Oxford University Press, Oxford, United Kingdom. https://doi.org/10 .1093/oso/9780197759066.001.0001 Van Eecke, Paul. 2018. Generalisation and Specialisation Operators for Computational Construction Gram… find
  51. Van Eecke, Paul, Lara Verheyen, Tom Willaert, and Katrien Beuls. 2023. The Candide model: How narratives emerge where observations meet beliefs. In Proceedings of the 5th Workshop on Narrative Understanding (WNU), pages 48–57. https://doi.org/10.18653/v1/2023 .wnu-1.7 Vaswani, Ashish, Noam Shazeer,… ↗
  52. Woods, William A. 1968. Procedural semantics for a question-answering machine. In Proceedings of the December 9–11, 1968, Fall Joint Computer Conference, Part I, pages 457–471, New York, NY, USA. https://doi.org/10.1145/1476589 .1476653 Yamazaki, Takato, Tomoya Mizumoto, Katsumasa Yoshikawa, Masaya… ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/beuls-2024-humans-learn-language-situated-communicative/