Elamur Research⌕
☰
home / papers / 2023 / Understanding Natural Language Understanding Systems. A Critical Analysis

Understanding Natural Language Understanding Systems. A Critical Analysis

Paper · 2023 · Alessandro Lenci

Summary

This is a critical-analysis (position) paper, not an empirical study. It introduces no new dataset, treebank, or model. Lenci asks whether recent NLU systems such as chatGPT and LaMDA actually "understand" language or only appear to. Heavy media hype and the human habit of describing these systems in mental terms make the question more pressing.

He first explains where foundation models get their abilities. Transformer networks are pretrained with self-supervised word prediction on very large unlabeled corpora. They are then adapted to many downstream tasks, replacing the older approach of training one model per task. On this account, their "knowledge" is purely distributional, built only from co-occurrence statistics, which links them to distributional semantics.

His central thesis is that these systems capture genuine aspects of how humans learn and process language. However, they lack key interpretive and inferential skills. They are unlikely to gain these without integration with structured knowledge and the ability to use it. The argument matters for anyone assessing claims about LLM capabilities.

(Note: the supplied text omits the paper's middle sections. This summary reflects the introduction, the start of Section 2, and the paper outline.)

Key points

  • The author tested chatGPT on 15 January 2023 with three prompts: a transitive inference, an indirect speech act ("Tomorrow, you have an exam"), and an arithmetic problem (10,300 − 4725 = 5575, which is less than 7300). It handled all three fluently.
  • The paper cites the July 2022 case of a Google engineer who was fired after claiming that LaMDA (Thoppilan et al. 2022) had become sentient. Lenci uses it as an extreme example of anthropomorphizing.
  • It contrasts two pretraining objectives:
    • autoregressive/causal models (GPT family), which predict the next word;
    • masked/denoising autoencoding models (BERT), which recover masked tokens using context on both sides, in a Cloze-style task.
  • It uses the "foundation models" framing (Bommasani et al. 2021). Their competence is tied to the Distributional Hypothesis, with meaning represented as embeddings learned from corpus co-occurrences.
  • Conclusion: current systems are not dismissed outright. Closing the gap with human understanding, however, likely requires adding structured knowledge and the means to exploit it in language use.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 works

Cite

Cite as Alessandro Lenci. 2023. Understanding Natural Language Understanding Systems. A Critical Analysis. arXiv:2303.04229.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2303.04229 · arXiv 2303.04229

@misc{lenci2023understanding,
  title = {{Understanding Natural Language Understanding Systems. A Critical Analysis}},
  author = {Alessandro Lenci},
  year = {2023},
  doi = {10.48550/arxiv.2303.04229},
  eprint = {2303.04229},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2303.04229}
}

References

84 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Andreas, J. (2022). Language models as agent models. ArXiv: 2212.01681.
  2. Baroni, M. (2019). Linguistic generalization and compositionality in modern artificial neural networks. Philosophical Transactions of the Royal Society B: Biological Sciences, 375(1791). find
  3. Bender, E. M., Koller, A. (2020). Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of ACL 2020, pp. 5185-5198.
  4. Bender, E. M., Gebru, T., McMillan-Major, A., Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of FAccT 2021. find
  5. Berent, I., Marcus, G. F. (2019). No integration without structured representations: Response to Pater. Language, 95(1), pp. e75-e86. find
  6. Binder, J. R. (2016). In defense of abstract conceptual representations. Psychonomic Bulletin & Review, 23, pp. 1096-1108. find
  7. Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., et al. (2020). Experience grounds language. In Proceedings of EMNLP 2020, pp. 8718–8735.
  8. Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S. et al. (2021). On the opportunities and risks of foundation models. ArXiv: 2108.07258.
  9. Bowman, S. R. (2022). The dangers of underclaiming: Reasons for caution when reporting how NLP systems fail. In Proceedings of ACL 2022, pp. 7484-7499. find
  10. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., et al. (2020). Language models are few-shot learners. ArXiv: 2005.14165.
  11. Calzavarini, F. (2017). Inferential and referential lexical semantic competence: A critical review of the supporting evidence. Journal of Neurolinguistics, 44, pp. 163-189. find
  12. Cohn, N., Jackendoff, R., Holcomb, P. J., Kuperberg, G. R. (2014). The grammar of visual narrative: Neural evidence for constituent structure in sequential image comprehension. Neuropsychologia, 64, pp. 63-70. find
  13. Collins, K. M., Wong, C., Feng, J., Wei, M., Tenenbaum, J. B. (2022). Structured, flexible, and robust: Benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks. In Proceedings of CogSci 2022, pp. 869-875. find
  14. Dale, R. (2021). GPT-3: What’s it good for? Natural Language Engineering, 27(1), pp. 113-118. find
  15. Dasgupta, I., Lampinen, A. K., Chan, S. C. Y., Creswell, A., Kumaran, D., McClelland, J. L., Hill, F. (2022). Language models show human-like content effects on reasoning. ArXiv: 2207.07051.
  16. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT 2019, pp. 4171-4186.
  17. Dove, G. (2009). Beyond perceptual symbols: A call for representational pluralism. Cognition, 110(3), pp. 412-431. find
  18. Dove, G. (2023). Rethinking the role of language in embodied cognition. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 378: 20210375(1870). find
  19. Elkins, K., Chun, J. (2020). Can GPT-3 pass a writer’s Turing Test? Journal of Cultural Analytics, 5(2). find
  20. Enrici, I., Bara, B. G., Adenzato, M. (2019). Theory of Mind, pragmatics and the brain. Pragmatics & Cognition, 26(1), pp. 5-38. find
  21. Floridi, L., Chiriatti, M. (2020). GPT-3: Its nature, scope, limits, and consequences. Minds and Machines, 30(4), pp. 681-694. find
  22. Fodor, J. A., Pylyshyn, Z. A. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28, pp. 3-71. find
  23. Glenberg, A. M., Robertson, D. A. (2000). Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning. Journal of Memory and Language, 43(3), pp. 379-401. find
  24. Goldberg, A. E. (2019). Explain me this. Creativity, competition, and the partial productivity of Constructions. find
  25. Princeton: Princeton University Press. Goldberg, Y. (2017). Neural network methods for natural language processing. find
  26. San Rafael: Morgan & Claypool. Goldstein, A., Zada, Z., Buchnik, E., Schain, M., Price, A. et al. (2022). Shared computational principles for language processing in humans and deep language models. Nature Neuroscience, 25(3), pp. 369-380. find
  27. Goodman, N. D., Frank, M. C. (2016). Pragmatic language interpretation as probabilistic inference. Trends in Cognitive Sciences, 20(11), pp. 818-829. find
  28. Goodwin, E., Sinha, K., O’Donnell, T. J. (2020). Probing linguistic systematicity. In Proceedings of ACL 2020, pp. 1958-1969. find
  29. Gordon, J., Van Durme, B. (2013). Reporting bias and knowledge acquisition. In Proceedings of the 2013 Workshop on Automated Knowledge Base Construction, pp. 25-29. find
  30. Grice, H. P. (1975). Logic and conversation. In Cole, P., Morgan, J. L. (eds), Syntax and semantics, Vol. 3, Speech acts. New York: Academic Press, pp. 41-58. find
  31. Han, X., Zhang, Z., Ding, N., Gu, Y., Liu, X., et al. (2021). Pre-trained models: Past, present, and future. ArXiv: 2106.07139. find
  32. Harman, G. (1982). Conceptual role semantics. Notre Dame Journal of Formal Logic, 23(2), pp. 242-256. find
  33. Harnad, S. (1990). The symbol grounding problem. Physica D, 42, pp. 335-346. find
  34. Helwe, C., Clavel, C., Suchanek, F. (2021). Reasoning with transformer-based models: Deep dearning, but shallow reasoning. In Proceedings of the Workshop on Automated Knowledge Base Construction. find
  35. Hu, J., Floyd, S., Jouravlev, O., Fedorenko, E., Gibson, E. (2022). A fine-grained comparison of pragmatic language understanding in humans and language models. ArXiv: 2212.06801.
  36. Hupkes, D., Dankers, V., Mul, M., Bruni, E. (2020). Compositionality decomposed: How do neural networks generalise? Journal of Artificial Intelligence Research, 67, pp. 757-795.
  37. Johnson, K. (2004). On the systematicity of language and thought. The Journal of Philosophy, 101(3), pp. 111-139. find
  38. Karimi, H., Ferreira, F. (2016). Good-enough linguistic representations and online cognitive equilibrium in language processing. Quarterly Journal of Experimental Psychology, 69(5), pp. 1013-1040. find
  39. Kauf, C., Ivanova, A. A., Rambelli, G., Chersoni, E., She, J. S., Chowdhury, Z., Fedorenko, F., Lenci, A. (2022). Event knowledge in large language models: The gap between the impossible and the unlikely. ArXiv: 2212.01488. find
  40. Lake, B. M., Baroni, M. (2018). Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In Proceedings of ICML 2018, pp. 4487-4499. find
  41. Lake, B. M., Murphy, G. L. (2021). Word meaning in minds and machines. find
  42. Psychological Review. Landauer, T. K., Dumais, S. T. (1997). A solution to Plato’s problem: The Latent Semantic Analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2), pp. 211-240. find
  43. Lenci, A. (2008). Distributional semantics in linguistic and cognitive research. Italian Journal of Linguistics, 20(1), pp. 1-31. find
  44. Lenci, A. (2018). Distributional models of word meaning. Annual Review of Linguistics, 4, pp. 151-171. find
  45. Lenci, A., Sahlgren M. (2023), Distributional semantics. find
  46. Cambridge: Cambridge University Press. Lenci, A., Sahlgren, M., Jeuniaux, P., Cuba Gyllensten, A., Miliani, M. (2022). A comparative evaluation and analysis of three generations of distributional semantic models. Language Resources and Evaluation, 56, pp. 1269-1313. find
  47. Levinson, S. C. (2000). Presumptive meanings. The theory of generalized conversational Implicature. find
  48. Cambridge: The MIT Press. Lewis, M., Zettersten, M., Lupyan, G. (2019). Distributional semantics as a source of visual knowledge. Proceedings of the National Academy of Sciences of the United States of America, 116(39), pp. 19237-19238. find
  49. Louwerse, M. M. (2011). Symbol interdependency in symbolic and embodied cognition. Topics in Cognitive Science, 3(2), pp. 273-302. find
  50. Lupyan, G. (2019). Language as a source of abstract concepts. Physics of Life Reviews, 29, pp. 154-156. find
  51. Lupyan, G., Lewis, M. (2019). From words-as-mappings to words-as-cues: The role of language in semantic knowledge. Language, Cognition and Neuroscience, 34(10), pp. 1319-1337. find
  52. Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., Fedorenko, E. (2023). Dissociating language and thought in large language models: A cognitive perspective. ArXiv: 2301.06627. find
  53. Manning, C. D. (2022). Human language understanding & reasoning. Daedalus, 151(2), pp. 127-138. find
  54. Manning, C. D., Clark, K., Hewitt, J., Khandelwal, U., Levy, O. (2020). Emergent linguistic structure in artificial neural networks trained by self-supervision. Proceedings of the National Academy of Sciences of the United States of America, 117(48), pp. 30046-30054. find
  55. Marconi, D. (1997). Lexical competence. find
  56. Cambridge: The MIT Press. McClelland, J. L., Hill, F., Rudolph, M., Baldridge, J., Schütze, H. (2020). Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models. Proceedings of the National Academy of Sciences of the United States of… find
  57. Mikolov, T., Chen, K., Corrado, G. S., Dean, J. (2013). Efficient estimation of word representations in vector space. In Proceedings of ICLR 2013.
  58. Mikolov, T., Yih, W., Zweig, G. (2013). Linguistic regularities in continuous space word representations. In Proceedings of NAACL-HLT 2013, pp. 746-751.
  59. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., et al. (2022). Training language models to follow instructions with human feedback. ArXiv: 2203.02155.
  60. Paik, C., Aroca-Ouellette, S., Roncone, A., Kann, K. (2021). The world of an octopus: How reporting bias influences a language model’s perception of color. In Proceedings EMNLP 2021, pp. 823-835. find
  61. Pedinotti, P., Rambelli, G., Chersoni, E., Santus, E., Lenci, A., Blache P. (2021). Did the cat drink the coffee? Challenging transformers with generalized event knowledge. In Proceedings *SEM 2021, pp. 1-11. find
  62. Piantadosi, S. T., Hill, F. (2022). Meaning without reference in large language models. In Proceedings of the Workshop on Neuro Causal and Symbolic AI (NCSI) @ NeurIPS. find
  63. Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., Lewis, M. (2022). Measuring and narrowing the compositionality gap in language models. ArXiv: 2210.03350. find
  64. Pustejovsky, J. (1995). The generative lexicon. find
  65. Cambridge: The MIT Press. Rabovsky, M., McClelland, J. L. (2019). Quasi-compositional mapping from form to meaning: A neural network-based approach to capturing neural responses during human language comprehension. Philosophical Transactions of the Royal Society B: Biological Sciences, 375(1791). find
  66. Radford, A., Narasimhan, K., Salimans, T., Sutskever, I. (2018). Improving language understanding by generative pre-training. OpenAI blog. find
  67. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI blog. find
  68. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G. et al. (2021). Learning transferable visual models from natural language supervision. ArXiv: 2103.00020.
  69. Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M. (2022). Hierarchical text-conditional image generation with CLIP latents. ArXiv: 2204.06125.
  70. Rogers, A., Kovaleva, O., Rumshisky, A. (2020). A primer in BERTology: What we know about how BERT works. Transactions of the Association for Computational Linguistics, 8, pp. 842-866.
  71. Rubio-Fernández, P. (2021). Pragmatic markers: the missing link between language and Theory of Mind. Synthese, 199(1–2), pp. 1125-1158. find
  72. Sap, M., LeBras, R., Fried, D., Choi, Y. (2022). Neural Theory-of-Mind? On the limits of social intelligence in large LMs. In Proceedings of EMNLP 2022, pp. 3762-3780. find
  73. Schuster, S., Linzen, T. (2022). When a sentence does not introduce a discourse entity, transformer-based models still sometimes refer to it. In Proceedings of NAACL-HLT, pp. 969-982. find
  74. Searle, J. R. (1980). Minds, brains, and programs. The Behavioral and Brain Sciences, 3, pp. 417-424. find
  75. Shanahan, M. (2022). Talking about large language models. ArXiv: 2212.03551. — [arXiv:2212.03551](https://arxiv.org/abs/2212.03551) find
  76. Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., et al. (2022). Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. ArXiv: 2206.04615.
  77. Strubell, E., Ganesh, A., McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. In Proceedings of ACL 2019, pp. 3645-3650.
  78. Tenney, I., Xia, P., Chen, B., Wang, A., Poliak, A., et al. (2019). What do you learn from context? Probing for sentence structure in contextualized word representations. In Proceedings of ICLR 2019.
  79. Thoppilan, R., de Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., et al. (2022). LaMDA: Language models for dialog applications. ArXiv: 2201.08239.
  80. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., Polosukhin, I. (2017). Attention is all you need. In Proceedings of NIPS 2017.
  81. Vigliocco, G., Meteyard, L., Andrews, M., Kousta, S. T. (2009). Toward a theory of semantic representation. Language and Cognition, 1(2), pp. 219-247. find
  82. Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., et al. (2022). Emergent abilities of large language models. ArXiv: 2206.07682.
  83. Yogatama, D., D’Autume, C. de M., Connor, J., Kocisky, T., Chrzanowski, M., Kong, L., Lazaridou, A., Ling, W., Yu, L., Dyer, C., Blunsom, P. (2019). Learning and evaluating general linguistic intelligence. ArXiv: 1901.11373.
  84. Zhang, C., Van Durme, B., Li, Z., Stengel-Eskin, E. (2022). Visual commonsense in pretrained unimodal and multimodal models. In Proceedings of the NAACL-HLT 2022, pp. 5321-5335. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lenci-2023-understanding-natural-language-understanding-systems/