Elamur Research⌕
☰
home / papers / 2021 / Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?

Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?

Paper · 2021 · William Merrill, Yoav Goldberg, Roy Schwartz et al.

Summary

Can a system trained only on raw text, with no grounding, ever acquire meaning? Merrill, Goldberg, Schwartz and Smith turn the informal debate started by Bender and Koller (2020) into a precise mathematical question. One counterargument in that debate holds that unit-test assertions, which check whether two expressions are equal, provide indirect grounding. The authors model this as an oracle that reports whether two expressions share a value in a given context. A learner "understands" a language if it can build representations that preserve semantic equivalence using these queries (ℵ-emulation). The work is purely theoretical, with no corpora or trained models. The authors prove that emulation is possible when every expression keeps a fixed denotation across all contexts (strong transparency). Once meaning can shift with context, as with bound variables in code, they construct a class of languages where emulation becomes uncomputable. This gives formal support to the view that assertion-like signals in code or text are not enough for full semantic understanding by future language models.

Key points

  • Framework: Formal languages carry contextual denotations over syntactic contexts ⟨l, r⟩. The assertion oracle ℵ_L returns 1 exactly when two expressions denote the same referent in a context. It is motivated by Python-style assert e1 == e2 and by Grice's bias toward truthful statements.
  • Positive result (Section 4): Strongly transparent languages, such as simple integer arithmetic, can be emulated with a finite number of assertion queries.
  • Negative result (Section 5): For a constructed class of languages where values depend on context (variable binding), emulation would require infinitely many queries. It is therefore impossible even for a Turing-complete emulator.
  • Scope and extensions (Section 6, Appendix B): The results are discussed in a modal setting and for relations beyond equality, such as entailment. The notion of emulation is strict and tolerates no performance errors, in the sense of Chomsky's competence vs. performance distinction.
  • Publication and open questions: The paper appeared in TACL vol. 9, pp. 1047–1060 (2021), under CC-BY 4.0, with authors from the Allen Institute for AI, Bar Ilan, Hebrew University and UW. Open questions include dialogue settings (the "octopus" scenario), multimodal oracles and probabilistic emulation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of WashingtonSeattle · United States263 works · 180,581 citations🔬Allen Institute for Artificial IntelligenceSeattle · United States219 works · 89,712 citations🎓Bar-Ilan UniversityRamat Gan · Israel118 works · 21,077 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 1,344 works

Cite

Cite as William Merrill, Yoav Goldberg, Roy Schwartz and Noah A. Smith. 2021. Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00412

@article{merrill2021provable,
  title = {{Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?}},
  author = {William Merrill, Yoav Goldberg, Roy Schwartz and Noah A. Smith},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.1162/tacl\_a\_00412},
  url = {https://doi.org/10.1162/tacl_a_00412}
}

References

30 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. Open…
  2. Dana Angluin. 1987. Learning regular sets from queries and counterexamples. Information and Computation, 75(2):87–106. https://doi.org/10.1016/0890-5401(87) 90052-6 ↗
  3. Yonatan Belinkov and James Glass. 2019. Analysis methods in neural language processing: A survey. Transactions of the Association for Computational Linguistics, 7:49–72. https://doi.org/10.1162/tacl a 00254 ↗
  4. Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185–5198, Online. Association for Computational Linguistics. https://doi.org /1… ↗
  5. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  6. Noam Chomsky. 1965. Aspects of the Theory of Syntax, volume 11. MIT Press. edition. Metaphysics Research Lab, Stanford University. find
  7. Alexander Clark. 2010. Three learnable models for the description of language. In Language and Automata Theory and Applications, pages 16–31, Berlin, Heidelberg. Springer Berlin Heidelberg. https://doi.org/10 .1007/978-3-642-13089-2 2 find
  8. Dieuwke Hupkes and Willem Zuidema. 2018. Visualisation and ’diagnostic classifiers’ reveal how recurrent and recursive neural networks process hierarchical structure (extended abstract). In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pages 5… — [doi:10.24963/ijcai.2018/796](https://doi.org/10.24963/ijcai.2018/796) find
  9. Alexis Conneau, German Kruszewski, Guillaume Lample, Loı̈c Barrault, and Marco Baroni. 2018. What you can cram into a single $&!\#* vector: Probing sentence embeddings for linguistic properties. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long…
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  11. Herbert P Grice. 1975. Logic and conversation. In Speech Acts, pages 41–58. Brill. https://doi.org/10.1163/9789004368811 003 ↗
  12. Zellig S. Harris. 1954. Distributional structure. WORD, 10(2–3):146–162. find
  13. Irene Heim and Angelika Kratzer. 1998. Semantics in Generative Grammar. Blackwell. find
  14. John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2733–2743, Hong Kong, Chi… find
  15. Laurence R. Horn and Heinrich Wansing. 2020. Negation. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy, spring 2020 find
  16. Hector Levesque. 2014. On our best behaviour. Artificial Intelligence, 212. https://doi.org/10.1016/j.artint.2014.03.007 ↗
  17. Julian Michael. 2020. To dissect an octopus: Making sense of the form/meaning debate. find
  18. Christopher Potts. 2020. Is it possible for language models to achieve understanding? https://doi.org/10.1305/ndjfl/1153858644 — [doi:10.1305/ndjfl/1153858644](https://doi.org/10.1305/ndjfl/1153858644) ↗
  19. Ian Pratt-Hartmann and Allan Third. 2006. More fragments of language. Notre Dame Journal of Formal Logic, 47(2):151–177. find
  20. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. The arXiv: 1910.10683.
  21. Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. A primer in BERTology: What we know about how BERT works. arXiv: 2002.12327. https://doi.org/10.1162/tacl a 00349 ↗
  22. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4593–4601, Florence, Italy. Association for Computational Linguistics. https://doi.org/10.18653/v1/P19-145… ↗
  23. Alan M. Turing. 1936. On computable numbers, with an application to the Entscheidungsproblem. Journal of Math, 58(345–363):5. find
  24. Alan M. Turing. 1950. Computing machinery and intelligence. Mind, LIX(236):433–460. https://doi.org/10.1093/mind/LIX.236 .433 ↗
  25. Kai Von Fintel and Irene Heim. 2011. Intensional semantics. Unpublished Lecture Notes. find
  26. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355,…
  27. Alfred North Whitehead and Bertrand Russell. 1925–1927. Principia Mathematica, Cambridge University Press. find
  28. Yoad Winter. 2016. Elements of Formal Semantics: An Introduction to the Mathematical Theory of Meaning in Natural Language. Edinburgh University Press. find
  29. Dani Yogatama, Cyprien de Masson d’Autume, Jerome Connor, Tomas Kocisky, Mike Chrzanowski, Lingpeng Kong, Angeliki Lazaridou, Wang Ling, Lei Yu, Chris Dyer, and Phil Blunsom. 2019. Learning and evaluating general linguistic intelligence. arXiv: 1901 .11373. find
  30. Hongming Zhang, Xinran Zhao, and Yangqiu Song. 2020. WinoWhy: A deep diagnosis of essential commonsense knowledge for answering Winograd schema challenge. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5736–5745, Online. Association for Computationa… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/merrill-2021-provable-limitations-acquiring-meaning-ungrounded/