Elamur Research⌕
☰
home / papers / 2024 / Exceptions, Instantiations, and Overgeneralization: Insights into How Language Models Process Generics

Exceptions, Instantiations, and Overgeneralization: Insights into How Language Models Process Generics

Paper · 2024 · Emily Allaway, Chandra Bhagavatula, Jena D. Hwang et al.

Summary

The paper investigates how large language models (LLMs) reason about generics. Generics are generalizations without an explicit quantifier, such as "birds fly." They remain acceptable despite exceptions (penguins) and extend to specific cases (sparrows). The authors first propose a theoretical framework, GenerIX. It builds on the pragmatic notion of focus and provides logical definitions of several readings of a generic, as well as of its exceptions and instantiations (collectively called exemplars). These definitions are implemented in the ExempliFI system, which automatically generates such exemplars using LLMs. This yields a dataset of approximately 370 thousand exemplars for about 17 thousand generics, part of which was manually verified by humans. The dataset was then used to test two phenomena known from human psychology: overgeneralization (GOG) and property inheritance. The models tend to overgeneralize, as humans do, and exceptions do not fully eliminate this tendency. The models do draw inferences from generics, but when contradictory information appears, they revise them inconsistently. This matters because most natural language reasoning systems are built on LLMs.

Key points

  • Two resources: the theoretical framework GenerIX, which determines what counts as an exception depending on focus: in "BIRDS fly," the exceptions are other flyers, such as bats; and the generator ExempliFI, which works with LLMs.
  • Dataset: ≈370K exemplars for ≈17K generics. They include both knowledge-based examples ("ostriches do not fly") and reasoning-based examples (a bird with a broken wing). Quality was verified by humans on a sample.
  • The generics are taken from the sets AnimalG, GGTest and GGSmall. For comparison, the authors present a simple intermediate baseline on GPT-3.5-Turbo. Its exceptions rely mostly on knowledge rather than on reasoning.
  • The GOG effect (Generic OverGeneralization) consists in treating a statement with "all" as a generic compatible with exceptions. In humans who have just thought about an exception, it drops to about 20% of accepted responses, while without such a prompt it is about 50%. This effect was also found in LLMs, and exceptions do not remove it.
  • Property inheritance: the models infer from a generic to an individual member, but after a counterexample they adjust this inference less consistently than humans do.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom396 works · 82,279 citations🔬Allen Institute for Artificial IntelligenceSeattle · United States317 works · 106,633 citations🎓Columbia UniversityNew York · United States263 works · 57,820 citations🎓Princeton UniversityPrinceton · United States137 works · 53,849 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 4,145 works

Cite

Cite as Emily Allaway, Chandra Bhagavatula, Jena D. Hwang, Kathleen McKeown and Sarah-Jane Leslie. 2024. Exceptions, Instantiations, and Overgeneralization: Insights into How Language Models Process Generics. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00530

@article{allaway2024exceptions,
  title = {{Exceptions, Instantiations, and Overgeneralization: Insights into How Language Models Process Generics}},
  author = {Emily Allaway, Chandra Bhagavatula, Jena D. Hwang, Kathleen McKeown and Sarah-Jane Leslie},
  journal = {Computational Linguistics},
  year = {2024},
  doi = {10.1162/coli\_a\_00530},
  url = {https://doi.org/10.1162/coli_a_00530}
}

References

49 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Advances in Neural Information Processing Systems, 30:1–9. Clark, Kevin, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. In ICLR, pages 1–18. find
  2. Cohen, Ariel. 1996. Think generic! The Meaning and Use of Generic Sentences. find
  3. Carnegie Mellon University. Cohen, Ariel. 1999. Generics, frequency adverbs, and probability. Linguistics and Philosophy, 22(3):221–253. https://doi.org/10.1023/A:1005497727784 Cohen, Ariel. 2004. Generics and mental representations. Linguistics and Philosophy, 27(5):529–556. https://doi.org /10.10… ↗
  4. Ehrlich, Paul R., David S. Dobkin, and Darryl Wheye. 1988. How fast and high do birds fly? https://web.stanford.edu/group/stanfordbirds/text/essays/How Fast .html Elio, Renée and Francis Jeffry Pelletier. 1996. On reasoning with default rules and exceptions. In Proceedings of the 18th Conference o… ↗
  5. Fellbaum, Christiane D. 2000. Wordnet: An electronic lexical database. Language, 76:706. https://doi.org/10.2307/417141 — [doi:10.2307/417141](https://doi.org/10.2307/417141) ↗
  6. Friedrich, Annemarie, Alexis Palmer, and Manfred Pinkal. 2016. Situation entity types: Automatic classification of clause-level aspect. In Annual Meeting of the Association for Computational Linguistics, pages 1757–1768. https://doi.org/10 .18653/v1/P16-1166 Friedrich, Annemarie, Alexis Palmer, Meli… find
  7. Allaway, Emily, Jena D. Hwang, Chandra Bhagavatula, Kathleen McKeown, Doug Downey, and Yejin Choi. 2023. Penguins don’t fly: Reasoning about generics through instantiations and exceptions. In Conference of the European Chapter of the Association for Computational Linguistics, pages 2618–2635. https:…
  8. Asher, Nicholas and Michael Morreau. 1995. What some generic sentences mean. The Generic Book, pages 300–338. find
  9. Bhagavatula, Chandra, Jena D. Hwang, Doug Downey, Ronan Le Bras, Ximing Lu, Keisuke Sakaguchi, Swabha Swayamdipta, Peter West, and Yejin Choi. 2022. I2d2: Inductive knowledge distillation with neurologic and self-imitation. In Annual Meeting of the Association for Computational Linguistics, pages 96… find
  10. Brewka, Gerhard. 1987. The logic of Inheritance in frame systems. In IJCAI, pages 483–488. find
  11. Brown, Tom B., Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165.
  12. Carlson, Greg N. 1977. Reference to Kinds in English. Ph.D. thesis, University of Massachusetts, Amherst. find
  13. Carlson, Greg N. 1989. On the semantic composition of English generic sentences. In Gennaro Chierchia, Barbara H. Partee, and Raymond Turner, editors, Properties, Types and Meaning, Vol. II. Semantic Issues. Dordrecht: Kluwer, pages 167–192. https://doi.org/10.1007/978-94-009 -2723-0 5 CDC. 2022. Ab… — [doi:10.1007/978-94-009](https://doi.org/10.1007/978-94-009) ↗
  14. Christiano, Paul F., Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. find
  15. Friedrich, Annemarie and Manfred Pinkal. 2015. Discourse-sensitive automatic identification of generic expressions. In Annual Meeting of the Association for Computational Linguistics, pages 1272–1281. https://doi.org/10 .3115/v1/P15-1123 Gelman, Susan A., Ingrid Sanchez Tapia, and Sarah-Jane Leslie.… find
  16. Govindarajan, Venkata S, Benjamin Van Durme, and Aaron Steven White. 2019. Decomposing generalization: Models of generic, habitual, and episodic statements. Transactions of the Association for Computational Linguistics, 7:501–517. https://doi.org/10.1162/tacl_a_00285 ↗
  17. Greenberg, Yael. 2007. Exceptions to generics: Where vagueness, context dependence and modality interact. Journal of Semantics, 24(2):131–167. https://doi.org/10.1093/jos/ffm002 Grice, Herbert P. 1975. Logic and conversation. In Speech Acts. Brill, pages 41–58. https://doi.org/10.1163 /978900436881… — [doi:10.1093/jos/ffm002](https://doi.org/10.1093/jos/ffm002) ↗
  18. Hamblin, C. L. 1973. Questions in Montague English. Foundations of Language, 10(1):41–53. find
  19. Hanks, Steve and Drew McDermott. 1986. Default reasoning, nonmonotonic logics, and the frame problem. In AAAI, pages 328–333. find
  20. Haward, Paul, Laura Wagner, Susan Carey, and Sandeep Prasada. 2018. The development of principled connections and kind representations. Cognition, 176:255–268. https://doi.org/10.1016/j.cognition.2018.02.001, PubMed: 29609100 Hoes, Emma, Sacha Altay, and Juan Bermeo. 2023. Leveraging ChatGPT for ef… — [doi:10.31234/osf.io/qnjkf](https://doi.org/10.31234/osf.io/qnjkf) ↗
  21. Hu, Jennifer and Roger Levy. 2023. Prompting is not a substitute for probability measurements in large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5040–5060. https://doi.org/10.18653/v1/2023.emnlp-main.306 Jackendoff, Ray S. 1972… ↗
  22. MIT Press. Kadmon, Nirit. 2001. find
  23. Formal Pragmatics: Semantics, Pragmatics, Presupposition, and Focus. Wiley-Blackwell. Kadmon, Nirit and Fred Landman. 1993. Any. Linguistics and Philosophy, 16(4):353–422. https://doi.org/10 .1007/BF00985272 Karczewski, Daniel, Edyta Wajda, and Radosław Poniat. 2020. Do all storks fly to Africa? Uni… — Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly
  24. Khemlani, Sangeet, Sarah-Jane Leslie, and Sam Glucksberg. 2008. Syllogistic reasoning with generic premises: The generic overgeneralization effect. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 30, pages 1–6. https://doi.org/10.1037/e527312012-078 Khemlani, Sangeet,… ↗
  25. Kochari, Arnold, Robert Van Rooij, and Katrin Schulz. 2020. Generics and alternatives. Frontiers in Psychology, 11:1274. https://doi.org/10.3389/fpsyg.2020.01274 Krifka, Manfred. 1987. An outline of genericity. In Seminar für natürlich-sprachliche Systeme der Universität Tübingen. ↗
  26. Krifka, Manfred. 2008. Basic notions of information structure. Acta Linguistica Hungarica, 55(3-4):243 –276. https://doi.org/10.1556/aling.55.2008.3-4.2 Krifka, Manfred, Francis Jeffry Pelletier, Gregory Carlson, Alice Ter Meulen, Gennaro Chierchia, and Godehard Link. 1995. Genericity: An introduct… ↗
  27. Krifka, Manfred, et al. 2012. Definitional generics. Genericity, pages 372–389. https://doi.org/10.1093/acprof:oso/9780199691807.003.0015 Lampinen, Andrew Kyle. 2023. Can language models handle recursively nested grammatical structures? A case study on comparing models and humans. arXiv preprint ar… — [doi:10.1093/acprof:oso](https://doi.org/10.1093/acprof:oso) ↗
  28. Leshin, Rachel A., Sarah-Jane Leslie, and Marjorie Rhodes. 2021. Does it matter how we speak about social kinds? A large, preregistered, online experimental study of how language shapes the development of essentialist beliefs. Child Development, 92(4):531–547. https://doi.org/10 .1111/cdev.13527, Pu… — [doi:10.1111/j.1520-8583.2007.00138.x](https://doi.org/10.1111/j.1520-8583.2007.00138.x) find
  29. Metaphysics Research Lab, Stanford University. Lewis, David. 1975. Adverbs of quantification. In Edward L. Keenan, editor, Formal Semantics of Natural Language. Cambridge University Press, pages 3–15. https://doi.org/10.1017/CBO9780511897696.003 Lifschitz, Vladimir. 1989. Benchmark problems for non… ↗
  30. Lu, Ximing, Sean Welleck, Peter West, Liwei Jiang, Jungo Kasai, Daniel Khashabi, Ronan Le Bras, Lianhui Qin, Youngjae Yu, Rowan Zellers, Noah A. Smith, and Yejin Choi. 2022. Neurologic A*esque decoding: Constrained text generation with lookahead heuristics. In NAACL, pages 780–799. https://doi.org /… ↗
  31. McCarthy, John. 1986. Applications of circumscription to formalizing common-sense knowledge. Artificial Intelligence, 28(1):89–116. https://doi .org/10.1016/0004-3702(86)90032-9 Meyer, Meredith, Susan A. Gelman, and Sarah M. Stilwell. 2011. Generics are a cognitive default: Evidence from sentence pr… find
  32. Misra, Kanishka, Allyson Ettinger, and Kyle Mahowald. 2024. Experimental contexts can facilitate robust semantic property inference in language models, but inconsistently. arXiv preprint arXiv:2401.06640. find
  33. Misra, Kanishka, Allyson Ettinger, and Julia Taylor Rayz. 2021. Do language models learn typicality judgments from text? ArXiv, abs/2105.02987. — [arXiv:2105.02987](https://arxiv.org/abs/2105.02987) find
  34. Mun, Jimin, Emily Allaway, Akhila Yerukola, Laura Vianna, Sarah-Jane Leslie, and Maarten Sap. 2023. Beyond denouncing hate: Strategies for countering implied biases and stereotypes in language. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Findings, pages… find
  35. OpenAI. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  36. Ouyang, Long, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, Jan Leike, and Ryan J. Lo… — Training language models to follow instructions with human feedback
  37. Partee, Barbara. 1991. Topic, focus and quantification. In Semantics and Linguistic Theory, volume 1, pages 159–188. https://doi.org/10.3765/salt .v1i0.2918 Pelletier, Francis Jeffry and Renée Elio. 2005. The case for psychologism in default and inheritance reasoning. Synthese, 146(1):7–35. https:/… — [doi:10.3765/salt](https://doi.org/10.3765/salt) ↗
  38. Ralethe, Sello and Jan Buys. 2022. Generic overgeneralization in pre-trained language models. In International Conference on Computational Linguistics, pages 3187–3196. find
  39. Reiter, Nils and Anette Frank. 2010. Identifying generic noun phrases. In Annual Meeting of the Association for Computational Linguistics. find
  40. Reiter, R. 1978. On reasoning by default. In Proceedings of TINLAP-2, pages 210–218. https://doi.org/10.3115/980262 .980297 Reiter, Raymond. 1980. A logic for default reasoning. Artificial Intelligence, 13:81–132. https://doi.org/10.1016/0004 -3702(80)90014-4 Rhodes, Marjorie, Sarah-Jane Leslie, and… ↗
  41. Rohrbach, Anna, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035–4045. https://doi.org/10.18653/v1/D18-1437 van Rooij, Robert and K… ↗
  42. Sidner, Candace Lee. 1979. Towards a Computational Theory of Definite Anaphora Comprehension in English Discourse. Ph.D. thesis, Massachusetts Institute of Technology. find
  43. Speer, Robyn, Joshua Chin, and Catherine Havasi. 2017. ConceptNet 5.5: An open multilingual graph of general knowledge. In Thirty-first AAAI Conference on Artificial Intelligence, pages 4444–4451. https://doi.org/10.1609/aaai.v31i1.11164 Sridharan, Mohan, Michael Gelfond, Shiqi Zhang, and Jeremy L.… ↗
  44. Subbiah, Melanie, Sean Zhang, Lydia B. Chilton, and Kathleen McKeown. 2024. find
  45. Reading subtext: Evaluating large language models on short story summarization with writers. arXiv preprint arXiv:2403.01061. https://doi.org/10.1162/tacl a 00702 Suh, Sangweon. 2006. Extracting generic statements for the semantic web. Master’s thesis, University of Edinburgh. — [doi:10.1162/tacl](https://doi.org/10.1162/tacl) ↗
  46. Sutherland, Shelbie L. Andrei Cimpian, Sarah-Jane Leslie, and Susan A. Gelman. 2015. Memory errors reveal a bias to spontaneously generalize to categories. Cognitive Science, 39(5):1021–1046. https://doi.org/10.1111/cogs.12189 — [doi:10.1111/cogs.12189](https://doi.org/10.1111/cogs.12189) ↗
  47. Talmor, Alon, Oyvind Tafjord, Peter Clark, Yoav Goldberg, and Jonathan Berant. 2020. Leap-of-thought: Teaching pre-trained models to systematically reason over implicit knowledge. Advances in Neural Information Processing Systems, 33:20227–20237.
  48. Tardif, Twila, Susan A. Gelman, Xiaolan Fu, and Liqi Zhu. 2012. Acquisition of generic noun phrases in Chinese: Learning about lions without an ‘-s’. Journal of Child Language, 39(1):130–161. https://doi.org/10.1017/S0305000910000735 Tessler, Michael Henry and Noah D. Goodman. 2019. The language of… ↗
  49. Wei, Jason, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V. Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/allaway-2024-exceptions-instantiations-overgeneralization-insights-into/