Elamur Research
☰
home / papers / 2022 / Can language models learn from explanations in context?

Can language models learn from explanations in context?

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2022.findings-emnlp.38.pdf

Summary

Large language models can pick up new tasks from a handful of in-context examples. The authors test whether adding explanations to those examples, as people do when teaching, also helps models. They took 40 difficult tasks and subtasks from BIG-bench. For each, one author wrote post-answer explanations for 15 sampled examples, plus matched controls: scrambled-word explanations, true but non-explanatory statements, and explanations shuffled across items. They also wrote task instructions and non-instructions.

Decoder-only models from 1B to 280B parameters (the Rae et al., 2021 family) were evaluated with 0- and 5-shot prompts. The results were analyzed with hierarchical (multi-level) statistical models that account for nesting across tasks, items, prompts and model sizes.

Even untuned explanations gave a modest gain and beat every control. This suggests that the benefit comes from how each explanation connects to its specific example, rather than from surface features such as prompt length or wording. Explanations tuned on a small validation set helped considerably more. The benefit appeared only in the largest model. This matters both for prompt engineering and for understanding what in-context learning really involves.

Key points

  • Data and annotations: The resource covers 40 BIG-bench tasks and subtasks, such as metaphor_boolean and penguins_in_a_table. Each task has 15 hand-explained examples, and all annotations are released.
  • Prompt design: Explanations come after the answer on a line beginning "Explanation:". The target question is therefore scored identically with or without explanations, unlike chain-of-thought-style reasoning placed before the answer.
  • Controls: Three matched conditions separate explanatory content from low-level features:
    • scrambled explanations;
    • true non-explanations;
    • other-item (permuted) explanations.
  • Main findings:
    • Untuned explanations give small but statistically significant gains over the controls.
    • Hand-tuned explanations give larger gains.
    • Choosing examples and explanations jointly works better than choosing examples alone.
    • Only the 280B model benefits.
  • Limitations:
    • Annotations come from a single author.
    • The task set is small and adversarially sampled.
    • Only one model family with fixed 2048-token contexts was tested.
    • The authors stress that explanations are no panacea.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

148 papers of our library cite it (of the 140673 whose bibliographies we read); 16 of them have a page here, 132 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.findings-emnlp.38 · arXiv 2204.02329

@inproceedings{lampinen2022can,
  title = {{Can language models learn from explanations in context?}},
  author = {Lampinen, Andrew and Dasgupta, Ishita and Chan, Stephanie and Mathewson, Kory and Tessler, Mh and Creswell, Antonia and McClelland, James and Wang, Jane and Hill, Felix},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2022},
  year = {2022},
  doi = {10.18653/v1/2022.findings-emnlp.38},
  eprint = {2204.02329},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2022.findings-emnlp.38}
}

References

67 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… find
  2. Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-snli: Natural language inference with natural language explanations. Advances in Neural Information Processing Systems, 31:9539–9549. find our card
  3. Jörg Cassens, Lorenz Habenicht, Julian Blohm, Rebekah Wegener, Joanna Korman, Sangeet Khemlani, Giorgio Gronchi, Ruth MJ Byrne, Greta Warren, Molly S Quinn, et al. 2021. Explanation in human thinking. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 43. find
  4. Susan Chinn. 2000. A simple method for converting an odds ratio to effect size for use in meta-analysis. Statistics in medicine, 19(22):3127–3131. find
  5. Herbert H Clark. 1973. The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. Journal of verbal learning and verbal behavior, 12(4):335–359. find
  6. Woo-kyoung Ahn, William F Brewer, and Raymond J Mooney. 1992. Schema acquisition from a single example. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(2):391. find
  7. Marina Danilevsky, Kun Qian, Ranit Aharonov, Yannis Katsis, Ban Kawas, and Prithviraj Sen. 2020. A survey of the state of explainable ai for natural language processing. arXiv preprint arXiv:2010.00711. find
  8. Philip W Anderson. 1972. More is different: broken symmetry and the nature of the hierarchical structure of science. Science, 177(4047):393–396. find
  9. Deep Ganguli, Danny Hernandez, Liane Lovitt, Nova DasSarma, Tom Henighan, Andy Jones, Nicholas Joseph, Jackson Kernion, Ben Mann, Amanda Askell, et al. 2022. Predictability and surprise in large generative models. arXiv preprint arXiv:2202.07785. find
  10. Jacob Andreas, Dan Klein, and Sergey Levine. 2018. Learning with latent language. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2166–2179. find
  11. R Harald Baayen, Douglas J Davidson, and Douglas M Bates. 2008. Mixed-effects modeling with crossed random effects for subjects and items. Journal of memory and language, 59(4):390–412. find
  12. Andrew Gelman and Jennifer Hill. 2006. Data analysis using regression and multilevel/hierarchical models. Cambridge university press. find
  13. Ariel Goldstein, Zaid Zada, Eliav Buchnik, Mariano Schain, Amy Price, Bobbi Aubrey, Samuel A Nastase, Amir Feder, Dotan Emanuel, Alon Cohen, et al. 2022. Shared computational principles for language processing in humans and deep language models. Nature Neuroscience, 25(3):369–380. find
  14. Mark Harrower and Cynthia A Brewer. 2003. Col-orbrewer. org: an online tool for selecting colour schemes for maps. The Cartographic Journal, 40(1):27–37. find
  15. Peter Hase and Mohit Bansal. 2021. When can models learn from explanations? a formal framework for understanding the roles of explanation data. arXiv preprint arXiv:2102.02201. find
  16. Lisa Anne Hendricks, Zeynep Akata, Marcus Rohrbach, Jeff Donahue, Bernt Schiele, and Trevor Darrell. 2016. Generating visual explanations. In European conference on computer vision, pages 3–19. Springer. find
  17. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, E… find
  18. Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021. Surface form competition: Why the highest probability answer isn’t always right. arXiv preprint arXiv:2104.08315. find our card
  19. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361. find
  20. Taku Kudo and John Richardson. 2018. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. arXiv preprint arXiv:1808.06226. find our card
  21. Andrew K Lampinen, Nicholas A Roy, Ishita Dasgupta, Stephanie CY Chan, Allison C Tam, James L McClelland, Chen Yan, Adam Santoro, Neil C Rabinowitz, Jane X Wang, et al. 2021. Tell me why!–explanations support learning of relational and causal structure. arXiv preprint arXiv:2112.03753. — [arXiv:2112.03753](https://arxiv.org/abs/2112.03753) find
  22. Teven Le Scao and Alexander M Rush. 2021. How many data points is a prompt worth? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2627–2636. find our card
  23. Piyawat Lertvittayakumjorn and Francesca Toni. 2021. Explanation-based human debugging of nlp models: A survey. arXiv preprint arXiv:2104.15135. find
  24. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017. Program induction by rationale generation: Learning to solve and explain algebraic word problems. arXiv preprint arXiv:1705.04146. find
  25. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586. find our card
  26. Tania Lombrozo. 2006. The structure and function of explanations. Trends in cognitive sciences, 10(10):464– 470. find
  27. Tania Lombrozo and Susan Carey. 2006. Functional explanation and the function of explanation. Cognition, 99(2):167–204. find
  28. Bodhisattwa Prasad Majumder, Oana-Maria Camburu, Thomas Lukasiewicz, and Julian McAuley. 2021. Rationale-inspired natural language explanations with commonsense. arXiv preprint arXiv:2106.13876. — [arXiv:2106.13876](https://arxiv.org/abs/2106.13876) find
  29. Ana Marasović, Iz Beltagy, Doug Downey, and Matthew E Peters. 2021. Few-shot self-rationalization with natural language prompts. arXiv preprint arXiv:2111.08284. find
  30. James L McClelland, Felix Hill, Maja Rudolph, Jason Baldridge, and Hinrich Schütze. 2020. Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models. Proceedings of the National Academy of Sciences, 117(42):25966–25974. find
  31. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? find our card
  32. Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, and Hannaneh Hajishirzi. 2021a. Reframing instructional prompts to gptk’s language. arXiv preprint arXiv:2109.07830. find our card
  33. Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2021b. Cross-task generalization via natural language crowdsourcing instructions. arXiv preprint arXiv:2104.08773. find
  34. Melanie Mitchell. 2021. Why ai is harder than we think. arXiv preprint arXiv:2104.12871. — [arXiv:2104.12871](https://arxiv.org/abs/2104.12871) find
  35. Jesse Mu, Percy Liang, and Noah Goodman. 2020. Shaping visual representations with language for few-shot classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4823–4830, Online. Association for Computational Linguistics. find
  36. Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, and Karishma Malkan. 2020. Wt5?! training text-to-text models to explain their predictions. arXiv preprint arXiv:2004.14546. find
  37. Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. 2021. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114. find our card
  38. Joe O’Connor and Jacob Andreas. 2021. What context features can transformer language models use? arXiv preprint arXiv:2106.08367. find
  39. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Preprint. find
  40. Isabel Papadimitriou, Richard Futrell, and Kyle Mahowald. 2022. When classifying arguments, bert doesn’t care about word order... except when it matters. Proceedings of the Society for Computation in Linguistics, 5(1):203–205. find
  41. Ethan Perez, Douwe Kiela, and Kyunghyun Cho. 2021. True few-shot learning with language models. Advances in Neural Information Processing Systems, 34. find our card
  42. Thang M Pham, Trung Bui, Long Mai, and Anh Nguyen. 2020. Out of order: How important is the sequential order of words in a sentence in natural language understanding tasks? arXiv preprint arXiv:2012.15180. find
  43. Ofir Press, Noah A Smith, and Mike Lewis. 2021. Train short, test long: Attention with linear biases enables input length extrapolation. arXiv preprint arXiv:2108.12409. find
  44. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. find
  45. Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. 2021. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446. find
  46. Jack W Rae, Anna Potapenko, Siddhant M Jayakumar, and Timothy P Lillicrap. 2019. Compressive transformers for long-range sequence modelling. arXiv preprint arXiv:1911.05507. find our card
  47. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21:1– 67. find
  48. Laria Reynolds and Kyle McDonell. 2021. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–7. find our card
  49. Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. 2021. Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207. find our card
  50. Adam Santoro, Andrew Lampinen, Kory Mathewson, Timothy Lillicrap, and David Raposo. 2021. Symbolic behaviour in artificial intelligence. arXiv preprint arXiv:2102.03406. — [arXiv:2102.03406](https://arxiv.org/abs/2102.03406) find
  51. Patrick Schramowski, Wolfgang Stammer, Stefano Teso, Anna Brugger, Franziska Herbert, Xiaoting Shao, Hans-Georg Luigs, Anne-Katrin Mahlein, and Kristian Kersting. 2020. Making deep neural networks right for the right scientific reasons by interacting with their explanations. Nature Machine Intellige… find
  52. Martin Schrimpf, Idan Asher Blank, Greta Tuckute, Carina Kauf, Eghbal A Hosseini, Nancy Kanwisher, Joshua B Tenenbaum, and Evelina Fedorenko. 2021. The neural architecture of language: Integrative modeling converges on predictive processing. Proceedings of the National Academy of Sciences, 118(45). find
  53. Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Unsupervised commonsense question answering with self-talk. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4615–4629. find
  54. Bas Van Fraassen. 1988. The pragmatic theory of explanation. Theories of Explanation, 8:135–155. find
  55. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. find
  56. Albert Webson and Ellie Pavlick. 2021. Do prompt-based models really understand the meaning of their prompts? arXiv preprint arXiv:2109.01247. find our card
  57. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2021. Finetuned language models are zero-shot learners. find
  58. Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain yourself! leveraging language models for commonsense reasoning. arXiv preprint arXiv:1906.02361. find our card
  59. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903. find our card
  60. Gabriel Recchia. 2021. Teaching autoregressive language models complex tasks by demonstration. arXiv preprint arXiv:2109.02102. — [arXiv:2109.02102](https://arxiv.org/abs/2109.02102) find
  61. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. 2021. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359. find our card
  62. Hadley Wickham, Mara Averick, Jennifer Bryan, Winston Chang, Lucy D’Agostino McGowan, Romain François, Garrett Grolemund, Alex Hayes, Lionel Henry, Jim Hester, Max Kuhn, Thomas Lin Pedersen, Evan Miller, Stephan Milton Bache, Kirill Müller, Jeroen Ooms, David Robinson, Dana Paige Seidel, Vitalie Spi… find
  63. Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, and Yejin Choi. 2021. Reframing human-AI collaboration for generating free-text explanations. In NAACL. find
  64. Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. 2021. An explanation of in-context learning as implicit bayesian inference. arXiv preprint arXiv:2111.02080. find our card
  65. Tal Yarkoni. 2022. The generalizability crisis. Behavioral and Brain Sciences, 45. find
  66. Eric Zelikman, Yuhuai Wu, and Noah D. Goodman. 2022. STaR: Bootstrapping reasoning with reasoning. find our card
  67. Wangchunshu Zhou, Jinyi Hu, Hanlin Zhang, Xiaodan Liang, Maosong Sun, Chenyan Xiong, and Jian Tang. 2020. Towards interpretable natural language understanding with explanations as latent variables. Advances in Neural Information Processing Systems, 33:6803–6814. find