Elamur Research
☰
home / papers / 2023 / Larger language models do in-context learning differently

Larger language models do in-context learning differently

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2303.03846

Summary

In-context learning (ICL) can work in two ways. A model can lean on what it learned in pretraining about what the label words mean, or it can learn the input–label pattern shown in the prompt. This paper separates the two. The authors use two altered prompting setups. In the first, the labels of the demonstration examples are inverted. In the second, called SUL-ICL, the labels are swapped for meaningless tokens such as Foo/Bar. They test GPT-3, InstructGPT, Codex, PaLM and Flan-PaLM at several sizes on seven classification datasets. Small models mostly ignore inverted labels and keep using their pretrained knowledge. The largest models follow the contradicting examples, even when that pushes accuracy well below chance, and this ability appears only at scale. Learning from arbitrary label tokens also mainly appears in big models, which can even do high-dimensional linear classification in context. Instruction tuning helps models learn from arbitrary labels but makes them hold on more tightly to their prior knowledge. These findings help explain when a few-shot prompt actually teaches a model something new.

Key points

  • Tasks: SST-2, SUBJ, TREC, QQP, RTE, Financial Phrasebank (FP) and ETHOS. The default setup uses 16 demonstrations per class and 100 randomly drawn test items per dataset.
  • Effect of inverted labels: text-davinci-002 falls from 90.3% accuracy with correct labels to 22.5% when every label is inverted. GPT-3 models (ada through davinci, roughly 350M–175B) only drop to chance level, so the authors group them with the "small" models.
  • Scale needed for arbitrary labels: with Foo/Bar labels, small models lose a lot of accuracy compared with ordinary ICL. On some datasets, only PaLM-540B does better than random.
  • PaLM sizes: PaLM comes in 8B, 62B and 540B versions trained the same way, so it isolates the effect of model size alone.
  • Instruction tuning: Flan-PaLM does better than PaLM with arbitrary labels but is less willing to follow inverted ones. Instruction tuning strengthens reliance on prior knowledge more than it improves learning from the examples.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing · AI & machine learning

Cited by

177 papers of our library cite it (of the 140673 whose bibliographies we read); 15 of them have a page here, 162 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2303.03846 · arXiv 2303.03846

@misc{wei2023larger,
  title = {{Larger language models do in-context learning differently}},
  author = {Wei, Jerry and Wei, Jason and Tay, Yi and Tran, Dustin and Webson, Albert and Lu, Yifeng and Chen, Xinyun and Liu, Hanxiao and Huang, Da and Zhou, Denny and Ma, Tengyu},
  year = {2023},
  doi = {10.48550/arxiv.2303.03846},
  eprint = {2303.03846},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2303.03846}
}

References

44 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. What learning algorithm is in-context learning? Investigations with linear models. In International Conference on Learning Representations (ICLR), 2023. URL https://arxiv.org/abs/2211.15661. ↗ our card
  2. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Conference on Neural Information Processing Systems (NeurIPS), 2020. URL https://papers.nips.cc/pap… ↗
  3. Stephanie C.Y. Chan, Adam Santoro, Andrew K. Lampinen, Jane X. Wang, Aaditya Singh, Pierre H. Richemond, Jay McClelland, and Felix Hill. Data distributional properties drive emergent few-shot learning in transformers. Conference on Neural Information Processing Systems (NeurIPS), 2022. URL https://a… find
  4. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. URL https:// arxiv.org/abs/2107.03374. find our card
  5. Zihang Chen, Hongbo Zhang, Xiaoji Zhang, and Leqi Zhao. Quora question pairs, 2017. URL https://www.kaggle.com/c/quora-question-pairs. ↗
  6. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, et al. PaLM: Scaling language modeling with Pathways. arXiv preprint arXiv:2204.02311, 2022. URL https://arxiv. org/abs/2204.02311. find
  7. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022. URL https://arxiv.org/abs/2210.11416. ↗
  8. Alexis Conneau and Douwe Kiela. SentEval: An evaluation toolkit for universal sentence representations. Language Resources and Evaluation Conference (LREC), 2018. URL http: //arxiv.org/abs/1803.05449. find our card
  9. Ido Dagan, Oren Glickman, and Bernardo Magnini. The PASCAL recognising textual entailment challenge. In First PASCAL Machine Learning Challenges Workshop, 2006. URL https://www.researchgate.net/publication/221366753\_The\_ PASCAL\_recognising\_textual\_entailment\_challenge. ↗
  10. Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Zhifang Sui, and Furu Wei. Why can GPT learn in-context? Language models secretly perform gradient descent as meta-optimizers, 2022. URL https://arxiv.org/abs/2212.10559. ↗
  11. Deep Ganguli, Danny Hernandez, Liane Lovitt, Nova DasSarma, Tom Henighan, Andy Jones, Nicholas Joseph, Jackson Kernion, Ben Mann, Amanda Askell, et al. Predictability and surprise in large generative models. In 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2022. URL http… find
  12. Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, Jason Phang, Laria Reynolds, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, 2021. URL… — [doi:10.5281/zenodo.5371628](https://doi.org/10.5281/zenodo.5371628) find
  13. Shivam Garg, Dimitris Tsipras, Percy Liang, and Gregory Valiant. What can transformers learn in-context? A case study of simple function classes, 2022. URL https://arxiv.org/abs/ 2208.01066. ↗
  14. Yoon Kim. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2014. URL https://aclanthology.org/D14-1181. ↗ our card
  15. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Conference on Neural Information Processing Systems (NeurIPS), 2022. URL https://arxiv.org/abs/2205.11916. ↗ our card
  16. Hector Levesque, Ernest Davis, and Leora Morgenstern. The winograd schema challenge. In Thirteenth international conference on the principles of knowledge representation and reasoning (KR), 2012. URL http://commonsensereasoning.org/2011/papers/Levesque. pdf. ↗
  17. Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario Šaško, Gunjan Chhablani, Bhavitvya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Canwen Xu, Nicolas Patry,… find our card
  18. Xin Li and Dan Roth. Learning question classifiers. In The 19th International Conference on Computational Linguistics (COLING), 2002. URL https://www.aclweb.org/anthology/ C02-1150. ↗
  19. Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. The Flan collection: Designing data and methods for effective instruction tuning. arXiv preprint arXiv:2301.13688, 2023. URL https://arxiv.org/abs/ 2301.13688. ↗ our card
  20. Aman Madaan and Amir Yazdanbakhsh. Text and patterns: For effective chain of thought, it takes two to tango. arXiv preprint arXiv:2209.07686, 2022. URL https://arxiv.org/abs/ 2209.07686. ↗
  21. P. Malo, A. Sinha, P. Korhonen, J. Wallenius, and P. Takala. Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology (JASIST), 2014. URL https://arxiv.org/abs/1307.5336. — [arXiv:1307.5336](https://arxiv.org/abs/1307.5336) ↗
  22. Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. MetaICL: Learning to learn in context. Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2022a. URL https://arxiv.org/abs/2110.15943. ↗
  23. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022b. URL https://arxiv.org/abs/2202.12837… ↗ our card
  24. Ioannis Mollas, Zoe Chrysopoulou, Stamatis Karlos, and Grigorios Tsoumakas. ETHOS: an online hate speech detection dataset, 2020. URL https://arxiv.org/abs/2006.08328. — [arXiv:2006.08328](https://arxiv.org/abs/2006.08328) ↗
  25. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. In Conference on Neural Information Processing Systems (NeurIPS), 2022. URL https… find
  26. Bo Pang and Lillian Lee. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the Association for Computational Linguistics (ACL), 2005. URL https://aclanthology.org/P05-1015/. ↗
  27. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2016. URL https://aclanthology.org/ D16-1264. ↗ our card
  28. Laria Reynolds and Kyle McDonell. Prompt programming for large language models: Beyond the few-shot paradigm. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 2021. URL https://arxiv.org/abs/2102.07350. ↗ our card
  29. Frieda Rong. Extrapolating to unnatural language processing with GPT-3’s in-context learning: The good, the bad, and the mysterious, 2021. URL https://ai.stanford.edu/blog/ in-context-learning/. ↗
  30. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2013. URL… find
  31. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models, 2022. URL https://arxiv.org/abs/2206.04… ↗ our card
  32. Dustin Tran, Jeremiah Liu, Michael W. Dusenberry, Du Phan, Mark Collier, Jie Ren, Kehang Han, Zi Wang, Zelda Mariet, Huiyi Hu, Neil Band, Tim G. J. Rudner, Karan Singhal, Zachary Nado, Joost van Amersfoort, Andreas Kirsch, Rodolphe Jenatton, Nithum Thain, Honglin Yuan, Kelly Buchanan, Kevin Murphy,… — [arXiv:2207.07411](https://arxiv.org/abs/2207.07411) find
  33. Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent, 2022. URL https://arxiv.org/abs/2212.07677. ↗ our card
  34. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, 2018. URL https:/… find our card
  35. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. SuperGLUE: A stickier benchmark for general-purpose language understanding systems. Conference on Neural Information Processing Systems (NeurIPS), 2019. URL https://arxiv.org/abs/… ↗ our card
  36. Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. Towards understanding chain-of-thought prompting: An empirical study of what matters. arXiv preprint arXiv:2212.10001, 2022. URL https://arxiv.org/abs/2212.10001. ↗ our card
  37. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations (ICLR), 2023. URL https: //openreview.net/forum?id=1PL1N… find our card
  38. Albert Webson and Ellie Pavlick. Do prompt-based models really understand the meaning of their prompts? In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL): Human Language Technologies, 2022. URL https://aclanthology.org/2022.n… ↗ our card
  39. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. Finetuned language models are zero-shot learners. International Conference on Learning Representations (ICLR), 2022a. URL https://openreview.net/ forum?id=gEZrGCozdqR. ↗
  40. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language models. Transactions on Machine… find our card
  41. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. Conference on Neural Information Processing Systems (NeurIPS), 2022c. URL https://arxiv.org/abs/2201. 11903. ↗ our card
  42. Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit bayesian inference. International Conference on Learning Representations (ICLR), 2022. URL https://arxiv.org/abs/2111.02080. ↗ our card
  43. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models. International Conference on Machine Learning (ICML), 2021. URL https://arxiv.org/abs/2102.09690. ↗ our card
  44. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoc Le, and Ed Chi. Least-to-most prompting enables complex reasoning in large language models. International Conference on Learning Representations (ICLR), 2023. URL https://arxiv.org/a… ↗ our card