Elamur Research⌕
☰
home / papers / 2023 / Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers

Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers

Summary

This paper asks why large GPT-style models can solve new tasks from a few demonstrations without any weight updates. The authors simplify Transformer attention by dropping the softmax and scaling, which gives a relaxed linear form. In this form, attending to demonstration tokens works like adding an update matrix to the zero-shot weights. That matches a known dual relationship between linear attention and linear layers trained by gradient descent. They therefore treat a pretrained GPT as a meta-optimizer: its forward pass turns demonstrations into "meta-gradients," and attention applies them, so in-context learning (ICL) acts as implicit finetuning.

To test this, they compare ICL with a tightly matched finetuning baseline on six classification tasks. The comparison examines what the models predict, what their attention layers produce, and how attention is distributed, and the two behave alike on every measure. The analogy also suggests a new design: a momentum-based attention, modelled on gradient descent with momentum. It beats vanilla attention on both perplexity and ICL accuracy. The work offers a mechanistic account of ICL in real models and real NLP tasks, not toy regression settings.

Key points

  • Theory: Under relaxed linear attention, the attention output splits into two terms. One is the zero-shot term W_ZSL·q. The other is an update ΔW_ICL built from the demonstration tokens. The values W_V X′ play the role of meta-gradients.

  • Finetuning baseline: It was built to mirror ICL as closely as possible:

    • only the key and value projections are updated;
    • the training examples are the demonstrations themselves;
    • each example gets one step, in the same order and with the same template;
    • training uses a causal language modeling objective.

  • Empirical comparison: ICL and finetuning were compared on six classification tasks across four views:

    • the final outputs each model predicts;
    • the results produced by the attention layers;
    • how much attention is placed on the query tokens;
    • how much attention is placed on the training tokens.

    Because recording intermediate states is costly, only GPT models of up to 2.7B parameters were analysed.

  • Momentum-based attention: It lowered perplexity consistently at input lengths of 256, 512 and 1024. It also raised ICL accuracy, using up to 32 demonstrations, on SST5, IMDB, MR, CB, ARC-E and PIQA.

  • Limitations:

    • The analysis rests on a linear approximation of attention.
    • It covers only Transformers.
    • The main analysis uses only classification tasks.

    Code is available at aka.ms/icl.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

105 papers of our library cite it (of the 140884 whose bibliographies we read); 11 of them have a page here, 94 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.findings-acl.247

@inproceedings{dai2023why,
  title = {{Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers}},
  author = {Dai, Damai and Sun, Yutao and Dong, Li and Hao, Yaru and Ma, Shuming and Sui, Zhifang and Wei, Furu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-acl.247},
  url = {https://doi.org/10.18653/v1/2023.findings-acl.247}
}

References

25 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mark A Aizerman, Emmanuil M Braverman, and Lev I Rozonoer. 1964. Theoretical foundation of potential functions method in pattern recognition. Avtomatika i Telemekhanika, 25(6):917–936. find
  2. Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. 2022. What learning algorithm is in-context learning? investigations with linear models. CoRR, abs/2211.15661. find our card
  3. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… find
  4. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the AI2 reasoning challenge. CoRR, abs/1803.05457. find
  5. Marie-Catherine De Marneffe, Mandy Simons, and Judith Tonhauser. 2019. The commitmentbank: Investigating projection in naturally occurring discourse. In proceedings of Sinn und Bedeutung, volume 23, pages 107–124. find our card
  6. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, Lei Li, and Zhifang Sui. 2023. A survey for in-context learning. CoRR, abs/2301.00234. find our card
  7. Shivam Garg, Dimitris Tsipras, Percy Liang, and Gregory Valiant. 2022. What can transformers learn in-context? A case study of simple function classes. CoRR, abs/2208.01066. find
  8. J Stuart Hunter. 1986. The exponentially weighted moving average. Journal of quality technology, 18(4):203–210. find
  9. Kazuki Irie, Róbert Csordás, and Jürgen Schmidhuber. 2022. The dual form of neural networks revisited: Connecting test time predictions to training patterns via spotlights of attention. In International Conference on Machine Learning, ICML 2022, volume 162 of Proceedings of Machine Learning Research… find
  10. Maurice George Kendall. 1948. Rank correlation methods. find
  11. Louis Kirsch, James Harrison, Jascha Sohl-Dickstein, and Luke Metz. 2022. General-purpose in-context learning by meta-learning transformers. CoRR, abs/2212.04458. — [arXiv:2212.04458](https://arxiv.org/abs/2212.04458) find
  12. Louis Kirsch and Jürgen Schmidhuber. 2021. Meta learning backpropagation and improving it. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, pages 14122– 14134. find
  13. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142–150, Portland, Oregon… find our card
  14. Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amode… find
  15. Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), pages 271–278, Barcelona, Spain. find our card
  16. Bo Pang and Lillian Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In ACL 2005, 43rd Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, pages 115–124. The Association for Computer Lingui… find our card
  17. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–… find
  18. Ilya Sutskever, James Martens, George E. Dahl, and Geoffrey E. Hinton. 2013. On the importance of initialization and momentum in deep learning. In Proceedings of the 30th International Conference on Machine Learning, ICML 2013, volume 28 of JMLR Workshop and Conference Proceedings, pages 1139– 1147.… find
  19. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998–6008. Curran Associates, Inc. find
  20. Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. 2022. Transformers learn in-context by gradient descent. ArXiv preprint, abs/2212.07677. find our card
  21. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. Emergent abilities of large language models. CoRR, abs/2206.076… find our card
  22. Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. 2022. An explanation of in-context learning as implicit bayesian inference. In The Tenth International Conference on Learning Representations, ICLR 2022. OpenReview.net. find our card
  23. Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, pages 649– 657. find our card
  24. Boris T Polyak. 1964. Some methods of speeding up the convergence of iteration methods. Ussr computational mathematics and mathematical physics, 4(5):1–17. find
  25. Ofir Press, Noah A. Smith, and Mike Lewis. 2022. Train short, test long: Attention with linear biases enables input length extrapolation. In The Tenth International Conference on Learning Representations, ICLR 2022. OpenReview.net. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dai-2023-why-can-gpt-learn-context/