Elamur Research
☰
home / papers / 2023 / Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning

Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2305.14160

Summary

In-context learning (ICL) lets large language models solve tasks from a handful of demonstrations without weight updates, but how the model uses those demonstrations is poorly understood. The authors trace information flow inside GPT2-XL (1.5B) and GPT-J (6B) on four text classification benchmarks. They compute saliency scores over attention matrices to measure how strongly information moves from demonstration text to label words, from label words to the prediction position, and among the remaining tokens. The results suggest that label words work as anchors. Early layers compress each demonstration's meaning into its label token, and later layers read those tokens to choose the output. Two interventions support this. Blocking attention into label words in shallow layers damages accuracy most. The attention the final position gives a label also tracks that label's predicted probability. The authors turn this insight into three tools: anchor re-weighting for better accuracy, compression of demonstrations into precomputed anchor states for faster inference, and diagnosis of class confusions from anchor key distances. The work makes ICL more transparent and cheaper to run.

Key points

  • Setup: GPT2-XL and GPT-J were tested on SST-2, TREC, AGNews and EmoC. Each run used 1000 sampled test inputs and one demonstration per class, with results averaged over five random seeds.
  • Saliency metrics: Three metrics track different information flows: S_wp (text → label words), S_pq (label words → target position) and S_ww (all other flows). S_wp dominates in shallow layers and S_pq takes over in deep layers. Both usually exceed S_ww.
  • Isolation test: Cutting label words off from the preceding text in the first 5 layers hurts performance far more than the same cut in the last 5 layers. It also hurts more than isolating random non-label words.
  • Applications:
    • Anchor re-weighting uses a learnable vector and raises accuracy by 16.7% on average over standard ICL.
    • Anchor-based demonstration compression speeds up inference by about 1.8× on average, with little accuracy loss. Per-setting speedups range from 1.1× to 2.9× and are larger on GPT-J.
  • Error diagnosis and code: On the TREC test set (500 samples) with GPT2-XL, confusion predicted from label-word key vector distances matches the AUC-ROC-based real confusion. It singles out Description–Entity as the most confused pair. Code is available at github.com/lancopku/label-words-are-anchors.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

121 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 119 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2305.14160 · arXiv 2305.14160

@misc{wang2023label,
  title = {{Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning}},
  author = {Wang, Lean and Li, Lei and Dai, Damai and Chen, Deli and Zhou, Hao and Meng, Fandong and Zhou, Jie and Sun, Xu},
  year = {2023},
  doi = {10.48550/arxiv.2305.14160},
  eprint = {2305.14160},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.14160}
}

References

27 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. 2022. What learning algorithm is in-context learning? investigations with linear models. ArXiv preprint, abs/2211.15661. find our card
  2. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… find
  3. Ankush Chatterjee, Kedhar Nath Narahari, Meghana Joshi, and Puneet Agrawal. 2019. SemEval-2019 task 3: EmoContext contextual emotion detection in text. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 39–48, Minneapolis, Minnesota, USA. Association for Computational Li… find
  4. Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Zhifang Sui, and Furu Wei. 2022. Why can gpt learn in-context? language models secretly perform gradient descent as meta optimizers. ArXiv preprint, abs/2212.10559. find
  5. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2023. A survey for in-context learning. ArXiv preprint, abs/2301.00234. find our card
  6. Eduard Hovy, Laurie Gerber, Ulf Hermjakob, Chin-Yew Lin, and Deepak Ravichandran. 2001. Toward semantics-based answer pinpointing. In Proceedings of the First International Conference on Human Language Technology Research. find
  7. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. find
  8. Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. 2020. Attention is not only a weight: Analyzing transformers with vector norms. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7057–7075, Online. Association for Computational… find our card
  9. Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, and Xipeng Qiu. 2023a. Unified demonstration retriever for in-context learning. ArXiv preprint, abs/2305.04320. find
  10. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. ArXiv, a… find
  11. Xin Li and Dan Roth. 2002. Learning question classifiers. In COLING 2002: The 19th International Conference on Computational Linguistics. find
  12. Yingcong Li, Muhammed Emrullah Ildiz, Dimitris Papailiopoulos, and Samet Oymak. 2023b. Transformers as algorithms: Generalization and stability in in-context learning. find
  13. Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2022. What makes good in-context examples for GPT-3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 100–114… find our card
  14. Paul Michel, Omer Levy, and Graham Neubig. 2019. Are sixteen heads really better than one? In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 14014–14024. find our card
  15. Sewon Min, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022a. Noisy channel language model prompting for few-shot text classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5316–5330, Dublin, Ireland. Asso… find our card
  16. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022b. Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11048–11064, A… find our card
  17. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. find
  18. Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. CoRR, abs/1312.6034. find our card
  19. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for Johannes von Oswald, Eyvind Niklasson, E. Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. 2022. Transformers learn… find our card
  20. Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. https://github.com/kingoflolz/ mesh-transformer-jax. ↗
  21. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Huai hsin Chi, F. Xia, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. ArXiv preprint, abs/2201.11903. find our card
  22. Jerry W. Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, and Tengyu Ma. 2023. Larger language models do in-context learning differently. ArXiv, abs/2303.03846. find our card
  23. Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. 2022. An explanation of in-context learning as implicit bayesian inference. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. find our card
  24. Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong. 2023. Compositional exemplars for in-context learning. ArXiv preprint, abs/2302.05698. find
  25. Kang Min Yoo, Junyeob Kim, Hyuhng Joon Kim, Hyunsoo Cho, Hwiyeol Jo, Sang-Woo Lee, Sang-goo Lee, and Taeuk Kim. 2022. Ground-truth labels matter: A deeper look into input-label demonstrations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2422– 2437… find
  26. Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 649–657.… find our card
  27. Review: \<S1\> Sentiment: \<L\> Review: \<S\> Sentiment: Positive, Negative Question: \<S1\> Answer Type: \<L\> Question: \<S\> Answer Type: Abbreviation, Entity Description, Person Location, Number Article: \<S1\> Answer: \<L\> Article: \<S\> Answer: World, Sports Business, Technology Dialogue: \<S1\> Emotion: \<L\> Dia… find our card