Elamur Research
☰
home / papers / 2022 / Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2205.05638

Summary

Few-shot in-context learning (ICL) lets a large language model take on a new task without training. However, every prediction must reprocess all demonstration examples, which is costly, and results shift with prompt wording and example order. The authors test whether parameter-efficient fine-tuning (PEFT) is a better option when only a handful of labeled examples are available. They propose T-Few, a fixed recipe built on the T0 model (a multitask-prompted variant of T5) that needs no per-task tuning. Its core is (IA)3, a new PEFT method that learns vectors to element-wise rescale attention keys and values and the inner feed-forward activations. Training also adds an unlikelihood term that penalizes wrong answer choices and a length-normalized loss, and the (IA)3 vectors are pre-trained before task adaptation. T-Few outperforms ICL, even against models 16× larger, while using over 1,000× fewer inference FLOPs. It is also the first method to beat the human baseline on RAFT. The work argues that cheap fine-tuning, rather than prompting, is the practical default for few-shot learning.

Key points

  • On RAFT (11 real-world tasks, 50 training examples each, hidden test labels), T-Few scores 75.8%. This beats the human baseline (73.5%), the previous best system by about 6 points, and GPT-3 175B (62.7%).
  • Per-example inference costs about 1.1e12 FLOPs for T-Few versus roughly 1.4e15 FLOPs for few-shot ICL with GPT-3 175B. Caching keys and values would reduce the ICL cost by only about 41×.
  • Training T-Few takes around 2.7e16 FLOPs, about 30 minutes on one NVIDIA A100 GPU (roughly $2 on Microsoft Azure).
  • The (IA)3 parameters for one task take 4.2 MB. Adapters for 10,000 tasks would roughly equal the size of the T0 checkpoint (41.5 GB).
  • In ablations, dropping (IA)3 pre-training lowers average accuracy by 1.6%, and dropping the unlikelihood and length-normalization losses lowers it by 4.1%. Code is released at github.com/r-three/t-few.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

358 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 347 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2205.05638 · arXiv 2205.05638

@misc{liu2022few,
  title = {{Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning}},
  author = {Liu, Haokun and Tam, Derek and Muqeeth, Mohammed and Mohta, Jay and Huang, Tenghao and Bansal, Mohit and Raffel, Colin},
  year = {2022},
  doi = {10.48550/arxiv.2205.05638},
  eprint = {2205.05638},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2205.05638}
}

References

81 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207, 2021. find our card
  2. Neel Alex, Eli Lifland, Lewis Tunstall, Abhishek Thakur, Pegah Maham, C Jess Riedel, Emmie Hine, Carolyn Ashurst, Paul Sedille, Alexis Carlier, et al. RAFT: A real-world few-shot text classification benchmark. arXiv preprint arXiv:2109.14076, 2021. — [arXiv:2109.14076](https://arxiv.org/abs/2109.14076) find
  3. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI blog, 2019. find
  4. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020. find
  5. Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, and He He. Meta-learning via language model in-context tuning. arXiv preprint arXiv:2110.07814, 2021. find
  6. Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. Metaicl: Learning to learn in context. arXiv preprint arXiv:2110.15943, 2021. find
  7. Andrew Kyle Lampinen, Ishita Dasgupta, Stephanie C. Y. Chan, Kory Matthewson, Michael Henry Tessler, Antonia Creswell, James L. McClelland, Jane X. Wang, and Felix Hill. Can language models learn from explanations in context? ArXiv, abs/2204.02329, 2022. find our card
  8. Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and Nikolai Grigorev. Internet-augmented language models through few-shot prompting for open-domain question answering. arXiv preprint arXiv:2203.05115, 2022. find our card
  9. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837, 2022. find our card
  10. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Benchmarking generalization via in-context instructions on 1,600+ language tasks. arXiv preprint arXiv:2204.07705, 2022. find
  11. Albert Webson and Ellie Pavlick. Do prompt-based models really understand the meaning of their prompts? arXiv preprint arXiv:2109.01247, 2021. find our card
  12. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models. arXiv preprint arXiv:2102.09690, 2021. find our card
  13. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. ArXiv, abs/2106.09685, 2021. find
  14. Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691, 2021. find
  15. Colin Raffel, Noam M. Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. ArXiv, abs/1910.10683, 2020. find
  16. Derek Tam, Rakesh R Menon, Mohit Bansal, Shashank Srivastava, and Colin Raffel. Improving and simplifying pattern exploiting training. arXiv preprint arXiv:2103.11955, 2021. find
  17. Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. Neural text generation with unlikelihood training. arXiv preprint arXiv:1908.04319, 2019. find our card
  18. Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. PPT: Pre-trained prompt tuning for few-shot learning. arXiv preprint arXiv:2109.04332, 2021. find our card
  19. Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou, and Daniel Cer. SPoT: Better frozen model adaptation through soft prompt transfer. arXiv preprint arXiv:2110.07904, 2021. find our card
  20. Jared Kaplan, Sam McCandlish, T. J. Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeff Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020. find
  21. Sewon Min, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Noisy channel language model prompting for few-shot text classification. arXiv preprint arXiv:2108.04106, 2021. find our card
  22. Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. Learning multiple visual domains with residual adapters. Advances in neural information processing systems, 30, 2017. find our card
  23. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. arXiv preprint arXiv:1902.00751, 2019. find
  24. Ankur Bapna, Naveen Arivazhagan, and Orhan Firat. Simple, scalable adaptation for neural machine translation. arXiv preprint arXiv:1909.08478, 2019. find our card
  25. Demi Guo, Alexander M. Rush, and Yoon Kim. Parameter-efficient transfer learning with diff pruning. arXiv preprint arXiv:2012.07463, 2020. find our card
  26. Yi-Lin Sung, Varun Nair, and Colin Raffel. Training neural networks with fixed sparse masks. arXiv preprint arXiv:2111.09839, 2021. — [arXiv:2111.09839](https://arxiv.org/abs/2111.09839) find
  27. Armen Aghajanyan, Luke Zettlemoyer, and Sonal Gupta. Intrinsic dimensionality explains the effectiveness of language model fine-tuning. arXiv preprint arXiv:2012.13255, 2020. find our card
  28. Rabeeh Karimi Mahabadi, James Henderson, and Sebastian Ruder. Compacter: Efficient low-rank hypercomplex adapter layers. arXiv preprint arXiv:2106.04647, 2021. — [arXiv:2106.04647](https://arxiv.org/abs/2106.04647) find
  29. Xiang Lisa Li and Percy Liang. Prefix-Tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190, 2021. find
  30. Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. Towards a unified view of parameter-efficient transfer learning. arXiv preprint arXiv:2110.04366, 2021. find our card
  31. Ethan Perez, Douwe Kiela, and Kyunghyun Cho. True few-shot learning with language models. arXiv preprint arXiv:2105.11447, 2021. find our card
  32. Avital Oliver, Augustus Odena, Colin Raffel, Ekin Dogus Cubuk, and Ian Goodfellow. Realistic evaluation of deep semi-supervised learning algorithms. Advances in Neural Information Processing Systems, 2018. find
  33. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 2017. find
  34. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. find
  35. Stephen H. Bach, Victor Sanh, Zheng-Xin Yong, Albert Webson, Colin Raffel, Nihal V. Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Févry, et al. PromptSource: An integrated development environment and repository for natural language prompts. arXiv preprint arXiv:2202.01279, 2022. find our card
  36. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, et al. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processi… find our card
  37. Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S. Gordon. Choice of plausible alternatives: An evaluation of commonsense causal reasoning. 2011 AAAI Spring Symposium Series, 2011. find
  38. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019. find
  39. Rishi Sharma, James Allen, Omid Bakhshandeh, and Nasrin Mostafazadeh. Tackling the story ending biases in the story cloze test. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 752–757, 2018. find
  40. Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adversarial NLI: A new benchmark for natural language understanding. arXiv preprint arXiv:1910.14599, 2019. find
  41. Marie-Catherine de Marneffe, Mandy Simons, and Judith Tonhauser. The CommitmentBank: Investigating projection in naturally occurring discourse. Proceedings of Sinn und Bedeutung 23, 2019. find our card
  42. Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Machine Learning Challenges Workshop, pages 177–190. Springer, 2005. find
  43. Hector Levesque, Ernest Davis, and Leora Morgenstern. The winograd schema challenge. Thirteenth International Conference on the Principles of Knowledge Representation and Reasoning, 2012. find
  44. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. In Proceedings of the AAAI Conference on Artificial Intelligence, 2020. find
  45. Mohammad Taher Pilehvar and Jose Camacho-Collados. WiC: the word-in-context dataset for evaluating context-sensitive meaning representations. arXiv preprint arXiv:1808.09121, 2018. find our card
  46. Stefan Van Der Walt, S. Chris Colbert, and Gael Varoquaux. The numpy array: a structure for efficient numerical computation. Computing in science & engineering, 13(2), 2011. find
  47. Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. arXiv preprint arXiv:2106.10199, 2021. find our card
  48. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V. Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021. find
  49. Noam Shazeer and Mitchell Stern. Adafactor: Adaptive learning rates with sublinear memory cost. In International Conference on Machine Learning. PMLR, 2018. find
  50. Timo Schick and Hinrich Schütze. True few-shot learning with prompts–a real-world perspective. arXiv preprint arXiv:2111.13440, 2021. find
  51. Moshe Wasserblat. Sentence transformer fine-tuning (SetFit): Outperforming GPT-3 on few-shot text-classification while being 1600 times smaller, 2021. find
  52. Mostafa Dehghani, Anurag Arnab, Lucas Beyer, Ashish Vaswani, and Yi Tay. The efficiency misnomer. arXiv preprint arXiv:2110.12894, 2021. — [arXiv:2110.12894](https://arxiv.org/abs/2110.12894) find
  53. Guanghui Qin and Jason Eisner. Learning how to ask: Querying LMs with mixtures of soft prompts. arXiv preprint arXiv:2104.06599, 2021. find our card
  54. Xiao Liu, Kaixuan Ji, Yicheng Fu, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-Tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602, 2021. find our card
  55. Shengnan An, Yifei Li, Zeqi Lin, Qian Liu, Bei Chen, Qiang Fu, Weizhu Chen, Nanning Zheng, and Jian-Guang Lou. Input-Tuning: Adapting unfamiliar inputs to frozen pretrained models. arXiv preprint arXiv:2203.03131, 2022. — [arXiv:2203.03131](https://arxiv.org/abs/2203.03131) find
  56. Yulong Chen, Yang Liu, Li Dong, Shuohang Wang, Chenguang Zhu, Michael Zeng, and Yue Zhang. AdaPrompt: Adaptive model training for prompt-based NLP. arXiv preprint arXiv:2202.04824, 2022. find
  57. Shizhe Diao, Xuechun Li, Yong Lin, Zhichao Huang, and Tong Zhang. Black-box prompt learning for pre-trained language models. arXiv preprint arXiv:2201.08531, 2022. — [arXiv:2201.08531](https://arxiv.org/abs/2201.08531) find
  58. Daniel Khashabi, Shane Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sameer Singh, Sean Welleck, Hannaneh Hajishirzi, Tushar Khot, Ashish Sabharwal, et al. Prompt wayward-ness: The curious case of discretized interpretation of continuous prompts. arXiv preprint arXiv:2112.08348, 2021. — [arXiv:2112.08348](https://arxiv.org/abs/2112.08348) find
  59. Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. arXiv preprint arXiv:2112.08654, 2021. find
  60. Zonghan Yang and Yang Liu. On robust prefix-tuning for text classification. arXiv preprint arXiv:2203.10378, 2022. — [arXiv:2203.10378](https://arxiv.org/abs/2203.10378) find
  61. Yuting Yang, Pei Huang, Juan Cao, Jintao Li, Yun Lin, Jin Song Dong, Feifei Ma, and Jian Zhang. A prompting-based approach for adversarial example generation and robustness enhancement. arXiv preprint arXiv:2203.10714, 2022. — [arXiv:2203.10714](https://arxiv.org/abs/2203.10714) find
  62. Xiaochen Liu, Yu Bai, Jiawei Li, Yinan Hu, and Yang Gao. PSP: Pre-trained soft prompts for few-shot abstractive summarization. arXiv preprint arXiv:2204.04413, 2022. find
  63. Xavier Garcia and Orhan Firat. Using natural language prompts for machine translation. arXiv preprint arXiv:2202.11822, 2022. — [arXiv:2202.11822](https://arxiv.org/abs/2202.11822) find
  64. Hunter Lang, Monica Agrawal, Yoon Kim, and David Sontag. Co-training improves prompt-based learning for large language models. arXiv preprint arXiv:2202.00828, 2022. — [arXiv:2202.00828](https://arxiv.org/abs/2202.00828) find
  65. Boshi Wang, Xiang Deng, and Huan Sun. Shepherd pre-trained language models to develop a train of thought: An iterative prompting approach. arXiv preprint arXiv:2203.08383, 2022. find
  66. Xu Zou, Da Yin, Qingyang Zhong, Hongxia Yang, Zhilin Yang, and Jie Tang. Controllable generation from pre-trained language models via inverse prompting. arXiv preprint arXiv:2103.10685, 2021. — [arXiv:2103.10685](https://arxiv.org/abs/2103.10685) find
  67. Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, et al. On transferability of prompt tuning for natural language understanding. arXiv preprint arXiv:2111.06719, 2021. — [arXiv:2111.06719](https://arxiv.org/abs/2111.06719) find
  68. Yun He, Huaixiu Steven Zheng, Yi Tay, Jai Gupta, Yu Du, Vamsi Aribandi, Zhe Zhao, YaGuang Li, Zhao Chen, Donald Metzler, et al. HyperPrompt: Prompt-based task-conditioning of transformers. arXiv preprint arXiv:2203.00759, 2022. — [arXiv:2203.00759](https://arxiv.org/abs/2203.00759) find
  69. Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. arXiv preprint arXiv:2203.12119, 2022. — [arXiv:2203.12119](https://arxiv.org/abs/2203.12119) find
  70. Timo Schick and Hinrich Schütze. Exploiting cloze questions for few shot text classification and natural language inference. arXiv preprint arXiv:2001.07676, 2020. find our card
  71. Teven Le Scao and Alexander M. Rush. How many data points is a prompt worth? arXiv preprint arXiv:2103.08493, 2021. find our card
  72. Sen Yang, Yunchen Zhang, Leyang Cui, and Yue Zhang. Do prompts solve NLP tasks using natural language? arXiv preprint arXiv:2203.00902, 2022. — [arXiv:2203.00902](https://arxiv.org/abs/2203.00902) find
  73. Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. Auto-Prompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980, 2020. find our card
  74. Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. arXiv preprint arXiv:2012.15723, 2020. find our card
  75. Ningyu Zhang, Luoqiu Li, Xiang Chen, Shumin Deng, Zhen Bi, Chuanqi Tan, Fei Huang, and Huajun Chen. Differentiable prompt makes pre-trained language models better few-shot learners. arXiv preprint arXiv:2108.13161, 2021. — [arXiv:2108.13161](https://arxiv.org/abs/2108.13161) find
  76. Rabeeh Karimi Mahabadi, Luke Zettlemoyer, James Henderson, Marzieh Saeidi, Lambert Mathias, Veselin Stoyanov, and Majid Yazdani. PERFECT: Prompt-free and efficient few-shot learning with language models. arXiv preprint arXiv:2204.01172, 2022. find
  77. Nafise Sadat Moosavi, Quentin Delfosse, Kristian Kersting, and Iryna Gurevych. Adaptable adapters. arXiv preprint arXiv:2205.01549, 2022. find
  78. Eleni Triantafillou, Hugo Larochelle, Richard Zemel, and Vincent Dumoulin. Learning a universal template for few-shot dataset generalization. arXiv preprint arXiv:/2105.07029, 2021. find
  79. James Requeima, Jonathan Gordon, John Bronskill, Sebastian Nowozin, and Richard E. Turner. Fast and flexible multi-task classification using conditional neural adaptive processes. arXiv preprint arXiv:1906.07697, 2019. — [arXiv:1906.07697](https://arxiv.org/abs/1906.07697) find
  80. Wei-Hong Li, Xialei Liu, and Hakan Bilen. Universal representation learning from multiple domains for few-shot classification. Proceedings of the IEEE/CVF International Conference on Computer Vision., 2021. find
  81. Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, Jason Phang, Laria Reynolds, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, September… — [doi:10.5281/zenodo.5371628](https://doi.org/10.5281/zenodo.5371628) find