Calibrate Before Use: Improving Few-Shot Performance of Language Models
Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2102.09690
Summary
Few-shot in-context learning lets GPT-3 handle a task from a handful of examples in the prompt, with no weight updates. The authors show this is fragile: accuracy swings widely depending on the prompt format, which examples are chosen, and even their order. They trace the problem to systematic answer preferences: recency bias, majority label bias and common token bias. These shift the output distribution toward certain labels. Their fix, contextual calibration, pairs the prompt with a dummy, content-free input like "N/A". The model's skewed prediction on that input serves as an estimate of its bias, and calibration parameters are then fitted so the dummy receives uniform scores over all answers. No additional labeled data is required.
Evaluation covers text classification (SST-2, TREC, CB, RTE, AGNews, DBPedia), fact retrieval (LAMA) and slot filling (ATIS, MIT Movies trivia10k13). The models are GPT-3 2.7B, 13B and 175B, plus GPT-2 1.5B. Calibration raises average accuracy by as much as 30.0 points absolute and makes results far less sensitive to prompt choices, which reduces the need for prompt engineering.
Key points
- On 4-shot SST-2 with GPT-3 2.7B, merely reordering the same training examples moved accuracy from 54.3% (near chance) to 93.4% (near state of the art).
- More shots or bigger models do not remove the instability. Variance stays high at 16 examples and for 175B. On DBPedia, mean accuracy even dropped from 36.0% (0-shot) to 25.9% (1-shot).
- Bias example: putting "N/A" in place of the test review in a two-shot sentiment prompt still yields 62% Positive.
- Contextual calibration needs no training data, yet on AGNews with GPT-3 175B it performs about as well as an "oracle" calibration whose matrix W is tuned on the validation set.
- Metrics are accuracy for classification and LAMA, and Exact Match for ATIS/MIT Movies slots. Code is released at github.com/tonyzhaozh/few-shot-learning.
Cited by
- MEGA: Multilingual Evaluation of Generative AI
- Flamingo: a Visual Language Model for Few-Shot Learning
- PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts
- Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering
- On the Opportunities and Risks of Foundation Models
- Evaluating Large Language Models Trained on Code
- Adapting Language Models to Compress Contexts
- RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
- A Survey on In-context Learning
- Complexity-Based Prompting for Multi-Step Reasoning
- Bias and Fairness in Large Language Models: A Survey
- Surface Form Competition: Why the Highest Probability Answer Isn't Always Right
- Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting
- Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations
- Challenges and Applications of Large Language Models
- Making Language Models Better Reasoners with Step-Aware Verifier
- LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
- Few-shot Learning with Multilingual Language Models
- GPT Understands, Too
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
- Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
- What Makes Good In-Context Examples for GPT-3?
- Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning
- Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
- Generating Training Data with Language Models: Towards Zero-Shot Language Understanding
- Noisy Channel Language Model Prompting for Few-Shot Text Classification
- Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
- Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
- Reframing Instructional Prompts to GPTk's Language
- True Few-Shot Learning with Language Models
- Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions
- Leveraging Large Language Models for Multiple Choice Question Answering
- Learning To Retrieve Prompts for In-Context Learning
- Multitask Prompted Training Enables Zero-Shot Task Generalization
- On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
- mGPT: Few-Shot Learners Go Multilingual
- Prompting GPT-3 To Be Reliable
- Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
- Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
- SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
- Want To Reduce Labeling Cost? GPT-3 Can Help
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning
- GPT-NER: Named Entity Recognition via Large Language Models
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Emergent Abilities of Large Language Models
- Larger language models do in-context learning differently
- Ethical and social risks of harm from Language Models
- An Explanation of In-context Learning as Implicit Bayesian Inference
- Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
- Large Language Models as Optimizers
- The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning
- GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
- Active Example Selection for In-Context Learning
- Instruction Tuning for Large Language Models: A Survey
- Prompting Large Language Model for Machine Translation: A Case Study
- A Survey of Large Language Models
- Can Large Language Models Transform Computational Social Science?
Cite
@misc{zhao2021calibrate,
title = {{Calibrate Before Use: Improving Few-Shot Performance of Language Models}},
author = {Zhao, Tony Z. and Wallace, Eric and Feng, Shi and Klein, Dan and Singh, Sameer},
year = {2021},
doi = {10.48550/arxiv.2102.09690},
eprint = {2102.09690},
archivePrefix = {arXiv},
url = {https://doi.org/10.48550/arxiv.2102.09690}
}References
- Bosselut, A., Rashkin, H., Sap, M., Malaviya, C., Celikyilmaz, A., and Choi, Y. COMET: Commonsense transformers for automatic knowledge graph construction. In ACL, 2019. find our card
- Braverman, M., Chen, X., Kakade, S., Narasimhan, K., Zhang, C., and Zhang, Y. Calibration, entropy rates, and memory in language models. In ICML, 2020. find
- Brier, G. W. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 1950. find
- Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Ch… find
- Dagan, I., Glickman, O., and Magnini, B. The PASCAL recognising textual entailment challenge. In Machine Learning Challenges Workshop, 2005. de Marneffe, M.-C., Simons, M., and Tonhauser, J. The CommitmentBank: Investigating projection in naturally occurring discourse. In Sinn und Bedeutung, 2019. find our card
- Devlin, J., Chang, M., Lee, K., and Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL, 2019. find
- Gao, T., Fisch, A., and Chen, D. Making pre-trained language models better few-shot learners. arXiv preprint arXiv:2012.15723, 2020. find our card
- Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. On calibration of modern neural networks. In ICML, 2017. find
- Hemphill, C. T., Godfrey, J. J., and Doddington, G. R. The ATIS spoken language systems pilot corpus. In Speech and Natural Language Workshop, 1990. find
- Holtzman, A., Buys, J., Du, L., Forbes, M., and Choi, Y. The curious case of neural text degeneration. In ICLR, 2020. find
- Jiang, Z., Araki, J., Ding, H., and Neubig, G. How can we know when language models know? arXiv preprint arXiv:2012.00955, 2020a. find
- Jiang, Z., Xu, F. F., Araki, J., and Neubig, G. How can we know what language models know? In TACL, 2020b. find our card
- Khandelwal, U., He, H., Qi, P., and Jurafsky, D. Sharp nearby, fuzzy far away: How neural language models use context. In ACL, 2018. find
- Lake, B. M., Salakhutdinov, R., and Tenenbaum, J. B. Human-level concept learning through probabilistic program induction. In Science, 2015. find
- Li, J., Galley, M., Brockett, C., Gao, J., and Dolan, B. A diversity-promoting objective function for neural conversation models. In NAACL, 2016. find our card
- Liu, J., Cyphers, S., Pasupat, P., McGraw, I., and Glass, J. A conversational movie search system based on conditional random fields. In INTERSPEECH, 2012. find
- Logan, R. L., Liu, N. F., Peters, M. E., Gardner, M., and Singh, S. Barack’s wife Hillary: Using knowledge-graphs for fact-aware language modeling. In ACL, 2019. find
- Paulus, R., Xiong, C., and Socher, R. A deep reinforced model for abstractive summarization. In ICLR, 2018. find our card
- Petroni, F., Rocktäschel, T., Lewis, P., Bakhtin, A., Wu, Y., Miller, A. H., and Riedel, S. Language models as knowledge bases? In EMNLP, 2019. find our card
- Platt, J. C. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances in Large Margin Classifiers, 1999. find
- Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. Improving language understanding by generative pre-training. Technical Report, 2018. find
- Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language models are unsupervised multitask learners. Technical Report, 2019. find
- Ravfogel, S., Goldberg, Y., and Linzen, T. Studying the inductive biases of RNNs with synthetic variations of natural languages. In NAACL, 2019. find
- Schick, T. and Schütze, H. It’s not just size that matters: Small language models are also few-shot learners. arXiv preprint arXiv:2009.07118, 2020. find our card
- Schick, T. and Schütze, H. Exploiting cloze questions for few-shot text classification and natural language inference. In EACL, 2021. find our card
- Schwartz, R., Sap, M., Konstas, I., Zilles, L., Choi, Y., and Smith, N. A. The effect of different writing tasks on linguistic style: A case study of the ROC story cloze task. In ACL, 2017. find
- Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., and Singh, S. AutoPrompt: Eliciting knowledge from language models with automatically generated prompts. In EMNLP, 2020. find our card
- Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., and Potts, C. Recursive deep models for semantic compositionality over a sentiment treebank. In EMNLP, 2013. find
- Trinh, T. H. and Le, Q. V. A simple method for commonsense reasoning. arXiv preprint arXiv:1806.02847, 2018. find
- Voorhees, E. M. and Tice, D. M. Building a question answering test collection. In SIGIR, 2000. find
- Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S. SuperGLUE: A stickier benchmark for general-purpose language understanding systems. In NeurIPS, 2019. find our card
- Yogatama, D., d’Autume, C. d. M., Connor, J., Kocisky, T., Chrzanowski, M., Kong, L., Lazaridou, A., Ling, W., Yu, L., Dyer, C., et al. Learning and evaluating general linguistic intelligence. arXiv preprint arXiv:1901.11373, 2019. find
- Zhang, X., Zhao, J., and LeCun, Y. Character-level convolutional networks for text classification. In NeurIPS, 2015. find our card
- A. Additional Results on Variance and Calibration Table 2 shows an example of the sensitivity to ordering. Prompt (test input not shown) Acc. Review: the whole thing ’s fairly lame , making it par for the course for disney sequels . Answer: Negative 88.5% Review: this quiet , introspective and enter… find