Elamur Research⌕
☰
home / papers / 2023 / In-Context Unlearning: Language Models as Few Shot Unlearners

In-Context Unlearning: Language Models as Few Shot Unlearners

Paper · 2023 · Martin Pawelczyk, Seth Neel and Himabindu Lakkaraju

Summary

Privacy rules such as the Right to be Forgotten, along with copyright takedown demands, can require removing the effect of particular training examples from a deployed model. Exact removal means retraining, which is too expensive for LLMs. Existing approximate methods still have to edit the weights, which is impossible when the model is only reachable through queries. The authors propose In-Context Unlearning (ICUL), which never touches the parameters. At inference time, the prompt starts with the examples to be forgotten, each given a randomly changed label, followed by correctly labelled samples drawn from the training distribution. To measure whether forgetting worked, they build LiRA-Forget, an adaptation of the LiRA membership inference attack that tries to separate unlearned points from genuinely unseen test points. On text classification and question answering, ICUL matches or sometimes outperforms Gradient Ascent, a weight-updating baseline, while largely keeping test accuracy. This makes ICUL a practical temporary fix for black-box models until a retrained version can be deployed.

Key points

  • Method: ICUL builds a context from the forget-set examples with flipped labels plus L correctly labelled examples. Setting L too low can cause unlearning to fail.
  • Datasets: Experiments use AG-News, SST-2 and Amazon reviews (classification) and SQUAD (question answering).
  • Evaluation: LiRA-Forget extends the LiRA MIA (Carlini et al., 2022). Similar metrics appeared concurrently (Kurmanji et al., 2023) and later as U-LiRA (Hayes et al., 2024). It needs many shadow models, which is costly for large LLMs.
  • Memory: On Llama-2 7B, ICUL fits on a 32GB Tesla V100, whereas Gradient Ascent needs an 80GB A100.
  • Limitations: In question answering, deleting 10 points at once cut accuracy by more than 15%. Possible prompt-stealing attacks on the unlearning context remain an open risk. The work was published at ICML 2024.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Martin Pawelczyk, Seth Neel and Himabindu Lakkaraju. 2023. In-Context Unlearning: Language Models as Few Shot Unlearners. arXiv:2310.07579.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2310.07579 · arXiv 2310.07579

@misc{pawelczyk2023context,
  title = {{In-Context Unlearning: Language Models as Few Shot Unlearners}},
  author = {Martin Pawelczyk, Seth Neel and Himabindu Lakkaraju},
  year = {2023},
  doi = {10.48550/arxiv.2310.07579},
  eprint = {2310.07579},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2310.07579}
}

References

49 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Garg, S., Tsipras, D., Liang, P. S., and Valiant, G. What can transformers learn in-context? a case study of simple function classes. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
  2. Belrose, N., Schneider-Joseph, D., Ravfogel, S., Cotterell, R., Raff, E., and Biderman, S. Leace: Perfect linear concept erasure in closed form. arXiv preprint arXiv:2306.03819, 2023. — [arXiv:2306.03819](https://arxiv.org/abs/2306.03819) find
  3. Ginart, A., Guan, M. Y., Valiant, G., and Zou, J. Making ai forget you: Data deletion in machine learning. In Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), 2019. find
  4. Biega, A. J., Potash, P., Daumé, H., Diaz, F., and Finck, M. Operationalizing the legal principle of data minimization for personalization. In ACM(43) SIGIR ’20, pp. 399–408, 2020. find
  5. Goel, S., Prabhu, A., Sanyal, A., Lim, S.-N., Torr, P., and Kumaraguru, P. Towards adversarial evaluations for inexact machine unlearning. arXiv preprint arXiv:2201.06640, 2022. — [arXiv:2201.06640](https://arxiv.org/abs/2201.06640) find
  6. Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., and Kasneci, G. Language models are realistic tabular data generators. In International Conference on Learning Representations (ICLR), 2023. find
  7. Golatkar, A., Achille, A., and Soatto, S. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020a. find
  8. Golatkar, A., Achille, A., and Soatto, S. Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations. arXiv:2003.02960, 2020b. — [arXiv:2003.02960](https://arxiv.org/abs/2003.02960) find
  9. Leemann, T., Pawelczyk, M., and Kasneci, G. Gaussian membership inference privacy. In Advances in neural information processing systems (NeurIPS), 2023. find
  10. Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., and Chen, W. What makes good in-context examples for GPT-3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, 2022.
  11. Golatkar, A., Achille, A., Ravichandran, A., Polito, M., and Soatto, S. Mixed-privacy forgetting in deep networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 792–801, 2021. find
  12. Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55, 2023.
  13. Goldsteen, A., Ezov, G., Shmelkin, R., Moffie, M., and Farkash, A. Data minimization for gdpr compliance in machine learning models. AI and Ethics, pp. 1–15, 2021. find
  14. Graves, L., Nagisetty, V., and Ganesh, V. Amnesiac machine learning. In Proceedings of the AAAI Conference on Artificial Intelligence, 2021. find
  15. Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., and Zettlemoyer, L. Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 11048– 11064, 2022.
  16. Guo, C., Goldstein, T., Hannun, A., and Van Der Maaten, L. Certified data removal from machine learning models. In International Conference on Machine Learing (ICML), 2019. find
  17. Morris, J. X., Zhao, W., Chiu, J. T., Shmatikov, V., and Rush, A. M. Language model inversion. arXiv preprint arXiv:2311.13647, 2023. — [arXiv:2311.13647](https://arxiv.org/abs/2311.13647) find
  18. Hayes, J., Shumailov, I., Triantafillou, E., Khalifa, A., and Papernot, N. Inexact unlearning needs more careful evaluations to avoid a false sense of privacy, 2024. find
  19. Nasr, M., Carlini, N., Hayase, J., Jagielski, M., Cooper, A. F., Ippolito, D., Choquette-Choo, C. A., Wallace, E., Tramèr, F., and Lee, K. Scalable extraction of training data from (production) language models. arXiv preprint arXiv:2311.17035, 2023.
  20. Henderson, P., Li, X., Jurafsky, D., Hashimoto, T., Lemley, M. A., and Liang, P. Foundation models and fair use. arXiv:2303.15715, 2023. find
  21. Huang, Y. and Canonne, C. L. Tight bounds for machine unlearning via differential privacy. arXiv:2309.00886, 2023. — [arXiv:2309.00886](https://arxiv.org/abs/2309.00886) find
  22. Neel, S., Roth, A., and Sharifi-Malvajerdi, S. Descent-to-delete: Gradient-based methods for machine unlearning. In Proceedings of the 32nd International Conference on Algorithmic Learning Theory (ALT), 2021. find
  23. Izzo, Z., Anne Smart, M., Chaudhuri, K., and Zou, J. Approximate data deletion from machine learning models. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics (AISTATS), 2021. find
  24. Neyman, J. and Pearson, E. S. Ix. on the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 231(694-706):289–337, 1933. find
  25. Jagielski, M., Thakkar, O., Tramer, F., Ippolito, D., Lee, K., Carlini, N., Wallace, E., Song, S., Thakurta, A., Papernot, N., et al. Measuring forgetting of memorized training examples. In International Conference on Learning Representations (ICLR), 2023. find
  26. OAG, C. Ccpa regulations: Final regulation text. Office of the Attorney General, California Department of Justice, 2021. find
  27. Jang, J., Yoon, D., Yang, S., Cha, S., Lee, M., Logeswaran, L., and Seo, M. Knowledge unlearning for mitigating privacy risks in language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, 2023.
  28. Pang, B. and Lee, L. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. arXiv preprint cs/0506075, 2005.
  29. Jia, J., Liu, J., Ram, P., Yao, Y., Liu, G., Liu, Y., Sharma, P., and Liu, S. Model sparsity can simplify machine unlearning, 2024. find
  30. Pawelczyk, M., Leemann, T., Biega, A., and Kasneci, G. On the trade-off between actionable explanations and the right to be forgotten. In International Conference on Learning Representations (ICLR), 2023. find
  31. Kurmanji, M., Triantafillou, P., and Triantafillou, E. Towards unbounded machine unlearning. arXiv preprint arXiv:2302.09880, 2023. — [arXiv:2302.09880](https://arxiv.org/abs/2302.09880) find
  32. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. find
  33. Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. SQuAD: 100,000+ questions for machine comprehension of text. In Su, J., Duh, K., and Carreras, X. (eds.), Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383–2392, Austin, Texas, November 2016. Associat…
  34. Thakker, U., Raunak, V., Tang, X., Yong, Z.-X., Sun, Z., Brody, S., Uri, Y., Tojarieh, H., Roberts, A., Chung, H. W., Tae, J., Phang, J., Press, O., Li, C., Narayanan, D., Bourfoune, H., Casper, J., Rasley, J., Ryabinin, M., Mishra, M., Zhang, M., Shoeybi, M., Peyrounette, M., Patry, N., Tazi, N., S… find
  35. Ravfogel, S., Twiton, M., Goldberg, Y., and Cotterell, R. D. Linear adversarial concept erasure. In International Conference on Machine Learning, pp. 18400–18421. PMLR, 2022a. find
  36. Ravfogel, S., Vargas, F., Goldberg, Y., and Cotterell, R. Adversarial concept erasure in kernel space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 6034–6055, 2022b. find
  37. Scao, T. L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., Castagné, R., Luccioni, A. S., Yvon, F., Gallé, M., Tow, J., Rush, A. M., Biderman, S., Webson, A., Ammanamanchi, P. S., Wang, T., Sagot, B., Muennighoff, N., del Moral, A. V., Ruwase, O., Bawden, R., Bekman, S., McMillan-Major, A… find
  38. Zhang, X., Zhao, J., and LeCun, Y. Character-level convolutional networks for text classification. In Advances in neural information processing systems (NeurIPS), volume 28, 2015.
  39. Shokri, R., Stronati, M., Song, C., and Shmatikov, V. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pp. 3–18.
  40. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., and Potts, C. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 2013.
  41. Tirumala, K., Markosyan, A., Zettlemoyer, L., and Aghajanyan, A. Memorization without overfitting: Analyzing the training dynamics of large language models. In Advances in Neural Information Processing Systems, 2022.
  42. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.
  43. Tsybakov, A. B. Introduction to Nonparametric Estimation. Springer Publishing Company, Incorporated, 1st edition, 2008. ISBN 0387790519. find
  44. Union, E. Regulation (eu) 2016/679 of the european parliament and of the council. Official Journal of the European Union, 2016. find
  45. Voigt, P. and Von dem Bussche, A. The eu general data protection regulation (gdpr). A Practical Guide, 1st Ed., Cham: Springer International Publishing, 10:3152676, 2017. find
  46. Wang, L., Chen, T., Yuan, W., Zeng, X., Wong, K.-F., and Yin, H. Kga: A general machine unlearning framework based on knowledge gap alignment. arXiv preprint arXiv:2305.06535, 2023. find
  47. Wei, J., Wei, J., Tay, Y., Tran, D., Webson, A., Lu, Y., Chen, X., Liu, H., Huang, D., Zhou, D., et al. Larger language models do in-context learning differently. arXiv:2303.03846, 2023.
  48. Wu, Y., Dobriban, E., and Davidson, S. Deltagrad: Rapid retraining of machine learning models. In International Conference on Machine Learning (ICML), 2020. find
  49. Zhang, R. and Zhang, S. Rethinking influence functions of neural networks in the over-parameterized regime. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2021. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pawelczyk-2023-context-unlearning-language-models-few/