Elamur Research⌕
☰
home / papers / 2023 / Who's Harry Potter? Approximate Unlearning in LLMs

Who's Harry Potter? Approximate Unlearning in LLMs

Paper · 2023 · Ronen Eldan and Mark Russinovich

Summary

LLMs absorb copyrighted and other problematic material during pretraining, and retraining from scratch to remove it is prohibitively expensive. Eldan and Russinovich propose an approximate unlearning method that makes Llama2-7b behave as though it had never read the Harry Potter books. Naively reversing the training loss works poorly, so the method instead builds substitute next-token targets that a model unfamiliar with the series might plausibly produce.

The method combines two sources of these "generic" targets:

  • Reinforcement bootstrapping: the authors fine-tune a copy of the model further on the books. Tokens whose logits rise in that copy are treated as book-specific and suppressed.
  • Anchored terms: distinctive names and phrases, identified with help from GPT-4, are replaced with neutral equivalents. The model's own predictions on the altered text then serve as labels.

Fine-tuning on these labels for about one GPU hour largely removes the model's Harry Potter knowledge, while standard benchmark scores barely change. This is presented as an early demonstration that targeted forgetting in generative LLMs is feasible. The authors note that it may transfer poorly to non-fiction and could fail under adversarial probing.

Key points

  • Compute cost: pretraining Llama2-7b took over 184K GPU-hours, while unlearning took roughly 1 GPU hour of fine-tuning (about 30 minutes on 4 A100s for the example model). The released model is microsoft/Llama2-7b-WhoIsHarryPotter on Hugging Face.
  • Benchmark impact (baseline → fine-tuned): losses are small.
    • ARC-C: 0.439 → 0.416
    • ARC Easy: 0.744 → 0.728
    • BoolQ: 0.807 → 0.798
    • Hellaswag: 0.577 → 0.560
    • OpenBookQA: 0.338 → 0.334
    • PIQA: 0.767 → 0.762
    • Winogrande: 0.663 → 0.665
  • Familiarity scores: the completion-based score falls from 0.290 to about 0.007, and the probability-based score falls from 0.244 to about 0.01. For the prompt "Harry Potter studies", the probability of "magic" drops from 0.22 to 0 within 120 steps.
  • Ablation:
    • Bootstrapping alone never cut familiarity by more than a factor of 0.3.
    • Anchored terms alone matched the familiarity score but hurt benchmarks more (e.g., Hellaswag 0.54, Winogrande 0.61).
  • Limitations:
    • The method depends on GPT-4's prior knowledge of the series.
    • Some general, Wikipedia-level facts about the franchise remain.
    • Its effectiveness on non-fiction and against adversarial extraction is untested.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MicrosoftRedmond · United States378 works · 161,630 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Ronen Eldan and Mark Russinovich. 2023. Who's Harry Potter? Approximate Unlearning in LLMs. arXiv:2310.02238.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2310.02238 · arXiv 2310.02238

@misc{eldan2023who,
  title = {{Who's Harry Potter? Approximate Unlearning in LLMs}},
  author = {Ronen Eldan and Mark Russinovich},
  year = {2023},
  doi = {10.48550/arxiv.2310.02238},
  eprint = {2310.02238},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2310.02238}
}

References

14 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Y Chai, Mirella Lapata, Angeliki Lazaridou, Ryan J Maynez, Piyush Narang, et al. Piqa: Reasoning about physical commonsense in natural language. arXiv preprint arXiv:1911.11641, 2019.
  2. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human…
  3. Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, Jason Phang, Laria Reynolds, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, September… find
  4. Yiwen Jiang, Shenglong Liu, Tao Zhao, Wei Li, and Xianzhou Gao. Machine unlearning survey. In Fifth International Conference on Mechatronics and Computer Technology Engineering (MCTE 2022), volume 12500, pages 1596–1603. SPIE, 2022. find
  5. Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models. arXiv preprint arXiv:2210.01504, 2022.
  6. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789, 2018. find
  7. Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. A survey of machine unlearning. arXiv preprint arXiv:2209.02299, 2022. — [arXiv:2209.02299](https://arxiv.org/abs/2209.02299) find
  8. J.K. Rowling. The Harry Potter Series. Bloomsbury, 1997-2007. Comprising: Harry Potter and the Philosopher’s Stone; Harry Potter and the Chamber of Secrets; Harry Potter and the Prisoner of Azkaban; Harry Potter and the Goblet of Fire; Harry Potter and the Order of the Phoenix; Harry Potter and the… find
  9. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. arXiv preprint arXiv:1907.10641, 2019. find
  10. Lingzhi Wang, Tong Chen, Wei Yuan, Xingshan Zeng, Kam-Fai Wong, and Hongzhi Yin. Kga: A general machine unlearning framework based on knowledge gap alignment. arXiv preprint arXiv:2305.06535, 2023. find
  11. Vikas Yadav, Steven Bethard, and Mihai Surdeanu. Quick and (not so) dirty: Unsupervised selection of justification sentences for multi-hop question answering. arXiv preprint arXiv:1911.07176, 2019. find
  12. Dawen Zhang, Pamela Finckenberg-Broman, Thong Hoang, Shidong Pan, Zhenchang Xing, Mark Staples, and Xiwei Xu. Right to be forgotten in the era of large language models: Implications, challenges, and solutions. arXiv preprint arXiv:2307.03941, 2023. — [arXiv:2307.03941](https://arxiv.org/abs/2307.03941) find
  13. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791–4800, 2019. find
  14. Haibo Zhang, Toru Nakamura, Takamasa Isohara, and Kouichi Sakurai. A review on machine unlearning. SN Computer Science, 4(4):337, 2023. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/eldan-2023-who-s-harry-potter-approximate/