Elamur Research⌕
☰
home / papers / 2026 / Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models

Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models

Paper · 2026 · Huazheng Wang, Yongcheng Jing, Haifeng Sun et al.

Summary

The paper tackles a weakness in LLM machine unlearning: current techniques tend to suppress only the literal wording of the data to be forgotten. Knowledge connected to it can still be recovered. The authors define a wider unlearning scope that covers the target samples plus rephrased questions, subject-replaced queries, relation-reversed questions and one-hop reasoning variants. They then benchmark 15 existing methods on TOFU, Harry Potter and ZsRE using Phi-1.3B and LLaMA2-7B. Generalisation is consistently poor, for two reasons: unlearned models still encode target facts in intermediate layers, and they can still produce paraphrased answers.

To address this, they introduce PERMU, a probability-perturbation method. It identifies the most sensitive tokens with a new metric, MSM, and adds noise to their embeddings. It then subtracts the original next-token distribution from the perturbed one and fine-tunes the model toward that residual, which lowers the probability of answer-related tokens. The work matters for privacy and safety, where incomplete forgetting can leak protected information.

Key points

  • PERMU improves forgetting of target data by up to 50.40% and implicit-knowledge forgetting by up to 40.73% (Forget Probability on Rephrased TOFU). Truth Ratio improves by 15.67%.
  • The authors propose a new trade-off metric, Forget–Retain Trade-off (FRT). On WMDP (Vicuna-13B) and MUSE (MUSE-7B), PERMU has the best FRT, rising from 0.65 to 0.72 (+10.77%) on documents of up to 1,031 tokens.
  • On ZsRE, PERMU leads on FRT among non-ICL methods and gains up to 9.41% on reversed-relation data.
  • Forgetting of one-hop reasoning knowledge remains weak for every method, falling up to 2.23× short of the Retain model.
  • The token-level gradients PERMU needs add computational cost; the authors offer a fast variant to reduce it. Code: github.com/MaybeLizzy/PERMU.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Nanyang Technological UniversitySingapore · Singapore156 works · 25,644 citations🎓Beijing University of Posts and TelecommunicationsBeijing · China57 works · 32,843 citations🔬State Key Laboratory of Networking and Switching TechnologyBeijing · China5 works · 133 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Huazheng Wang, Yongcheng Jing, Haifeng Sun, Yingjie Wang, Jingyu Wang, Jianxin Liao and Dacheng Tao. 2026. Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.88

@inproceedings{wang2026erasing,
  title = {{Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models}},
  author = {Huazheng Wang, Yongcheng Jing, Haifeng Sun, Yingjie Wang, Jingyu Wang, Jianxin Liao and Dacheng Tao},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.88},
  url = {https://doi.org/10.18653/v1/2026.acl-long.88}
}

References

64 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shun-ichi Amari and Hiroshi Nagaoka. 2000. Methods of information geometry, volume 191. American Mathematical Soc. find
  2. Yinzhi Cao and Junfeng Yang. 2015. Towards making systems forget with machine unlearning. In 2015 IEEE Symposium on Security and Privacy, SP 2015, San Jose, CA, USA, May 17-21, 2015, pages 463–480. IEEE Computer Society. find
  3. Jiaao Chen and Diyi Yang. 2023. Unlearn what you want to forget: Efficient unlearning for llms. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 12041– 12052. Association for Computational Linguistics.
  4. Zigeng Chen, Gongfan Fang, Xinyin Ma, Ruonan Yu, and Xinchao Wang. 2026. dparallel: Learnable parallel decoding for dllms. In International Conference on Learning Representations. find
  5. Minseok Choi, Daniel Rim, Dohyun Lee, and Jaegul Choo. 2024. SNAP: unlearning selective knowledge in large language models with negative instructions. CoRR, abs/2406.12329. — [arXiv:2406.12329](https://arxiv.org/abs/2406.12329) find
  6. Minseok Choi, Daniel Rim, Dohyun Lee, and Jaegul Choo. 2025. Opt-out: Investigating entity-level unlearning for large language models via optimal transport. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, pages 28280– 2829… find
  7. Rishav Chourasia and Neil Shah. 2023. Forget unlearning: Towards true data-deletion in machine learning. In International Conference on Machine Learning, ICML 2023, volume 202 of Proceedings of Machine Learning Research, pages 6028–6073. PMLR. find
  8. Huu-Tien Dang, Hoang Thanh-Tung, Le-Minh Nguyen, and Naoya Inoue. 2025. Improving the robustness of representation misdirection for large language model unlearning. CoRR, abs/2501.19202. — [arXiv:2501.19202](https://arxiv.org/abs/2501.19202) find
  9. Jiacheng Du, Zhibo Wang, and Kui Ren. 2024. Textual unlearning gives a false sense of unlearning. CoRR, abs/2406.13348. — [arXiv:2406.13348](https://arxiv.org/abs/2406.13348) find
  10. Ronen Eldan and Mark Russinovich. 2023. Who’s harry potter? approximate unlearning in llms. CoRR, abs/2310.02238. find
  11. Chongyu Fan, Jiancheng Liu, Licong Lin, Jinghan Jia, Ruiqi Zhang, Song Mei, and Sijia Liu. 2024. Simplicity prevails: Rethinking negative preference optimization for LLM unlearning. CoRR, abs/2410.07163. — [arXiv:2410.07163](https://arxiv.org/abs/2410.07163) find
  12. Xiaohua Feng, Chaochao Chen, Yuyuan Li, and Zibin Lin. 2024. Fine-grained pluggable gradient ascent for knowledge unlearning in language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, pages 10141–10155. find
  13. Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. Dissecting recall of factual associations in auto-regressive language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 12216– 122… find
  14. Antonio Ginart, Melody Y. Guan, Gregory Valiant, and James Zou. 2019. Making AI forget you: Data deletion in machine learning. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, pages 3513–3526. find
  15. Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. 2024. Model editing can hurt general abilities of large language models. CoRR, abs/2401.04700. — [arXiv:2401.04700](https://arxiv.org/abs/2401.04700) find
  16. Phillip Guo, Aaquib Syed, Abhay Sheshadri, Aidan Ewart, and Gintare Karolina Dziugaite. 2024. Mechanistic unlearning: Robust knowledge unlearning and editing via mechanistic localization. CoRR, abs/2410.12949. — [arXiv:2410.12949](https://arxiv.org/abs/2410.12949) find
  17. Yihuai Hong, Lei Yu, Shauli Ravfogel, Haiqin Yang, and Mor Geva. 2024a. Intrinsic evaluation of unlearning using parametric knowledge traces. CoRR, abs/2406.11614. — [arXiv:2406.11614](https://arxiv.org/abs/2406.11614) find
  18. Yihuai Hong, Yuelin Zou, Lijie Hu, Ziqian Zeng, Di Wang, and Haiqin Yang. 2024b. Dissecting fine-tuning unlearning in large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, pages 3933–3941. find
  19. Xinshuo Hu, Dongfang Li, Baotian Hu, Zihao Zheng, Zhenyu Liu, and Min Zhang. 2024. Separate the wheat from the chaff: Model deficiency unlearning via parameter-efficient module operation. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, pages 18252–18260. find
  20. Gabriel Ilharco, Marco Túlio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations, ICLR 2023. OpenReview.net.
  21. Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. 2023. Knowledge unlearning for mitigating privacy risks in language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2…
  22. Jiabao Ji, Yujian Liu, Yang Zhang, Gaowen Liu, Ramana Rao Kompella, Sijia Liu, and Shiyu Chang. 2024. Reversing the forget-retain objectives: An efficient LLM unlearning framework from logit difference. CoRR, abs/2406.08607. — [arXiv:2406.08607](https://arxiv.org/abs/2406.08607) find
  23. Jinghan Jia, Yihua Zhang, Yimeng Zhang, Jiancheng Liu, Bharat Runwal, James Diffenderfer, Bhavya Kailkhura, and Sijia Liu. 2024. SOUL: unlocking the power of second-order optimization for LLM unlearning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP…
  24. Zhuoran Jin, Pengfei Cao, Chenhao Wang, Zhitao He, Hongbang Yuan, Jiachun Li, Yubo Chen, Kang Liu, and Jun Zhao. 2024. RWKU: benchmarking real-world knowledge unlearning for large language models. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Proces… find
  25. Dohyun Lee, Daniel Rim, Minseok Choi, and Jaegul Choo. 2024. Protecting privacy through approximating optimal parameters for sequence unlearning in language models. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, page… find
  26. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. Zero-shot relation extraction via reading comprehension. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), Vancouver, Canada, August 3-4, 2017, pages 333–342.
  27. Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen, Andrew B. Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xiaoyuan Zhu, Rishub Tamirisa, an… find
  28. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
  29. Chris Yuhao Liu, Yaxuan Wang, Jeffrey Flanigan, and Yang Liu. 2024a. Large language model unlearning via embedding-corrupted prompts. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024. find
  30. Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Xiaojun Xu, Yuguang Yao, Hang Li, Kush R. Varshney, Mohit Bansal, Sanmi Koyejo, and Yang Liu. 2024b. Rethinking machine unlearning for large language models. CoRR, abs/2402.08787. find
  31. Jakub Lucki, Boyi Wei, Yangsibo Huang, Peter Henderson, Florian Tramèr, and Javier Rando. 2025. An adversarial perspective on machine unlearning for AI safety. find
  32. Trans. Mach. Learn. Res., 2025. find
  33. Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C. Lipton, and J. Zico Kolter. 2024. TOFU: A task of fictitious unlearning for llms. CoRR, abs/2401.06121.
  34. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022. find
  35. OpenAI. 2023. GPT-4 technical report. abs/2303.08774.
  36. CoRR, Vaidehi Patil, Peter Hase, and Mohit Bansal. 2024. Can sensitive information be deleted from llms? objectives for defending against extraction attacks. In The Twelfth International Conference on Learning Representations, ICLR 2024. find
  37. Martin Pawelczyk, Jimmy Z. Di, Yiwei Lu, Gautam Kamath, Ayush Sekhari, and Seth Neel. 2024a. Machine unlearning fails to remove data poisoning attacks. CoRR, abs/2406.17216. — [arXiv:2406.17216](https://arxiv.org/abs/2406.17216) find
  38. Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. 2024b. In-context unlearning: Language models as few-shot unlearners. In Forty-first International Conference on Machine Learning, ICML 2024,. find
  39. Xinchi Qiu, William F. Shen, Yihong Chen, Nicola Cancedda, Pontus Stenetorp, and Nicholas D. Lane. 2024. PISTOL: dataset compilation pipeline for structural unlearning of llms. CoRR, abs/2406.16810. — [arXiv:2406.16810](https://arxiv.org/abs/2406.16810) find
  40. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Sy…
  41. Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. Quantifying language models’ sensitivity to spurious features in prompt design or: How I learned to start worrying about prompt formatting. In The Twelfth International Conference on Learning Representations, ICLR 2024. find
  42. Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A. Smith, and Chiyuan Zhang. 2024. MUSE: machine unlearning six-way evaluation for language models. CoRR, abs/2407.06460. find
  43. Zesheng Shi, Yucheng Zhou, Jing Li, Yuxin Jin, Yu Li, Daojing He, Fangming Liu, Saleh Alharbi, Jun Yu, and Min Zhang. 2025. Safety alignment via constrained knowledge unlearning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2…
  44. Nianwen Si, Hao Zhang, Heyu Chang, Wenlin Zhang, Dan Qu, and Weiqiang Zhang. 2023. Knowledge unlearning for llms: Tasks, methods, and challenges. CoRR, abs/2311.15766. — [arXiv:2311.15766](https://arxiv.org/abs/2311.15766) find
  45. Haoyu Tang, Ye Liu, Xukai Liu, Kai Zhang, Yanghai Zhang, Qi Liu, and Enhong Chen. 2024. Learn while unlearn: An iterative unlearning framework for generative language models. CoRR, abs/2407.20271. — [arXiv:2407.20271](https://arxiv.org/abs/2407.20271) find
  46. Pratiksha Thaker, Yash Maurya, and Virginia Smith. 2024. Guardrail baselines for unlearning in llms. CoRR, abs/2403.03329. — [arXiv:2403.03329](https://arxiv.org/abs/2403.03329) find
  47. Bozhong Tian, Xiaozhuan Liang, Siyuan Cheng, Qingbin Liu, Mengru Wang, Dianbo Sui, Xi Chen, Huajun Chen, and Ningyu Zhang. 2024. To forget or not? towards practical knowledge unlearning for large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1524–15… find
  48. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. CoRR, ab…
  49. Fanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan, Wei Bi, and Shuming Shi. 2024. Knowledge fusion of large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024. find
  50. Bichen Wang, Yuzhe Zi, Yixin Sun, Yanyan Zhao, and Bing Qin. 2024a. RKLD: reverse kl-divergence-based knowledge distillation for unlearning personal information in large language models. CoRR, abs/2406.01983.
  51. Huazheng Wang, Haifeng Sun, Jingyu Wang, Qi Qi, Zixuan Xia, Menghao Zhang, and Jianxin Liao. 2024b. SSS: editing factual knowledge in language models towards semantic sparse space. In Findings of the Association for Computational Linguistics, ACL 2024, pages 5559–5570. find
  52. Huazheng Wang, Jinming Wu, Haifeng Sun, Zixuan Xia, Daixuan Cheng, Jingyu Wang, Qi Qi, and Jianxin Liao. 2024c. MDR: model-specific demonstration retrieval at inference time for in-context learning. In NAACL 2024, (Volume 1: Long Papers), Mexico City, Mexico, June 16-21, 2024, pages 4189–4204. Assoc… find
  53. Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. 2024a. Negative preference optimization: From catastrophic collapse to effective unlearning. CoRR, abs/2404.05868.
  54. Yaxuan Wang, Jiaheng Wei, Chris Yuhao Liu, Jinlong Pang, Quan Liu, Ankit Shah, Yujia Bao, Yang Liu, and Wei Wei. 2025. LLM unlearning via loss adjustment with only forget data. In The Thirteenth International Conference on Learning Representations, ICLR 2025. find
  55. Zhiwei Zhang, Fali Wang, Xiaomin Li, Zongyu Wu, Xianfeng Tang, Hui Liu, Qi He, Wenpeng Yin, and Suhang Wang. 2024b. Does your LLM truly unlearn? an embarrassingly simple approach to recover unlearned knowledge. CoRR, abs/2410.16454. — [arXiv:2410.16454](https://arxiv.org/abs/2410.16454) find
  56. Yuqiao Wen, Zichao Li, Wenyu Du, and Lili Mou. 2023. f-divergence minimization for sequence-level knowledge distillation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, pages 10817–10834. Association for Computational Lin…
  57. Ruihan Wu, Chhavi Yadav, Russ Salakhutdinov, and Kamalika Chaudhuri. 2024. Evaluating deep unlearning in large language models. CoRR, abs/2410.15153. — [arXiv:2410.15153](https://arxiv.org/abs/2410.15153) find
  58. Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. 2023. DEPN: detecting and editing privacy neurons in pre-trained language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10,… find
  59. Haoming Xu, Ningyuan Zhao, Liming Yang, Sendong Zhao, Shumin Deng, Mengru Wang, Bryan Hooi, Nay Oo, Huajun Chen, and Ningyu Zhang. 2025. Relearn: Unlearning via learning for large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: L… find
  60. Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, and Xiang Yue. 2024. Machine unlearning of pre-trained large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, pages 8403–8419.
  61. Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023. Editing large language models: Problems, methods, and opportunities. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, pages 10222–1024… find
  62. Jinghan Zhang, Shiqi Chen, Junteng Liu, and Junxian He. 2023. Composing parameter-efficient modules with arithmetic operations. CoRR, abs/2306.14870. — [arXiv:2306.14870](https://arxiv.org/abs/2306.14870) find
  63. Menghao Zhang, Huazheng Wang, Pengfei Ren, Kangheng Lin, Qi Qi, Haifeng Sun, Zirui Zhuang, Lei Zhang, Jianxin Liao, and Jingyu Wang. 2025. Do lvlms truly understand video anomalies? revealing hallucination via co-occurrence patterns. In Advances in Neural Information Processing Systems. find
  64. Chenxiao Zhao, P. Thomas Fletcher, Mixue Yu, Yaxin Peng, Guixu Zhang, and Chaomin Shen. 2019. The adversarial attack and detection under the fisher information metric. In AAAI2019, IAAI 2019, EAAI 2019, pages 5869–5876. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-erasing-without-remembering-implicit-knowledge/