Elamur Research⌕
☰
home / papers / 2026 / LDEDE: LRP-Driven Efficient Detection and Editing Framework for LLM Privacy Neurons

LDEDE: LRP-Driven Efficient Detection and Editing Framework for LLM Privacy Neurons

Paper · 2026 · Zhao Zhengyuan, Cao Lifeng, Sunhaodong et al.

Summary

LLMs can memorize personal data from their training corpora, such as phone numbers, emails and clinical details, and reveal it when prompted. Existing neuron-editing defences face two problems. Finding privacy neurons through gradient attribution is accurate but expensive. Suppressing one kind of private information can also raise leakage of another, an effect the authors call the "privacy seesaw."

LDEDE replaces gradient attribution with Layer-wise Relevance Propagation (LRP) over FFN neurons. It aggregates relevance across multiple tokens and votes over many samples. This lowers localization cost from O(N²) to O(N) while matching gradient-based accuracy. The authors also report "coupled privacy neurons," which increase leakage for one privacy type while suppressing it for another. They identify these as the source of the seesaw effect. Their Polarity-Aware Neuron Editing (PANE) therefore edits neurons differently depending on the sign of their LRP score.

On Enron and MIMIC data, LDEDE reduces leakage substantially while keeping general performance comparable to the baselines, and it runs much faster. The paper presents it as a lightweight option for privacy protection after deployment.

Key points

  • Leakage and speed: Against DP, DEPN and APNEAP, LDEDE lowers average leakage risk by 42.7%–73.5% and cuts computation time by 60%–90%. The paper is inconsistent about the privacy scenarios: the abstract lists Phone, Email and medical, while the conclusion lists Phone, Email and Name.
  • Ablation (NAME scenario):
    • LRP+PANE reaches the lowest MRR, 42.12 versus 53.24 for the unedited model.
    • It finishes in 5 min 35 s, compared with 19–26 min for the gradient-attribution (GA) variants.
    • It produces 92 seesaw samples, compared with 477 for GA+ZERO.
  • Model coverage: Results hold across GPT-2, BERT-base and LLAMA-7B, as well as GPT-2 models of different sizes. LLAMA-7B takes about 13 min 17 s to process.
  • Effect of memorization strength: Protection is strongest for strongly memorized samples. In the Phone scenario, high-memory samples see more than five times the risk reduction of low-memory ones.
  • Limitations and code: The method is tied to Transformers and the metrics cover only text. Code is available at github.com/YS-SM/LDEDE.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

Cite

Cite as Zhao Zhengyuan, Cao Lifeng, Sunhaodong, Shi Haotian, Du Xuehui, Liu Aodi, Niu Lanjie and Yang Xiaocheng. 2026. LDEDE: LRP-Driven Efficient Detection and Editing Framework for LLM Privacy Neurons. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1666

@inproceedings{zhengyuan2026ldede,
  title = {{LDEDE: LRP-Driven Efficient Detection and Editing Framework for LLM Privacy Neurons}},
  author = {Zhao Zhengyuan, Cao Lifeng, Sunhaodong, Shi Haotian, Du Xuehui, Liu Aodi, Niu Lanjie and Yang Xiaocheng},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1666},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1666}
}

References

43 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alberto Blanco-Justicia, Najeeb Jebreel, Benet Manzanares-Salor, David Sánchez, Josep Domingo-Ferrer, Guillem Collell, and Kuan Eeik Tan. 2025. Digital forgetting in large language models: a survey of unlearning methods. Artif. Intell. Rev., 58(3):90. find
  2. Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, and Colin Raffel. 2021. Extracting training data from large language models. In 30th USENIX Security Symposium, USENIX Security…
  3. Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2024. Evaluating the ripple effects of knowledge editing in language models. Trans. Assoc. Comput. Linguistics, 12:283–298. find
  4. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 8493–8502.… find
  5. Sunhao Dai, Chen Xu, Shicheng Xu, Liang Pang, Zhenhua Dong, and Jun Xu. 2024. Bias and unfairness in information retrieval systems: New challenges in the LLM era. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, August 25-29, 2024,… find
  6. Arpit Garg, Hemanth Saratchandran, Ravi Garg, and Simon Lucey. 2025. Stable forgetting: Bounded parameter-efficient unlearning in llms. CoRR, abs/2509.24166. — [arXiv:2509.24166](https://arxiv.org/abs/2509.24166) find
  7. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 548… find
  8. Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. 2024. find
  9. Model editing harms general abilities of large language models: Regularization to the rescue. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, pages 16801–16819. Association for Computational Linguistics.
  10. Anna Namrita Gummadi, Osvaldo Arreche, and Mustafa Abdallah. 2025. A systematic evaluation of white-box explainable AI methods for anomaly detection in iot systems. Internet Things, 30:101505. find
  11. Seung-Ho Han and Ho-Jin Choi. 2024. CLAM: Class-wise layer-wise attribute model for explaining neural networks. find
  12. Yu He, Boheng Li, Liu Liu, Zhongjie Ba, Wei Dong, Yiming Li, Zhan Qin, Kui Ren, and Chun Chen. 2025. Towards label-only membership inference attack against pre-trained large language models. In 34th USENIX Security Symposium, USENIX Security 2025, Seattle, WA, USA, August 13-15, 2025, pages 1609–162… find
  13. An Huang, Zhipeng Cai, and Zuobin Xiong. 2025a. A survey of machine unlearning in generative AI models: Methods, applications, security, and challenges. IEEE Internet Things J., 12(16):32563–32580. find
  14. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025b. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst., 43(2):42:1–42:55…
  15. Ranjit Kumar, Ravi Prakash, Narendra Kumar, Suchismita Chinara, and Mazin Abed Mohammed. 2025. Machine unlearning for trustworthy ai: A systematic review of techniques, challenges, and applications. Archives of Computational Methods in Engineering. find
  16. Hongkang Li, Yihua Zhang, Shuai Zhang, Pin-Yu Chen, Sijia Liu, and Meng Wang. 2025a. When is task vector provably effective for model editing? a generalization analysis of nonlinear transformers. In The Thirteenth International Conference on Learning Representations. find
  17. Na Li, Chunyi Zhou, Yansong Gao, Hui Chen, Zhi Zhang, Boyu Kuang, and Anmin Fu. 2025b. Machine unlearning: Taxonomy, metrics, applications, challenges, and prospects. IEEE Trans. Neural Networks Learn. Syst., 36(8):13709–13729. find
  18. Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. 2024. PMET: precise model editing in a transformer. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Sympo…
  19. Yansong Li, Zhixing Tan, and Yang Liu. 2023. Privacy-preserving prompt tuning for large language model services. CoRR, abs/2305.06212. — [arXiv:2305.06212](https://arxiv.org/abs/2305.06212) find
  20. Xuanqi Liu and Zhuotao Liu. 2023. Llms can understand encrypted prompt: Towards privacy-computing friendly transformers. CoRR, abs/2305.18396. — [arXiv:2305.18396](https://arxiv.org/abs/2305.18396) find
  21. Kun Wang, Guibin Zhang, Zhenhong Zhou, Jiahao Wu, Miao Yu, Shiqian Zhao, Chenlong Yin, Jinhu Fu, Yibo Yan, Hanjun Luo, Liang Lin, Zhihao Xu, Haolang Lu, Xinye Cao, Xinyun Zhou, Weifei Jin, Fanci Meng, Junyuan Mao, Hao Wu, and 63 others. 2025a. A comprehensive survey in llm(-agent) full stack safety:… — [arXiv:2504.15585](https://arxiv.org/abs/2504.15585) find
  22. Jinglong Luo, Zhuo Zhang, Yehong Zhang, Shiyu Liu, Ye Dong, Hui Wang, Yue Yu, Xun Zhou, and Zenglin Xu. 2025. Secp-tuning: Efficient privacy-preserving prompt tuning for large language models via mpc. find
  23. Liyuan Wang, Jingyi Xie, Xingxing Zhang, Hang Su, and Jun Zhu. 2025b. Hide-pet: Continual learning via hierarchical decomposition of parameter-efficient tuning. IEEE Trans. Pattern Anal. Mach. Intell., 47(8):6687–6702. find
  24. Yifan Niu, Miao Peng, Nuo Chen, Yatao Bian, Tingyang Xu, and Jia Li. 2025. Reledit: Evaluating conceptual knowledge editing in language models via relational reasoning. In Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 - August 1, 2025, pages 10220–1023… find
  25. Teng Wang, Lindong Zhai, Tengfei Yang, Zhucheng Luo, and Shuanggen Liu. 2024. Selective privacy-preserving framework for large language models fine-tuning. Inf. Sci., 678:121000. find
  26. Chen Qian, Dongrui Liu, Jie Zhang, Yong Liu, and Jing Shao. 2025. The tug of war within: Mitigating the fairness-privacy conflicts in large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria,… find
  27. Zihan Qiu, Zeyu Huang, Youcheng Huang, and Jie Fu. 2024. Empirical study on updating key-value memories in transformer feed-forward layers. In The Second Tiny Papers Track at ICLR 2024, Tiny Papers @ ICLR 2024, Vienna, Austria, May 11, 2024. OpenReview.net. find
  28. Jialei Shi, Kostis Gourgoulias, John F. Buford, Sean J. Moran, and Najah Ghalyan. 2024. Deepclean: Machine unlearning on the cheap by resetting privacy sensitive weights using the fisher diagonal. In Computer Vision - ECCV 2024 Workshops - Milan, Italy, September 29-October 4, 2024, Proceedings, Par… find
  29. Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, and Ji-Rong Wen. 2024. Language-specific neurons: The key to multilingual capabilities in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volu…
  30. Elena Voita, Javier Ferrando, and Christoforos Nalmpantis. 2024. Neurons in large language models: Dead, n-gram, positional. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, pages 1288–1301. Association for Computation… find
  31. Skyler Wu, Eric Meng Shen, Charumathi Badrinath, Jiaqi Ma, and Himabindu Lakkaraju. 2023a. Analyzing chain-of-thought prompting in large language models via gradient-based feature attributions. CoRR, abs/2307.13339.
  32. Xinwei Wu, Weilong Dong, Shaoyang Xu, and Deyi Xiong. 2024. Mitigating privacy seesaw in large language models: Augmented privacy neuron editing via activation patching. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024,… find
  33. Xinwei Wu, Li Gong, and Deyi Xiong. 2022. Adaptive differential privacy for language model training. In Proceedings of the First Workshop on Federated Learning for Natural Language Processing (FL4NLP 2022), page 21–26. Association for Computational Linguistics. find
  34. Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. 2023b. DEPN: detecting and editing privacy neurons in pretrained language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10,… find
  35. Shujun Xia, Haokun Lin, Yichen Wu, Yinan Zhou, Zixuan Li, Zhongwei Wan, Xingrun Xing, Yefeng Zheng, Xiang Li, Caifeng Shan, Zhenan Sun, and Quanzheng Li. 2025. Medrek: Retrieval-based editing for medical llms with key-aware prompts. CoRR, abs/2510.13500. — [arXiv:2510.13500](https://arxiv.org/abs/2510.13500) find
  36. Guang Yan, Yuhui Zhang, Zimu Guo, Lutan Zhao, Xiaojun Chen, Chen Wang, Wenhao Wang, Dan Meng, and Rui Hou. 2025. Comet: Accelerating private inference for large language model by predicting activation sparsity. In IEEE Symposium on Security and Privacy, SP 2025, San Francisco, CA, USA, May 12-15, 20… find
  37. Yunzhi Yao, Ningyu Zhang, Zekun Xi, Mengru Wang, Ziwen Xu, Shumin Deng, and Huajun Chen. 2024. Knowledge circuits in pretrained transformers. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada,… find
  38. Jingwen Ye, Yifang Fu, Jie Song, Xingyi Yang, Songhua Liu, Xin Jin, Mingli Song, and Xinchao Wang. 2022. Learning with recoverable forgetting. In Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XI, volume 13671 of Lecture Notes in Comp… find
  39. Lang Yu, Qin Chen, Jie Zhou, and Liang He. 2024. MELO: enhancing model editing with neuron-indexed dynamic lora. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on E… find
  40. Dawen Zhang, Shidong Pan, Thong Hoang, Zhenchang Xing, Mark Staples, Xiwei Xu, Lina Yao, Qinghua Lu, and Liming Zhu. 2024a. To be forgotten or to be fair: unveiling fairness implications of machine unlearning methods. AI Ethics, 4(1):83–93. find
  41. Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. 2024b. Negative preference optimization: From catastrophic collapse to effective unlearning. CoRR, abs/2404.05868.
  42. Ruoyan Zhang, Zhongxiang Zheng, and Wankang Bao. 2025. Practical secure inference algorithm for fine-tuned large language model based on fully homomorphic encryption. CoRR, abs/2501.01672. — [arXiv:2501.01672](https://arxiv.org/abs/2501.01672) find
  43. Zayd Muhammad Kawakibi Zuhri, Muhammad Farid Adilazuarda, Ayu Purwarianti, and Alham Fikri Aji. 2025. MLKV: multi-layer key-value heads for memory efficient transformer decoding. In Findings of the Association for Computational Linguistics: NAACL 2025, Albuquerque, New Mexico, USA, April 29 - May 4,… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhengyuan-2026-ldede-lrp-driven-efficient-detection/