Elamur Research⌕
☰
home / papers / 2026 / DKME: Rethinking Coupled Knowledge Memory for Lifelong Model Editing of Large Language Models

DKME: Rethinking Coupled Knowledge Memory for Lifelong Model Editing of Large Language Models

Paper · 2026 · Guanyu Zheng, Wang Zhenyu, He Tingting et al.

Summary

Memory-based editors for lifelong knowledge editing of LLMs tend to lose earlier corrections as the edit stream grows. This is worst when edits jump between unrelated domains or repeatedly change attributes of a single subject. The authors trace this to entanglement. In editors such as WISE and GRACE, one shared mechanism both chooses which memory to activate and absorbs the parameter updates, so each new write disturbs routing and previously stored facts.

DKME separates these two roles in two stages:

  • Decoupled Semantic Addressing trains an independent, fact-aware embedding manifold. It decides whether a query falls inside the scope of existing edits and which neighbourhood it belongs to.
  • Partitioned Memory Storage clusters edits in that space without supervision and writes each update only into its assigned memory partition.

The method was evaluated on HalluEditBench, CKnowEdit and WikiData counterfact. DKME balances edit success against locality better than the baselines and degrades less as edits accumulate. This matters for keeping deployed models current without retraining.

Key points

  • Metrics are reliability, generalization and locality, plus their mean (Avg.). The main backbone is LLaMA3.2-3B.

  • On HalluEditBench with 1000 edits, DKME reaches Rel. 0.923, Gen. 0.718, Loc. 0.834 and Avg. 0.825. Ablations show the contribution of each stage:

    • Removing the addressing stage cuts locality to 0.276 (Avg. −0.205).
    • Removing partitioned storage lowers Avg. by 0.160.

  • The boundary threshold τ works best at different values per dataset:

    DatasetτAvg.
    HalluEditBench0.850.825
    WikiData counterfact0.800.678
    CKnowEdit0.700.634

  • Editing mid-to-upper layers (17–23) of LLaMA3.2-3B works best, with layer 19 the strongest.

  • The authors list several limitations:

    • extra storage overhead, and a possible scalability bottleneck from the growing number of partitions;
    • reliance on an external encoder;
    • no evaluation of multi-hop reasoning;
    • a narrow range of model sizes and architectures.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Harbin Institute of TechnologyHarbin · China185 works · 29,140 citations🎓South China University of TechnologyGuangzhou · China27 works · 1,554 citations

Also on the byline: Northeast Petroleum University.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Guanyu Zheng, Wang Zhenyu, He Tingting, Xv Wang, Haochang Wang, Yaokai Huang and Tiejun Zhao. 2026. DKME: Rethinking Coupled Knowledge Memory for Lifelong Model Editing of Large Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.792

@inproceedings{zheng2026dkme,
  title = {{DKME: Rethinking Coupled Knowledge Memory for Lifelong Model Editing of Large Language Models}},
  author = {Guanyu Zheng, Wang Zhenyu, He Tingting, Xv Wang, Haochang Wang, Yaokai Huang and Tiejun Zhao},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.792},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.792}
}

References

65 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PmLR.
  3. Yingfa Chen, Zhengyan Zhang, Xu Han, Chaojun Xiao, Zhiyuan Liu, Chen Chen, Kuai Li, Tao Yang, and Maosong Sun. 2024. Robust and scalable model editing for large language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation… find
  4. Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2024. Evaluating the ripple effects of knowledge editing in language models. Transactions of the Association for Computational Linguistics, 12:283–298. find
  5. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8493– 8502. find
  6. Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6491– 6506. find
  7. Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. 2021. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7):3366–3385. find
  8. Zilu Dong, Xiangqing Shen, and Rui Xia. 2025. MEMIT-merge: Addressing MEMIT’s key-value conflicts in same-subject batch editing for LLMs. In Findings of the Association for Computational Linguistics: ACL 2025, pages 7952–7960. find
  9. Jizhan Fang, Tianhe Lu, Yunzhi Yao, Ziyan Jiang, Xin Xu, Huajun Chen, and Ningyu Zhang. 2025a. CKnowEdit: A new Chinese knowledge editing dataset for linguistics, facts, and logic error correction in LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Vo… find
  10. Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2025b. Alphaedit: Null-space constrained model editing for language models. In The Thirteenth International Conference on Learning Representations. find
  11. Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. Dissecting recall of factual associations in auto-regressive language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12216–12235. find
  12. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495. find
  13. Rhys Gould, Euan Ong, George Ogden, and Arthur Conmy. 2024. Successor heads: recurring, interpretable attention heads in the wild. In The Twelfth International Conference on Learning Representations. find
  14. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, and 1 others. 2025. Deepseekr1 incentivizes reasoning in llms through reinforcement learning. Nature, 645(8081):633–638. find
  15. Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  16. Xiangkun Hu, Dongyu Ru, Lin Qiu, Qipeng Guo, Tianhang Zhang, Yang Xu, Yun Luo, Pengfei Liu, Yue Zhang, and Zheng Zhang. 2024. Refchecker: Reference-based fine-grained hallucination checker and benchmark for large language models. arXiv preprint arXiv:2405.14486. — [arXiv:2405.14486](https://arxiv.org/abs/2405.14486) find
  17. Baixiang Huang, Canyu Chen, Xiongxiao Xu, Ali Payani, and Kai Shu. 2024a. Can knowledge editing really correct hallucinations? arXiv preprint arXiv:2410.16251. — [arXiv:2410.16251](https://arxiv.org/abs/2410.16251) find
  18. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and 1 others. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems,…
  19. Xiusheng Huang, Jiaxiang Liu, Yequan Wang, and Kang Liu. 2024b. Reasons and solutions for the decline in model performance after editing. In Advances in Neural Information Processing Systems, pages 68833–68853. find
  20. Akshat Gupta, Phudish Prateepamornkul, Maochuan Lu, Ahmed Alaa, Thomas Hartvigsen, and Gopala Anumanchipalli. 2025. Lifelong knowledge editing requires better regularization. arXiv preprint arXiv:2502.01636. find
  21. Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. 2023. Transformer-patcher: One mistake worth one neuron. In The Eleventh International Conference on Learning Representations.
  22. Akshat Gupta, Dev Sajnani, and Gopala Anumanchipalli. 2024. A unified framework for model editing. Preprint, arXiv:2403.14236. find
  23. Houcheng Jiang, Junfeng Fang, Ningyu Zhang, Mingyang Wan, Guojun Ma, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2025. Anyedit: Edit any knowledge encoded in language models. In Forty-second International Conference on Machine Learning. find
  24. Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. 2023. Aging with grace: Lifelong model editing with discrete key-value adaptors. Advances in Neural Information Processing Systems, 36:47934–47959. find
  25. Thomas Henn, Yasukazu Sakamoto, Clément Jacquet, Shunsuke Yoshizawa, Masamichi Andou, Stephen Tchen, Ryosuke Saga, Hiroyuki Ishihara, Katsuhiko Shimizu, Yingzhen Li, and 1 others. 2021. A principled approach to failure analysis and model repairment: Demonstration in medical imaging. In Medical Image… find
  26. Chenhui Hu, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. 2025. Knowledge in superposition: Unveiling the failures of lifelong knowledge editing for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24086–24094. find
  27. Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar, and Bryon Aragam. 2024. Do llms dream of elephants (when told not to)? latent concept association and associative memory in transformers. Advances in Neural Information Processing Systems, 37:67712– 67757. find
  28. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. 2017. Overcoming catastrophic forgetting in neural networks… find
  29. Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. 2025. Analyzing feed-forward blocks in transformers through the lens of attention maps. In The Twelfth International Conference on Learning Representations. find
  30. Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, and Sanjiv Kumar. 2023. Large language models with controllable working memory. In Findings of the Association for Computational Linguistics: ACL 2023, pages 1774–1793.
  31. Qi Li and Xiaowen Chu. 2025. Adaedit: Advancing continuous knowledge editing for large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4127–4149. find
  32. Xiaopeng Li, Shangwen Wang, Shasha Li, Shezheng Song, Bin Ji, Ma Jun, and Jie Yu. 2025. Rethinking residual distribution in locate-then-edit model editing. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  33. Bill Yuchen Lin, Sida I Wang, Xi Lin, Robin Jia, Lin Xiao, Xiang Ren, and Scott Yih. 2022. On continual model refinement in out-of-distribution data streams. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3128–3139. find
  34. Zihao Lin, Mohammad Beigi, Hongxuan Li, Yufan Zhou, Yuxiang Zhang, Qifan Wang, Wenpeng Yin, and Lifu Huang. 2024. Navigating the dual facets: A comprehensive evaluation of sequential memory editing in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computation… find
  35. Zhenhua Liu, Yunhe Wang, Kai Han, Wei Zhang, Siwei Ma, and Wen Gao. 2021. Post-training quantization for vision transformer. In Proceedings of the 35th International Conference on Neural Information Processing Systems, pages 28092–28103. find
  36. Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2023. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747. find
  37. Marc Masana, Xialei Liu, Bartłomiej Twardowski, Mikel Menta, Andrew D Bagdanov, and Joost Van De Weijer. 2022. Class-incremental learning: survey and performance evaluation on image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5513–5533. find
  38. Leland McInnes, John Healy, and Steve Astels. 2017. Hdbscan: Hierarchical density based clustering. Journal of Open Source Software, 2(11):205.
  39. Leland McInnes, John Healy, Nathaniel Saul, and Lukas Großberger. 2018. Umap: Uniform manifold approximation and projection. Journal of Open Source Software, 3(29):861. find
  40. Christian HX Ali Mehmeti-Göpel and Michael Wand. 2024. On the weight dynamics of deep normalized networks. In Proceedings of the 41st International Conference on Machine Learning, pages 992–1007. find
  41. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. In Proceedings of the 36th International Conference on Neural Information Processing Systems, pages 17359–17372. find
  42. Kevin Meng, Arnab Sen Sharma, Alex J Andonian, Yonatan Belinkov, and David Bau. 2023. Mass-editing memory in a transformer. In The Eleventh International Conference on Learning Representations. find
  43. Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. 2022. Memory-based model editing at scale. In International Conference on Machine Learning, pages 15817–15831. PMLR. find
  44. Zihan Qiu, Zeyu Huang, Youcheng Huang, and Jie Fu. 2024. Empirical study on updating key-value memories in transformer feed-forward layers. In The Second Tiny Papers Track at ICLR 2024. find
  45. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, and 1 others. 2018. Improving language understanding by generative pre-training. find
  46. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using eiamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992. find
  47. David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy P Lillicrap, and Greg Wayne. 2019. Experience replay for continual learning. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pages 350–360. find
  48. Haizhou Shi, Zihao Xu, Hengyi Wang, Weiyi Qin, Wenyuan Wang, Yibin Wang, Zifeng Wang, Sayna Ebrahimi, and Hao Wang. 2025. Continual learning of large language models: A comprehensive survey. ACM Comput. Surv. find
  49. Chenmien Tan, Ge Zhang, and Jie Fu. 2021. Massive editing for large language models via meta learning. In The Twelfth International Conference on Learning Representations. find
  50. Lukas Thede, Karsten Roth, Matthias Bethge, Zeynep Akata, and Thomas Hartvigsen. 2025. Wikibigedit: Understanding the limits of lifelong knowledge editing in llms. In Forty-second International Conference on Machine Learning. find
  51. Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan. 2022. Memorization without overfitting: Analyzing the training dynamics of large language models. Advances in Neural Information Processing Systems, 35:38274–38290.
  52. Frederik Träuble, Anirudh Goyal, Nasim Rahaman, Michael Curtis Mozer, Kenji Kawaguchi, Yoshua Bengio, and Bernhard Schölkopf. 2023. Discrete key-value bottleneck. In International conference on machine learning, pages 34431–34455. PMLR. find
  53. Peng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. 2024a. Wise: Rethinking the knowledge memory for lifelong model editing of large language models. Advances in Neural Information Processing Systems, 37:53764–53797. find
  54. Peng Wang, Ningyu Zhang, Bozhong Tian, Zekun Xi, Yunzhi Yao, Ziwen Xu, Mengru Wang, Shengyu Mao, Xiaohan Wang, Siyuan Cheng, and 1 others. 2024b. Easyedit: An easy-to-use knowledge editing framework for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computati… find
  55. Renzhi Wang and Piji Li. 2024. Lemoe: Advanced mixture of experts adaptor for lifelong model editing of large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 2551–2575. find
  56. Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. 2024c. Knowledge editing for large language models: A survey. ACM Computing Surveys, 57(3):1–37.
  57. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, pages 24824–2…
  58. Xin Wu, Yuqi Bu, Yi Cai, and Tao Wang. 2024. Updating large language models’ memories with time constraints. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13693– 13702. find
  59. Yichen Wu, Hongming Piao, Long-Kai Huang, Renzhen Wang, Wanhua Li, Hanspeter Pfister, Deyu Meng, Kede Ma, and Ying Wei. 2025. SD-loRA: Scalable decoupled low-rank adaptation for class incremental learning. In The Thirteenth International Conference on Learning Representations. find
  60. Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023. Editing large language models: Problems, methods, and opportunities. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10222–10240. find
  61. Zihan Yao, Yu He, Tianyu Qi, and Ming Li. 2024. Scalable model editing via customized expert networks. In First Conference on Language Modeling (COLM). find
  62. Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, and 1 others. 2024. A comprehensive study of knowledge editing for large language models. arXiv preprint arXiv:2401.01286. find
  63. Yinhe Zheng. 2022. Continuous qa learning with structured prompts. arXiv preprint arXiv:2208.14602. — [arXiv:2208.14602](https://arxiv.org/abs/2208.14602) find
  64. Shaochen Zhong, Yifan Lu, Lize Shao, Bhargav Bhushanam, Xiaocong Du, Yixin Wan, Yucheng Shi, Daochen Zha, Yiwei Wang, Ninghao Liu, Kaixiong Zhou, Shuai Xu, Kai-Wei Chang, Louis Feng, Vipin Chaudhary, and Xia Hu. 2025. Mquake-remastered: Multi-hop knowledge editing can only be advanced with reliable… find
  65. Zexuan Zhong, Zhengxuan Wu, Christopher D Manning, Christopher Potts, and Danqi Chen. 2023. Mquake: Assessing knowledge editing in language models via multi-hop questions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15686–15702. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zheng-2026-dkme-rethinking-coupled-knowledge-memory/