Elamur Research⌕
☰
home / papers / 2025 / Neuron-Level Sequential Editing for Large Language Models

Neuron-Level Sequential Editing for Large Language Models

Paper · 2025 · Houcheng Jiang, Junfeng Fang, Tianyu Zhang et al.

Summary

Large language models store facts that go out of date, and full retraining is expensive. Parameter-editing methods such as ROME and MEMIT work well for a single round of edits. When edits are applied one after another, however, they tend to lose earlier edits ("model forgetting") and can collapse into incoherent output ("model failure"). The authors propose Neuron-level Sequential Editing (NSE), which has three parts:

  1. Weights rewinding: target hidden-state values are always computed with the model's original weights, not the accumulated edited ones.
  2. Neuron selection: only the most strongly activated FFN neurons are updated, instead of whole weight matrices.
  3. Iterative multi-layer editing: large batches of facts are spread across several layers.

Experiments use GPT2-XL, GPT-J and Llama3 (8B) on Counterfact and related benchmarks. NSE outperforms MEND, ROME, MEMIT and GRACE on standard editing metrics, including efficacy, generalization and specificity. It also keeps general abilities close to the unedited model. This matters because it makes continual, low-cost knowledge updates more practical without the storage overhead of memory-based approaches.

Key points

  • Neuron selection: neurons are ranked by absolute activation. The smallest set whose combined score exceeds a chosen fraction p of the total is edited, and the rest of W_out stays fixed.
  • Storage cost of rewinding: only the original W_out matrices of the edited layers need to be kept, not a full copy of the model.
  • Batch-size test: 2000 sequential Counterfact edits were run with batch sizes 200, 100, 50 and 10. MEMIT degraded more as the number of rounds grew, especially at batch size 10. NSE stayed stronger across all settings.
  • General-ability test: six GLUE tasks were evaluated on Llama3 (8B) after 3000 edits. NSE remained near the pre-edit model, while ROME and MEMIT dropped sharply after about 1,000–2,000 edits.
  • Limitations and code: the authors note that the neuron selection rule is simple and that iterative editing costs some efficiency. Code is available at github.com/jianghoucheng/NSE.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Houcheng Jiang, Junfeng Fang, Tianyu Zhang, Baolong Bi, An Zhang, Ruipeng Wang, Tao Liang and Xiang Wang. 2025. Neuron-Level Sequential Editing for Large Language Models. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.815

@inproceedings{jiang2025neuron,
  title = {{Neuron-Level Sequential Editing for Large Language Models}},
  author = {Houcheng Jiang, Junfeng Fang, Tianyu Zhang, Baolong Bi, An Zhang, Ruipeng Wang, Tao Liang and Xiang Wang},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.815},
  url = {https://doi.org/10.18653/v1/2025.acl-long.815}
}

References

50 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. EMNLP (1), pages 6491–6506. Association for Computational Linguistics. find
  2. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In ACL (1), pages 8493– 8502. Association for Computational Linguistics. find
  3. William B. Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In IWP@IJCNLP. Asian Federation of Natural Language Processing.
  4. Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui, and Lei Li. 2022. Calibrating factual knowledge in pretrained language models. In EMNLP (Findings), pages 5937–5947. Association for Computational Linguistics.
  5. Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2025. Alphaedit: Null-space constrained knowledge editing for language models. In ICLR. OpenReview.net. find
  6. Akshat Gupta, Anurag Rao, and Gopala Anumanchipalli. 2024a. Model editing at scale leads to gradual and catastrophic forgetting. CoRR, abs/2401.07453. find
  7. James A Anderson. 1972. A simple neural network generating an interactive memory. Mathematical biosciences, 14(3-4):197–220. find
  8. Akshat Gupta, Dev Sajnani, and Gopala Anumanchipalli. 2024b. A unified framework for model editing. CoRR, abs/2403.14236. find
  9. Luisa Bentivogli, Bernardo Magnini, Ido Dagan, Hoa Trang Dang, and Danilo Giampiccolo. 2009. The fifth PASCAL recognizing textual entailment challenge. In TAC. NIST. find
  10. Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. 2023. Aging with GRACE: lifelong model editing with discrete key-value adaptors. In NeurIPS. find
  11. Baolong Bi, Shenghua Liu, Yiwei Wang, Lingrui Mei, and Xueqi Cheng. 2024a. Is factuality decoding a free lunch for llms? evaluation on knowledge editing benchmark. Preprint, arXiv:2404.00216. — [arXiv:2404.00216](https://arxiv.org/abs/2404.00216) find
  12. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In ICLR. OpenReview.net.
  13. Baolong Bi, Shenghua Liu, Yiwei Wang, Lingrui Mei, Hongcheng Gao, Junfeng Fang, and Xueqi Cheng. 2024b. Struedit: Structured outputs enable the fast and accurate knowledge editing for large language models. arXiv preprint arXiv:2409.10132. — [arXiv:2409.10132](https://arxiv.org/abs/2409.10132) find
  14. Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. 2023. Transformer-patcher: One mistake worth one neuron. In ICLR. OpenReview.net.
  15. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  16. Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Editing factual knowledge in language models. In Houcheng Jiang, Junfeng Fang, Ningyu Zhang, Guojun Ma, Mingyang Wan, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2025. Anyedit: Edit any knowledge encoded in language models. CoRR, abs/2502.05628. find
  17. Teuvo Kohonen. 1972. Correlation matrix memories. IEEE Trans. Computers, 21(4):353–359. find
  18. Serge Lang. 2012. Introduction to linear algebra. Springer Science & Business Media. find
  19. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. Zero-shot relation extraction via reading comprehension. In CoNLL, pages 333–342. Association for Computational Linguistics.
  20. Zherui Li, Houcheng Jiang, Hao Chen, Baolong Bi, Zhenhong Zhou, Fei Sun, Junfeng Fang, and Xiang Wang. 2025. Reinforced lifelong editing for language models. CoRR, abs/2502.05759. — [arXiv:2502.05759](https://arxiv.org/abs/2502.05759) find
  21. Yue Liu, Xiaoxin He, Miao Xiong, Jinlan Fu, Shumin Deng, and Bryan Hooi. 2024. Flipattack: Jailbreak llms via flipping. arXiv preprint arXiv:2410.02832. — [arXiv:2410.02832](https://arxiv.org/abs/2410.02832) find
  22. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in GPT. In NeurIPS. find
  23. Kevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. 2023. Mass-editing memory in a transformer. In ICLR. Open-Review.net. find
  24. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. 2022a. Fast model editing at scale. In ICLR. OpenReview.net. find
  25. Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. 2022b. Memory-based model editing at scale. In ICML, volume 162 of Proceedings of Machine Learning Research, pages 15817–15831. PMLR. find
  26. Haowen Pan, Yixin Cao, Xiaozhi Wang, and Xun Yang. 2023. Finding and editing multi-modal neurons in pre-trained transformer. CoRR, abs/2311.07470. — [arXiv:2311.07470](https://arxiv.org/abs/2311.07470) find
  27. Haowen Pan, Yixin Cao, Xiaozhi Wang, Xun Yang, and Meng Wang. 2024. Finding and editing multimodal neurons in pre-trained transformers. In ACL (Findings), pages 1012–1037. Association for Computational Linguistics. find
  28. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. 2019. Language models as knowledge bases? In EMNLP/IJCNLP (1), pages 2463–2473. Association for Computational Linguistics.
  29. Sarah Schwettmann, Neil Chowdhury, and Antonio Torralba. 2023b. Multimodal neurons in pretrained text-only transformers. CoRR, abs/2308.01544. — [arXiv:2308.01544](https://arxiv.org/abs/2308.01544) find
  30. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In EMNLP, pages 1631–1642. ACL.
  31. Chenmien Tan, Ge Zhang, and Jie Fu. 2024. Massive editing for large language models via meta learning. In ICLR. OpenReview.net. find
  32. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS, pages 5998–6008.
  33. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In ICLR (Poster). OpenReview.net.
  34. Ben Wang and Aran Komatsuzaki. 2021. Gpt-j-6b: A 6 billion parameter autoregressive language model. find
  35. Peng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. 2024. WISE: rethinking the knowledge memory for lifelong model editing of large language models. CoRR, abs/2405.14768. find
  36. Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, and Juanzi Li. 2022. Finding skill neurons in pre-trained transformer-based language models. In EMNLP, pages 11132–11152. Association for Computational Linguistics.
  37. Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2019. Neural network acceptability judgments. Trans. Assoc. Comput. Linguistics, 7:625–641. find
  38. Junnan Dong Shengyuan Chen Feiran Huang Xiao Huang Qinggang Zhang, Hao Chen. 2025. Structure-guided large language models for text-to-SQL generation. In Forty-second International Conference on Machine Learning. find
  39. Adina Williams, Nikita Nangia, and Samuel R. Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In NAACL-HLT, pages 1112–1122. Association for Computational Linguistics.
  40. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. find
  41. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. CoRR, abs/1910.03771.
  42. Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the parameters of a language model? In EMNLP (1), pages 5418–5426. Association for Computational Linguistics. find
  43. Sarah Schwettmann, Neil Chowdhury, Samuel Klein, David Bau, and Antonio Torralba. 2023a. Multimodal neurons in pretrained text-only transformers. In ICCV (Workshops), pages 2854–2859. IEEE. find
  44. Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. 2023. DEPN: detecting and editing privacy neurons in pre-trained language models. In EMNLP, pages 2875– 2886. Association for Computational Linguistics. find
  45. Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023. Editing large language models: Problems, methods, and opportunities. In EMNLP, pages 10222–10240. Association for Computational Linguistics. find
  46. Ningyu Zhang, Bozhong Tian, Siyuan Cheng, Xiaozhuan Liang, Yi Hu, Kouying Xue, Yanjie Gou, Xi Chen, and Huajun Chen. 2024a. Instructedit: Instruction-based knowledge editing for large language models. In IJCAI, pages 6633–6641. ijcai.org. find
  47. Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, Siyuan Cheng, Ziwen Xu, Xin Xu, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, and Huajun Chen. 2024b. A comprehensive… find
  48. Taolin Zhang, Qizhou Chen, Dongyang Li, Chengyu Wang, Xiaofeng He, Longtao Huang, Hui Xue’, and Jun Huang. 2024c. Dafnet: Dynamic auxiliary fusion for sequential model editing in large language models. In ACL (Findings), pages 1588–1602. Association for Computational Linguistics. find
  49. Tianyu Zhang, Junfeng Fang, Houcheng Jiang, Baolong Bi, Xiang Wang, and Xiangnan He. Explainable and efficient editing for large language models. In THE WEB CONFERENCE 2025. find
  50. Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix X. Yu, and Sanjiv Kumar. 2020. Modifying memories in transformer models. CoRR, abs/2012.00363. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/jiang-2025-neuron-level-sequential-editing-large/