Elamur Research⌕
☰
home / papers / 2026 / QueueEDIT: Structural Self-Correction for Sequential Model Editing in LLMs

QueueEDIT: Structural Self-Correction for Sequential Model Editing in LLMs

Paper · 2026 · Taolin Zhang, Haidong Kang, Dongyang Li et al.

Summary

Large language models still produce factual errors. Model editing can patch individual facts, but in sequential model editing (SME), where corrections arrive one after another, earlier edits may conflict with new ones and the model's general abilities can degrade. The authors propose QueueEDIT, which combines two ideas:

  • Structural mapping editing loss. Each knowledge triplet ⟨subject, relation, object⟩ is split across the FFN layer instead of being blended into one representation. The subject is tied to the first MLP matrix, the relation to the second, and the object to the backpropagated target.
  • Queue-based self-correction. Parameters located for each edit are stored in a first-in-first-out queue. At every new edit, the method picks the Top-K stored entries closest to the current parameters and realigns them, freezes the unrelated ones, and dequeues the oldest.

The method was tested on ZSRE, CounterFact and RIPE with GPT-J, LLaMA3 and Qwen2.5 backbones. QueueEDIT beats a broad set of baselines as the number of edits grows, stays competitive for single edits, and keeps performance on general NLP tasks. This matters for keeping deployed LLMs up to date without retraining.

Key points

  • Baselines span parameter modification (ROME, MEMIT, AlphaEdit, MEND, MALMEN, DAFNet), added parameters (T-Patcher, GRACE) and retrieval-style methods (MELO, LTE).
  • Ablations show that each part contributes: dropping the queue causes the largest decline, and replacing Top-K selection with random selection also hurts.
  • The queue-length study used 1,000 edits on LLaMA3 (8B) with queues of 10%, 30% and 50% of the edits. A 30% queue gave the best average, and longer queues added noise from irrelevant parameters.
  • On Qwen2.5-14B with 1,000 sequential edits, Figure 4 appears to report about 72.3% average accuracy for QueueEDIT, versus roughly 64.1 for DAFNet, 61.7 for LTE and 39.5 for T-Patcher. The value-to-method pairing is inferred from garbled PDF text.
  • QueueEDIT uses somewhat more memory than the baselines. Fixed settings (e.g., K = 50) are acknowledged as a limitation, and adaptive queue sizing is left for future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢Alibaba GroupHangzhou · China201 works · 17,172 citations🎓Northeastern UniversityBoston · United States68 works · 7,316 citations🎓East China Normal UniversityShanghai · China69 works · 2,490 citations🎓Hefei University of TechnologyHefei · China17 works · 1,702 citations🎓Shanghai University of Electric PowerYangpu · China5 works · 13 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Taolin Zhang, Haidong Kang, Dongyang Li, Qizhou Chen, Xiaofeng He, Chengyu Wang and Richang Hong. 2026. QueueEDIT: Structural Self-Correction for Sequential Model Editing in LLMs. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.231

@inproceedings{zhang2026queueedit,
  title = {{QueueEDIT: Structural Self-Correction for Sequential Model Editing in LLMs}},
  author = {Taolin Zhang, Haidong Kang, Dongyang Li, Qizhou Chen, Xiaofeng He, Chengyu Wang and Richang Hong},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.231},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.231}
}

References

47 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, and Weizhu Chen. 2023. Learning from mistakes makes LLM better reasoner. CoRR, abs/2310.20689.
  2. Christine Basta, Marta R. Costa-jussà, and Noe Casas. 2021. Extensive study on the underlying gender bias in contextualized word embeddings. Neural Comput. Appl., pages 3371–3384. find
  3. Sofia Blinova, Xinyu Zhou, Martin Jaggi, Carsten Eickhoff, and Seyed Ali Bahrainian. 2023. SIMSUM: document-level text simplification via simultaneous summarization. In ACL, pages 9927–9944. find
  4. Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In NIPS, pages 2787–2795.
  5. Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Editing factual knowledge in language models. In EMNLP, pages 6491–6506. find
  6. Qizhou Chen, Taolin Zhang, Xiaofeng He, Dongyang Li, Chengyu Wang, Longtao Huang, and Hui Xue’. 2024. Lifelong knowledge editing for llms with retrieval-augmented continuous prompt learning. In EMNLP, pages 13565–13580. find
  7. Ge Zhang Chenmien Tan and Jie Fu. 2023. Massive editing for large language models via meta learning. CoRR, abs/2311.04661. — [arXiv:2311.04661](https://arxiv.org/abs/2311.04661) find
  8. Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2023. Evaluating the ripple effects of knowledge editing in language models. CoRR, abs/2307.12976. find
  9. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In ACL, pages 8493–8502. find
  10. Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui, and Lei Li. 2022. Calibrating factual knowledge in pretrained language models. In EMNLP, pages 5937–5947.
  11. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, and 82 others. 2024… — The Llama 3 Herd of Models
  12. Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2025. Alphaedit: Null-space constrained knowledge editing for language models. In ICLR. find
  13. Akshat Gupta and Gopala Anumanchipalli. 2024. Rebuilding ROME : Resolving model collapse during sequential model editing. CoRR, abs/2403.07175. find
  14. Akshat Gupta, Anurag Rao, and Gopala Anumanchipalli. 2024. Model editing at scale leads to gradual and catastrophic forgetting. In ACL, pages 15202–15232. find
  15. Thomas Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. 2022. Aging with GRACE: lifelong model editing with discrete key-value adaptors. CoRR, abs/2211.11031. find
  16. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In ICLR.
  17. Chenhui Hu, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. 2024. Wilke: Wise-layer knowledge editor for lifelong knowledge editing. CoRR, abs/2402.10987. find
  18. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In ICLR.
  19. Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. 2023. Transformer-patcher: One mistake worth one neuron. In ICLR.
  20. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. 2022a. Fast model editing at scale. In ICLR. find
  21. Jonas Hübotter, Sascha Bongni, Ido Hakimi, and Andreas Krause. 2025. Efficiently learning at test-time: Active fine-tuning of llms. In ICLR. find
  22. Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. 2022b. Memory-based model editing at scale. In ICML, pages 15817– 15831. find
  23. Yuxin Jiang, Yufei Wang, Chuhan Wu, Wanjun Zhong, Xingshan Zeng, Jiahui Gao, Liangyou Li, Xin Jiang, Lifeng Shang, Ruiming Tang, Qun Liu, and Wei Wang. 2024. Learning to edit: Aligning llms with knowledge editing. CoRR, abs/2402.11905.
  24. Kyeonghyun Kim, Jinhee Jang, Juhwan Choi, Yoonji Lee, Kyohoon Jin, and YoungBin Kim. 2025. Plug-in and fine-tuning: Bridging the gap between small language models and large language models. find
  25. Sai Krishna, Balvinder Singh, Sujoy Roychowdhury, Giriprasad Sridhara, Sourav Mazumdar, Magnus Sandelin, Dimitris Rentas, Maciej Nalepa, Karol Sawicki, and Jakub Gajda. 2025. Test code generation at ericsson using program analysis augmented fine tuned llms. find
  26. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. Zero-shot relation extraction via reading comprehension. In CoNLL, pages 333–342.
  27. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In ACL, pages 7871–7880.
  28. Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023. Compressing context to enhance inference efficiency of large language models. In EMNLP, pages 6342–6353.
  29. Di Liu, Meng Chen, Baotong Lu, Huiqiang Jiang, Zhenhua Han, Qianxi Zhang, Qi Chen, Chengruidong Zhang, Bailu Ding, Kai Zhang, Chen Chen, Fan Yang, Yuqing Yang, and Lili Qiu. 2024. Retrievalattention: Accelerating long-context LLM inference via vector retrieval. CoRR, abs/2409.10516.
  30. Aman Madaan, Niket Tandon, Peter Clark, and Yiming Yang. 2022. Memory-assisted prompt editing to improve GPT-3 after deployment. In EMNLP, pages 2833–2861. find
  31. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in GPT. In NeurIPS. find
  32. Kevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. 2023. Mass-editing memory in a transformer. In ICLR. find
  33. Abhika Mishra, Akari Asai, Vidhisha Balachandran, Yizhong Wang, Graham Neubig, Yulia Tsvetkov, and Hannaneh Hajishirzi. 2024. Fine-grained hallucination detection and editing for language models. CoRR, abs/2401.06855.
  34. Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Adversarial NLI: A new benchmark for natural language understanding. In ACL, pages 4885–4901.
  35. Qwen Team. 2024. Qwen2.5: A party of foundation models. find
  36. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100, 000+ questions for machine comprehension of text. In EMNLP, pages 2383–2392.
  37. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In EMNLP, pages 3980–3990.
  38. Amrita Saha, Vardaan Pahuja, Mitesh M. Khapra, Karthik Sankaranarayanan, and Sarath Chandar. 2018. Complex sequential question answering: Towards learning to converse over linked question answer pairs with a knowledge graph. In AAAI, pages 705–713. find
  39. Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In ICML, pages 31210–31227.
  40. Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, and Colin Raffel. 2023. Evaluating the factual consistency of large language models through news summarization. In ACL, pages 5220–5255. find
  41. Peng Wang, Ningyu Zhang, Xin Xie, Yunzhi Yao, Bozhong Tian, Mengru Wang, Zekun Xi, Siyuan Cheng, Kangwei Liu, Guozhou Zheng, and Huajun Chen. 2023. Easyedit: An easy-to-use knowledge editing framework for large language models. CoRR, abs/2308.07269. find
  42. Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In AAAI, pages 1112–1119. find
  43. Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023. Editing large language models: Problems, methods, and opportunities. CoRR, abs/2305.13172. find
  44. Haoran You, Yipin Guo, Yichao Fu, Wei Zhou, Huihong Shi, Xiaofan Zhang, Souvik Kundu, Amir Yazdanbakhsh, Yingyan, and Lin. 2024. Shiftaddllm: Accelerating pretrained llms via post-training multiplication-less reparameterization. arXiv:2406.05981. — [arXiv:2406.05981](https://arxiv.org/abs/2406.05981) find
  45. Preprint, Lang Yu, Qin Chen, Jie Zhou, and Liang He. 2024. MELO: enhancing model editing with neuron-indexed dynamic lora. In AAAI, pages 19449–19457. find
  46. Taolin Zhang, Qizhou Chen, Dongyang Li, Chengyu Wang, Xiaofeng He, Longtao Huang, Hui Xue’, and Jun Huang. 2024. Dafnet: Dynamic auxiliary fusion for sequential model editing in large language models. In ACL, pages 1588–1602. find
  47. Heqi Zheng, Xiao Zhang, Zewen Chi, Heyan Huang, Yan Tan, Tian Lan, Wei Wei, and Xian-Ling Mao. 2022. Cross-lingual phrase retrieval. In ACL, pages 4193–4204. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-queueedit-structural-self-correction-sequential/