Elamur Research⌕
☰
home / papers / 2026 / Fisher-Driven Adaptive Locating for Knowledge Editing in Large Language Models

Fisher-Driven Adaptive Locating for Knowledge Editing in Large Language Models

Paper · 2026 · Chenghao Xu, Jiexi Yan, Guangtao Lyu et al.

Summary

Locate-then-edit methods for updating facts in LLMs, such as ROME, MEMIT, UnKE and AlphaEdit, usually modify a single layer that is chosen in advance and kept fixed for every edit. This ignores the fact that transformers store different kinds of information at different depths. The authors propose FiDAL, a strategy that chooses where to edit for each fact individually. It treats localization as a weight-level decision over the linear layers of the feed-forward blocks. Before the actual edit, a short probing phase trains only two low-rank modulation parameters. Fisher Information computed on those parameters then indicates which layers are most sensitive to the target fact. Each layer is either marked for kernel modulation or left frozen, and an existing editor then performs the update. FiDAL was tested on ZsRE and WikiData Counterfact with GPT-J, Qwen2.5-7B and LLaMA-3. Attached to several editors, it improved both edit success and preservation of unrelated knowledge, while adding only modest runtime. Its value is that it works as a plug-in upgrade to the rarely questioned locating step.

Key points

  • Method: FiDAL uses Fisher Information, computed only over low-rank modulation parameters, to make per-instance, per-layer edit/freeze decisions in FFN linear layers. It is compatible with any locate-then-edit method.
  • Gains on WikiData Counterfact with Qwen2.5-7B (K = 1):
    • MEMIT: SAA rose from 78.3 to 83.0 and FA from 42.7 to 52.8.
    • UnKE: SAA rose from 76.8 to 82.1, RA from 35.3 to 40.1, and FA from 48.6 to 61.1.
    • AlphaEdit: SAA rose from 75.2 to 81.4 and FA from 60.8 to 63.4.
  • Probing iterations (T): performance levels off after about 40 iterations, and T = 60 is used in all experiments.
  • Cost: the time overhead relative to fixed-layer baselines appears to be roughly 0.1 h per 100 samples. This figure is approximate, because the values in Figure 2 were not clearly labeled in the extracted text.
  • Limitations: the method only modulates FFN weights, not attention layers or embeddings. The probing stage also adds computation compared with fixed locating.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Institute for Infocomm ResearchSingapore · Singapore45 works · 14,598 citations🎓Xidian UniversityXi'an · China17 works · 1,357 citations🎓Hohai UniversityNanjing · China4 works · 5 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Chenghao Xu, Jiexi Yan, Guangtao Lyu, Qi Liu, Muli Yang and Cheng Deng. 2026. Fisher-Driven Adaptive Locating for Knowledge Editing in Large Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.957

@inproceedings{xu2026fisher,
  title = {{Fisher-Driven Adaptive Locating for Knowledge Editing in Large Language Models}},
  author = {Chenghao Xu, Jiexi Yan, Guangtao Lyu, Qi Liu, Muli Yang and Cheng Deng},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.957},
  url = {https://doi.org/10.18653/v1/2026.acl-long.957}
}

References

43 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Baolong Bi, Shenghua Liu, Lingrui Mei, Yiwei Wang, Pengliang Ji, and Xueqi Cheng. 2024. Decoding by contrasting knowledge: Enhancing llms’ confidence on edited facts. arXiv preprint arXiv:2405.11613. — [arXiv:2405.11613](https://arxiv.org/abs/2405.11613) find
  2. Jinhe Bi, Yifan Wang, Danqi Yan, Xun Xiao, Artur Hecker, Volker Tresp, and Yunpu Ma. 2025a. Prism: Self-pruning intrinsic selection method for training-free multimodal data selection. Preprint, arXiv:2502.12119. — [arXiv:2502.12119](https://arxiv.org/abs/2502.12119) find
  3. Jinhe Bi, Yujun Wang, Haokun Chen, Xun Xiao, Artur Hecker, Volker Tresp, and Yunpu Ma. 2025b. Llava steering: Visual instruction tuning with 500x fewer parameters through modality linear representation-steering. In ACL.
  4. Jinhe Bi, Danqi Yan, Yifan Wang, Wenke Huang, Haokun Chen, Guancheng Wan, Mang Ye, Xun Xiao, Hinrich Schuetze, Volker Tresp, and 1 others. 2025c. Cot-kinetics: A theoretical modeling assessing lrm reasoning process. arXiv preprint arXiv:2505.13408. — [arXiv:2505.13408](https://arxiv.org/abs/2505.13408) find
  5. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  6. Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R Glass, and Pengcheng He. 2023. Dola: Decoding by contrasting layers improves factuality in large language models. In The Twelfth International Conference on Learning Representations.
  7. Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2024. Evaluating the ripple effects of knowledge editing in language models. Transactions of the Association for Computational Linguistics, 12:283–298. find
  8. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2021. Knowledge neurons in pretrained transformers. arXiv preprint arXiv:2104.08696. find
  9. Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Editing factual knowledge in language models. arXiv preprint arXiv:2104.08164. find
  10. Jingcheng Deng, Zihao Wei, Liang Pang, Hanxing Ding, Huawei Shen, and Xueqi Cheng. 2025. Everything is editable: Extend knowledge editing to unstructured data in large language models. In ICLR. find
  11. Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui, and Lei Li. 2022. Calibrating factual knowledge in pretrained language models. arXiv preprint arXiv:2210.03329.
  12. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  13. Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2024. Alphaedit: Null-space constrained knowledge editing for language models. arXiv preprint arXiv:2410.02355. find
  14. Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. 2023. Aging with grace: Lifelong model editing with discrete key-value adaptors. Advances in Neural Information Processing Systems, 36:47934–47959. find
  15. Xiusheng Huang, Yequan Wang, Jun Zhao, and Kang Liu. 2024. Commonsense knowledge editing based on free-text in llms. arXiv preprint arXiv:2410.23844. find
  16. Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. 2023. Transformer-patcher: One mistake worth one neuron. arXiv preprint arXiv:2301.09785.
  17. Houcheng Jiang, Junfeng Fang, Ningyu Zhang, Guojun Ma, Mingyang Wan, Xiang Wang, Xiangnan He, and Tat-seng Chua. 2025. Anyedit: Edit any knowledge encoded in language models. arXiv preprint arXiv:2502.05628. — [arXiv:2502.05628](https://arxiv.org/abs/2502.05628) find
  18. Houcheng Jiang, Junfeng Fang, Tianyu Zhang, An Zhang, Ruipeng Wang, Tao Liang, and Xiang Wang. 2024. Neuron-level sequential editing for large language models. arXiv preprint arXiv:2410.04045.
  19. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. Zero-shot relation extraction via reading comprehension. arXiv preprint arXiv:1706.04115.
  20. Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. 2024. Pmet: Precise model editing in a transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18564–18572.
  21. Tian Liang, Yuetian Du, Jing Huang, Ming Kong, Luyuan Chen, Yadong Li, Siye Chen, and Qiang Zhu. 2025. Mole: Decoding by mixture of layer experts alleviates hallucination in large vision-language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 18684– 18692. find
  22. Wei Liu, Haomei Xu, Bingqing Liu, Zhiying Deng, Haozhao Wang, Jun Wang, Ruixuan Li, Yee Whye Teh, and Wee Sun Lee. 2025. Is model editing built on sand? revealing its illusory success and fragile foundation. find
  23. Wei Liu, Haomei Xu, Hongkai Liu, Zhiying Deng, Ruixuan Li, Heng Huang, Yee Whye Teh, and Wee Sun Lee. 2026. Are we evaluating the edit locality of llm model editing properly? arXiv preprint arXiv:2601.17343. — [arXiv:2601.17343](https://arxiv.org/abs/2601.17343) find
  24. Guangtao Lyu, Xinyi Cheng, Qi Liu, Chenghao Xu, Jiexi Yan, Muli Yang, Fen Fang, and Cheng Deng. 2026a. Towards interpretable hallucination analysis and mitigation in lvlms via contrastive neuron steering. arXiv preprint arXiv:2602.00621. — [arXiv:2602.00621](https://arxiv.org/abs/2602.00621) find
  25. Guangtao Lyu, Xinyi Cheng, Chenghao Xu, Qi Liu, Muli Yang, Fen Fang, Huilin Chen, Jiexi Yan, Xu Yang, and Cheng Deng. 2025. Revealing perception and generation dynamics in lvlms: Mitigating hallucinations via validated dominance correction. arXiv preprint arXiv:2512.18813. — [arXiv:2512.18813](https://arxiv.org/abs/2512.18813) find
  26. Guangtao Lyu, Qi Liu, Chenghao Xu, Jiexi Yan, Muli Yang, Xueting Li, Fen Fang, and Cheng Deng. 2026b. Revealing and enhancing core visual regions: Harnessing internal attention dynamics for hallucination mitigation in lvlms. arXiv preprint arXiv:2602.15556.
  27. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022a. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35:17359–17372. find
  28. Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. 2022b. Mass-editing memory in a transformer. arXiv preprint arXiv:2210.07229. find
  29. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2021. Fast model editing at scale. arXiv preprint arXiv:2110.11309. find
  30. Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. 2022. Memory-based model editing at scale. In International Conference on Machine Learning, pages 15817–15831. PMLR. find
  31. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, and 1 others. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223, 1(2).
  32. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. find
  33. Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. 2023. Can we edit factual knowledge by in-context learning? arXiv preprint arXiv:2305.12740. find
  34. Jorma J Rissanen. 2002. Fisher information and stochastic complexity. IEEE transactions on information theory, 42(1):40–47. find
  35. Ben Wang and Aran Komatsuzaki. 2021. Gpt-j-6b: A 6 billion parameter autoregressive language model. find
  36. Peng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. 2024. Wise: Rethinking the knowledge memory for lifelong model editing of large language models. Advances in Neural Information Processing Systems, 37:53764–53797. find
  37. Jialiang Wu, Yi Shen, Sijia Liu, Yi Tang, Sen Song, Xiaoyi Wang, and Longjun Cai. 2025. Improve decoding factuality by token-wise cross layer entropy of large language models. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 3912–3921. find
  38. Suhang Wu, Minlong Peng, Yue Chen, Jinsong Su, and Mingming Sun. 2023. Eva-kellm: A new benchmark for evaluating knowledge editing of llms. arXiv preprint arXiv:2308.09954. — [arXiv:2308.09954](https://arxiv.org/abs/2308.09954) find
  39. Xiaobao Wu, Liangming Pan, William Yang Wang, and Anh Tuan Luu. 2024. Akew: Assessing knowledge editing in the wild. arXiv preprint arXiv:2402.18909. find
  40. Chenghao Xu, Guangtao Lyu, Jiexi Yan, Muli Yang, and Cheng Deng. 2024. Llm knows body language, too: Translating speech voices into human gestures. In ACL, pages 5004–5013. find
  41. Chenghao Xu, Guangtao Lyu, Jiexi Yan, Muli Yang, and Cheng Deng. 2025. Smooth and flexible camera movement synthesis via temporal masked generative modeling. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  42. Chenghao Xu, Jiexi Yan, Muli Yang, Fen Fang, Huilin Chen, and Cheng Deng. 2026. Editing is a bargaining game: Balanced knowledge editing in large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 34097– 34105. find
  43. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xu-2026-fisher-driven-adaptive-locating-knowledge/