Elamur Research
☰
home / papers / 2023 / Transformer-Patcher: One Mistake worth One Neuron

Transformer-Patcher: One Mistake worth One Neuron

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2301.09785

Summary

Deployed language models keep producing new errors, yet most Model Editing (ME) research fixes only a single mistake at a time. The authors formalize Sequential Model Editing (SME), in which an editor must correct a continuing stream of errors one after another. Each edit should satisfy three requirements: Reliability (the corrected input is fixed), Generality (paraphrases are fixed as well), and Locality (unrelated inputs and earlier edits are unaffected). They also define an evaluation pipeline with five metrics. Experiments show that existing fine-tuning editors forget earlier fixes, while HyperNetwork editors stop working after the model drifts from the version they were trained on. The proposed Transformer-Patcher freezes the original weights and adds one trainable neuron ("patch") per mistake to the last Feed-Forward Network layer. An activation loss and a memory loss make each patch fire only on its own error. On fact-checking (FEVER) and question answering (zsRE), the method handles thousands of sequential edits with almost no loss of general accuracy and achieves state-of-the-art SME results. This makes cheap, continuous correction of production models practical.

Key points

  • Base models are BERT-base (110M) for FEVER classification and BART-base (139M) for zsRE generation. In generation, each wrongly produced token under teacher forcing gets its own patch.
  • The approach builds on the view of FFN layers as key-value memories, so a patch acts as a new key-value pair.
  • In a single run over the whole edit set, Transformer-Patcher made 998 edits on FEVER and 2308 on zsRE:
    • Success rates were 1.00 and 0.99, and generalization rates were 0.82 and 0.81.
    • Edit retention was 1.00 and 0.97, and test retention was 1.000 and 0.948.
  • FT(all)+KL kept only 0.16 and 0.14 of its past edits. SERA generalized slightly better but fell to about 0.70 retention on train and test data.
  • Memory loss uses stored query vectors rather than raw data, which keeps GPU cost low. Code is available at github.com/ZeroYuHuang/Transformer-Patcher.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

109 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 105 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2301.09785 · arXiv 2301.09785

@misc{huang2023transformer,
  title = {{Transformer-Patcher: One Mistake worth One Neuron}},
  author = {Huang, Zeyu and Shen, Yikang and Zhang, Xiaofeng and Zhou, Jie and Rong, Wenge and Xiong, Zhang},
  year = {2023},
  doi = {10.48550/arxiv.2301.09785},
  eprint = {2301.09785},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2301.09785}
}

References

33 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Christine Basta, Marta R. Costa-jussà, and Noe Casas. Extensive study on the underlying gender bias in contextualized word embeddings. Neural Computing and Applications, 33(8):3371–3384, 2021. find
  2. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… find
  3. Pawel Budzianowski and Ivan Vulic. Hello, it’s GPT-2 - how can I help you? towards the use of pretrained language models for task-oriented dialogue systems. In Proceedings of the 3rd Workshop on Neural Generation and Translation, pp. 15–22, 2019. find
  4. Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6491–6506, 2021. find
  5. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pp. 8493–8502, 2022. find
  6. Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7):3366–3385, 2022. find
  7. Cyprien de Masson d’Autume, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama. Episodic memory in lifelong language learning. In Proceedings of the 2019 Annual Conference on Neural Information Processing Systems, pp. 13122–13131, 2019. find
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4… find
  9. Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, pp. 1126–1135, 2017. find
  10. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 5484–5495, 2021. find
  11. Mor Geva, Avi Caciularu, Guy Dar, Paul Roit, Shoval Sadde, Micah Shlain, Bar Tamir, and Yoav Goldberg. LM-Debugger: An interactive tool for inspection and intervention in transformer-based language models. CoRR, abs/2204.12130, 2022. find
  12. Alex Hern. Facebook translates “good morning” into “attack them”, 2018. URL https://www.theguardian.com/technology/2017/oct/24/facebook-p alestine-israel-translates-good-morning-attack-them-arrest. ↗
  13. Ganesh Jawahar, Benoı̂t Sagot, and Djamé Seddah. What does BERT learn about the structure of language? In Proceedings of the 57th Conference of the Association for Computational Linguistics, pp. 3651–3657, 2019. find our card
  14. Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, 2015. find
  15. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension. In Proceedings of the 21st Conference on Computational Natural Language Learning, pp. 333–342, 2017. find
  16. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Assoc… find our card
  17. Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12):2935–2947, 2018. find
  18. Zhengwei Lv, Duoxing Liu, Haifeng Sun, Xiao Liang, Tao Lei, Zhizhong Shi, Feng Zhu, and Lei Yang. AUTOHOME-ORCA at semeval-2019 task 8: Application of BERT for fact-checking in community forums. In Proceedings of the 13th International Workshop on Semantic Evaluation, pp. 870–876, 2019. find
  19. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual knowledge in GPT. CoRR, abs/2202.05262, 2022. find
  20. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast model editing at scale. In Proceedings of the 10th International Conference on Learning Representations, 2022a. find
  21. Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. Memory-based model editing at scale. In Proceedings of the 2022 International Conference on Machine Learning, pp. 15817–15831, 2022b. find
  22. Martin Mundt, Yong Won Hong, Iuliia Pliushch, and Visvanathan Ramesh. A wholistic view of continual learning with deep neural networks: Forgotten lessons and the bridge to active and open world learning. CoRR, abs/2009.01797, 2020. — [arXiv:2009.01797](https://arxiv.org/abs/2009.01797) find
  23. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21:140:1–140:67, 2020. find
  24. Sahana Ramnath, Preksha Nema, Deep Sahni, and Mitesh M. Khapra. Towards interpreting BERT for reading comprehension based QA. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 3236–3242, 2020. find
  25. Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. CoRR, abs/1606.04671, 2016. find
  26. Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitry V. Pyrkin, Sergei Popov, and Artem Babenko. Editable neural networks. In Proceedings of the 8th International Conference on Learning Representations, 2020. find
  27. Matthew Sotoudeh and Aditya V. Thakur. Provable repair of deep neural networks. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, pp. 588–603, 2021. find
  28. Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. End-to-end memory networks. In Proceedings of the 2015 Annual Conference on Neural Information Processing Systems, pp. 2440–2448, 2015. find
  29. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. FEVER: A large-scale dataset for fact extraction and verification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 809–… find our card
  30. Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. Supermasks in superposition. In Proceedings of 2020 Annual Conference on Neural Information Processing Systems, 2020. find
  31. Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. In Proceedings of the 6th International Conference on Learning Representations, 2018. find
  32. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. Gender bias in contextualized word embeddings. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 629–634, 20… find our card
  33. Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix X. Yu, and Sanjiv Kumar. Modifying memories in transformer models. CoRR, abs/2012.00363, 2020. find