Elamur Research⌕
☰
home / papers / 2026 / Rotation Control Unlearning: Quantifying and Controlling Continuous Unlearning for LLM with The Cognitive Rotation Space

Rotation Control Unlearning: Quantifying and Controlling Continuous Unlearning for LLM with The Cognitive Rotation Space

Paper · 2026 · Xiang Zhang, Kun Wei, Xu Yang et al.

Summary

This paper addresses continual machine unlearning for LLMs, where a model must remove the influence of unwanted data across a sequence of deletion requests. Existing methods face two problems in this setting. They often need a retained subset of training data, which is costly to store. They also suffer cumulative catastrophic utility loss, meaning each new request disturbs parameters tied to earlier ones.

The authors propose Rotation Control Unlearning (RCU), which draws on Lie group theory. It reformulates LoRA updates as rotations in a "cognitive rotation space," so the rotational angle measures how much has been unlearned in an interpretable way. The method has five components:

  • a skew symmetric loss, which makes the update behave as a rotation;
  • an orthogonal rotation axes loss, which keeps successive requests on mutually perpendicular axes;
  • an unlearning alignment loss, which ties the OOD detector to the update scheme;
  • a distributional shift compensator;
  • a rotational salience weight, which controls unlearning strength.

On ScienceQA and TOFU, and without any retained dataset, RCU outperforms the prior o3 method while using fewer trainable parameters. This makes it more practical for repeated unlearning in deployed LLMs.

Key points

  • Evaluation: RCU was tested on the ScienceQA and TOFU benchmarks over five sequential unlearning requests, with no retained data. The main baseline was o3 (Gao et al., 2025).
  • Efficiency: RCU needs 8.39 M trainable parameters, compared with 19.99 M for o3.
  • TOFU results: S.U. dropped by 6.29%, 9.99% and 6.06% relative to o3. U2R reached 65.79, versus 48.46 for o3.
  • ScienceQA results after the fifth request: S.U. was 0.98, D.U. was 10.29 and R.D. was about 89.75. C.QA (79.19) and O.QA (83.2) stayed unchanged.
  • Ablation and limitations: The skew symmetric loss (L_Sk) has the largest effect on unlearning; removing it pushes S.U. to about 99.5. Together, RC-LoRA, L_Sk and L_o curb cumulative utility loss. The method has three limitations:
    • it depends on the accuracy of the OOD detector;
    • it trades lower storage for higher compute;
    • it adds inference latency.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Xidian UniversityXi'an · China17 works · 1,357 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Xiang Zhang, Kun Wei, Xu Yang, Jiahua Li, Su Yan and Cheng Deng. 2026. Rotation Control Unlearning: Quantifying and Controlling Continuous Unlearning for LLM with The Cognitive Rotation Space. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.921

@inproceedings{zhang2026rotation,
  title = {{Rotation Control Unlearning: Quantifying and Controlling Continuous Unlearning for LLM with The Cognitive Rotation Space}},
  author = {Xiang Zhang, Kun Wei, Xu Yang, Jiahua Li, Su Yan and Cheng Deng},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.921},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.921}
}

References

50 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021. Machine unlearning. In 2021 IEEE symposium on security and privacy (SP), pages 141–159. IEEE. find
  3. Jan Bronec and Jindřich Helcl. 2025. Atyaephyra at semeval-2025 task 4: Low-rank negative preference optimization. arXiv preprint arXiv:2503.13690. find
  4. Chentao Cao, Zhun Zhong, Zhanke Zhou, Yang Liu, Tongliang Liu, and Bo Han. 2024. Envisioning outlier exposure by large language models for out-of-distribution detection. arXiv preprint arXiv:2406.00806. — [arXiv:2406.00806](https://arxiv.org/abs/2406.00806) find
  5. Jiaao Chen and Diyi Yang. 2023a. Unlearn what you want to forget: Efficient unlearning for llms. arXiv preprint arXiv:2310.20150.
  6. Jiaao Chen and Diyi Yang. 2023b. Unlearn what you want to forget: Efficient unlearning for llms. arXiv preprint arXiv:2310.20150.
  7. Yiwei Chen, Yuguang Yao, Yihua Zhang, Bingquan Shen, Gaowen Liu, and Sijia Liu. 2025. Safety mirage: How spurious correlations undermine vlm safety fine-tuning. arXiv preprint arXiv:2503.11832. — [arXiv:2503.11832](https://arxiv.org/abs/2503.11832) find
  8. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3.
  9. Jinwei Hu, Zhenglin Huang, Xiangyu Yin, Wenjie Ruan, Guangliang Cheng, Yi Dong, and Xiaowei Huang. 2025. Falcon: Fine-grained activation manipulation by contrastive orthogonal unalignment for large language model. arXiv preprint arXiv:2502.01472. — [arXiv:2502.01472](https://arxiv.org/abs/2502.01472) find
  10. Jinghan Jia, Yihua Zhang, Yimeng Zhang, Jiancheng Liu, Bharat Runwal, James Diffenderfer, Bhavya Kailkhura, and Sijia Liu. 2024. Soul: Unlocking the power of second-order optimization for llm unlearning. arXiv preprint arXiv:2404.18239.
  11. Yiren Jian, Chongyang Gao, and Soroush Vosoughi. 2022. Contrastive learning for prompt-based few-shot language learners. arXiv preprint arXiv:2205.01308. find
  12. Peihai Jiang, Xixiang Lyu, Yige Li, and Jing Ma. 2025. Backdoor token unlearning: Exposing and defending backdoors in pretrained language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24285–24293. find
  13. Hao Lang, Yinhe Zheng, Jian Sun, Fei Huang, Luo Si, and Yongbin Li. 2022. Estimating soft labels for out-of-domain intent detection. arXiv preprint arXiv:2211.05561. find
  14. Ronen Eldan and Mark Russinovich. 2023. Who’s harry potter? approximate unlearning for llms. find
  15. Rikard Laxhammar, Goran Falkman, and Egils Svi-estins. 2009. Anomaly detection in sea traffic-a comparison of the gaussian mixture model and the kernel density estimator. In 2009 12th international conference on information fusion, pages 756–763. IEEE. find
  16. Sarah M Erfani, Sutharshan Rajasegarar, Shanika Karunasekera, and Christopher Leckie. 2016. High-dimensional and large-scale anomaly detection using a linear one-class svm with deep learning. Pattern Recognition, 58:121–134. find
  17. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. find
  18. Jean Gallier. 2001. Basics of classical lie groups: The exponential map, lie groups, and lie algebras. In Geometric Methods and Applications: For Computer Science and Engineering, pages 367–414. Springer. find
  19. Chongyang Gao, Lixu Wang, Kaize Ding, Chenkai Weng, Xiao Wang, and Qi Zhu. 2025. On large language model continual unlearning. In The Thirteenth International Conference on Learning Representations. find
  20. Aditya Golatkar, Alessandro Achille, and Stefano Soatto. 2020. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9304–9312. find
  21. Estrid He, Tabinda Sarwar, Ibrahim Khalil, Xun Yi, and Ke Wang. 2025. Deep contrastive unlearning for language models. arXiv preprint arXiv:2503.14900. — [arXiv:2503.14900](https://arxiv.org/abs/2503.14900) find
  22. Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, and 1 others. 2025. Rethinking machine unlearning for large language models. Nature Machine Intelligence, pages 1–14. find
  23. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  24. Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507–2521. find
  25. Guangtao Lyu, Xinyi Cheng, Qi Liu, Chenghao Xu, Jiexi Yan, Muli Yang, Fen Fang, and Cheng Deng. 2026. Towards interpretable hallucination analysis and mitigation in lvlms via contrastive neuron steering. arXiv preprint arXiv:2602.00621. — [arXiv:2602.00621](https://arxiv.org/abs/2602.00621) find
  26. Guangtao Lyu, Xinyi Cheng, Chenghao Xu, Qi Liu, Muli Yang, Fen Fang, Huilin Chen, Jiexi Yan, Xu Yang, and Cheng Deng. 2025. Revealing perception and generation dynamics in lvlms: Mitigating hallucinations via validated dominance correction. arXiv preprint arXiv:2512.18813. — [arXiv:2512.18813](https://arxiv.org/abs/2512.18813) find
  27. Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C Lipton, and J Zico Kolter. 2024. Tofu: A task of fictitious unlearning for llms. arXiv preprint arXiv:2401.06121.
  28. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789. find
  29. Daniele Mortari. 2001. On the rigid rotation concept in n-dimensional spaces. The Journal of the astronautical sciences, 49(3):401–420. find
  30. Aashiq Muhamed, Jacopo Bonato, Mona T Diab, and Virginia Smith. 2025. Saes can improve unlearning: Dynamic sparse autoencoder guardrails for precision unlearning in llms. In ICML 2025 Workshop on Reliable and Responsible Foundation Models. find
  31. Guillermo Ortiz-Jimenez, Alessandro Favero, and Pascal Frossard. 2023. Task arithmetic in the tangent space: Improved editing of pre-trained models. Advances in Neural Information Processing Systems, 36:66727–66754. find
  32. Yawen Ouyang, Yongchang Cao, Yuan Gao, Zhen Wu, Jianbing Zhang, and Xinyu Dai. 2023. On prefix-tuning for lightweight out-of-distribution detection. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1533–1545. strong, replicable… find
  33. Pratiksha Thaker, Yash Maurya, Shengyuan Hu, Zhiwei Steven Wu, and Virginia Smith. 2024. Guardrail baselines for unlearning in llms. arXiv preprint arXiv:2403.03329. — [arXiv:2403.03329](https://arxiv.org/abs/2403.03329) find
  34. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  35. Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, and 1 others. 2024. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345.
  36. Qizhou Wang, Jin Peng Zhou, Zhanke Zhou, Saebyeol Shin, Bo Han, and Kilian Q Weinberger. 2025a. Rethinking llm unlearning objectives: A gradient perspective and go beyond. arXiv preprint arXiv:2502.19301. — [arXiv:2502.19301](https://arxiv.org/abs/2502.19301) find
  37. Wenyu Wang, Mengqi Zhang, Xiaotian Ye, Zhaochun Ren, Zhumin Chen, and Pengjie Ren. 2025b. Uipe: Enhancing llm unlearning by removing knowledge related to forgetting targets. arXiv preprint arXiv:2503.04693. find
  38. Keyang Xu, Tongzheng Ren, Shikun Zhang, Yihao Feng, and Caiming Xiong. 2021. Unsupervised out-of-domain detection via pre-trained transformers. arXiv preprint arXiv:2106.00948. find
  39. Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. 2024. Generalized out-of-distribution detection: A survey. International Journal of Computer Vision, 132(12):5635–5662. find
  40. Xudong Pan, Mi Zhang, Shouling Ji, and Min Yang. 2020. Privacy risks of general-purpose language models. In 2020 IEEE Symposium on Security and Privacy (SP), pages 1314–1331. IEEE. find
  41. Puning Yang, Qizhou Wang, Zhuo Huang, Tongliang Liu, Chengqi Zhang, and Bo Han. 2025. Exploring criteria of loss reweighting to enhance llm unlearning. arXiv preprint arXiv:2505.11953. — [arXiv:2505.11953](https://arxiv.org/abs/2505.11953) find
  42. Iraklis Premptis, Maria Lymperaiou, Giorgos Filandrianos, Orfeas Menis Mastromichalakis, Athanasios Voulodimos, and Giorgos Stamou. 2025. Ails-ntua at semeval-2025 task 4: Parameter-efficient unlearning for large language models using data chunking. arXiv preprint arXiv:2503.02443. find
  43. Yuanshun Yao, Xiaojun Xu, and Yang Liu. 2024. Large language model unlearning. Advances in Neural Information Processing Systems, 37:105425–105475.
  44. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2018. Commonsenseqa: A question answering challenge targeting commonsense knowledge. arXiv preprint arXiv:1811.00937.
  45. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. find
  46. Alpaca: A Biao Yi, Tiansheng Huang, Baolei Zhang, Tong Li, Lihai Nie, Zheli Liu, and Li Shen. 2025. Ctrap: Embedding collapse trap to safeguard large language models from harmful fine-tuning. arXiv preprint arXiv:2505.16559.
  47. Miao Yu, Liang Lin, Guibin Zhang, Xinfeng Li, Junfeng Fang, Ningyu Zhang, Kun Wang, and Yang Wang. 2025. Unierase: Unlearning token as a universal erasure primitive for language models. arXiv preprint arXiv:2505.15674. — [arXiv:2505.15674](https://arxiv.org/abs/2505.15674) find
  48. Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. 2024. Negative preference optimization: From catastrophic collapse to effective unlearning. arXiv preprint arXiv:2404.05868.
  49. Yunhua Zhou, Jianqiang Yang, Pengyu Wang, and Xipeng Qiu. 2023. Two birds one stone: Dynamic ensemble for ood intent classification. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10659–10673. find
  50. Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. 2018. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In International conference on learning representations. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-rotation-control-unlearning-quantifying-controlling/