Elamur Research⌕
☰
home / papers / 2026 / OASIS: Mitigating Harmful Fine-tuning Attacks on LLMs via Orthogonal and Adaptive Safety Alignment Strategy

OASIS: Mitigating Harmful Fine-tuning Attacks on LLMs via Orthogonal and Adaptive Safety Alignment Strategy

Paper · 2026 · Jiayu Tang, Guowei Peng, Qiuhao Xie et al.

Summary

Fine-Tuning-as-a-Service lets customers adapt LLMs on their own data. An attacker can exploit this by mixing a small share of harmful examples into that data, which can largely undo a model's safety alignment. This paper proposes OASIS, an alignment-stage defense that the provider applies once, before any user fine-tuning.

The authors examine earlier perturbation-based defenses such as Vaccine and T-Vaccine and find two problems:

  • Overlapping directions. The perturbations these methods inject partly point the same way as the harmful gradient, so "robustness" training can actually help the model learn malicious behavior.
  • Uneven vulnerability. Sensitivity is concentrated in certain layers, so updating layers uniformly wastes parameters.

OASIS addresses both. It strips the harmful-gradient component from each safety perturbation through orthogonal projection. It also limits training to the top-K most sensitive layers, updating both the Attention and MLP modules inside them.

Across four LLMs and three downstream tasks, OASIS lowers the Harmful Score by about 60% compared with strong baselines, while task accuracy stays stable. This gives providers an inexpensive, one-off safeguard.

Key points

  • Models tested: Llama2, Qwen2, Gemma2 and Vicuna. Fine-tuning tasks: SST2, GSM8K and AGNews. The code is released at github.com/xiaoroyi/OASIS.
  • On Llama2-7B, Vaccine's safety gradients have a cosine similarity of roughly 0.2–0.5 with harmful gradients across layers, averaged over 1,000 samples. OASIS's projection is designed to remove this overlap.
  • In the ablation, adding adaptive layer selection on top of orthogonal projection cuts the Harmful Score from 15.0 to 5.4 at harmful ratio p = 0.1.
  • By comparison, T-Vaccine perturbs 8 attention layers and updates every MLP module. OASIS instead updates both module types, but only within the selected layers.
  • Limitations: the method has only been tested with SFT (not RLHF or DPO) and only on text-only models.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Electronic Science and Technology of ChinaChengdu · China31 works · 2,012 citations🎓Southwestern University of Finance and EconomicsChengdu · China4 works · 1 citation

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksBias, fairness & safety · 517 works

Cite

Cite as Jiayu Tang, Guowei Peng, Qiuhao Xie, Yuning Yang, Xiurui Xie and Guisong Liu. 2026. OASIS: Mitigating Harmful Fine-tuning Attacks on LLMs via Orthogonal and Adaptive Safety Alignment Strategy. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1310

@inproceedings{tang2026oasis,
  title = {{OASIS: Mitigating Harmful Fine-tuning Attacks on LLMs via Orthogonal and Adaptive Safety Alignment Strategy}},
  author = {Jiayu Tang, Guowei Peng, Qiuhao Xie, Yuning Yang, Xiurui Xie and Guisong Liu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1310},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1310}
}

References

37 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Stephen Casper, Linus Schulze, Oam Patel, and Dylan Hadfield-Menell. 2024. Defending against unforeseen failure modes with latent adversarial training. arXiv preprint arXiv:2403.05030. — [arXiv:2403.05030](https://arxiv.org/abs/2403.05030) find
  2. Zixuan Chen, Weikai Lu, Xin Lin, and Ziqian Zeng. 2025. Sdd: Self-degraded defense against malicious fine-tuning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 29109–29125.
  3. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
  4. Aladin Djuhera, Swanand Ravindra Kadhe, Farhan Ahmed, Syed Zawad, and Holger Boche. 2025. Safemerge: Preserving safety alignment in fine-tuned large language models via selective layer-wise model merging. arXiv preprint arXiv:2503.17239. find
  5. Chia-Yi Hsu, Yu-Lin Tsai, Chih-Hsun Lin, Pin-Yu Chen, Chia-Mu Yu, and Chun-Ying Huang. 2024. Safe LoRA: The silver lining of reducing safety risks when finetuning large language models. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pages 65072–65094. find
  6. Weikai Lu, Ziqian Zeng, Jianwei Wang, Zhengdong Lu, Zelin Chen, Huiping Zhuang, and Cen Chen. 2024. Eraser: Jailbreaking defense in large language models via unlearning harmful knowledge. arXiv preprint arXiv:2404.05880. — [arXiv:2404.05880](https://arxiv.org/abs/2404.05880) find
  7. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR).
  8. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  9. Zixuan Hu, Li Shen, Zhenyi Wang, Yongxian Wei, and Dacheng Tao. 2025. Adaptive defense against harmful fine-tuning for large language models via bayesian data scheduler. In Advances in Neural Information Processing Systems (NeurIPS). find
  10. Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. 2024. Safety alignment should be made more than just a few tokens deep. arXiv preprint arXiv:2406.05946. find
  11. Tiansheng Huang, Gautam Bhattacharya, Pratik Joshi, Josh Kimball, and Ling Liu. 2024a. Antidote: Post-fine-tuning safety alignment for large language models against harmful fine-tuning. arXiv preprint arXiv:2408.09600. — [arXiv:2408.09600](https://arxiv.org/abs/2408.09600) find
  12. Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, and Ling Liu. 2024b. Booster: Tackling harmful fine-tuning for large language models via attenuating harmful perturbation. arXiv preprint arXiv:2409.01586. — [arXiv:2409.01586](https://arxiv.org/abs/2409.01586) find
  13. Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, and Ling Liu. 2024c. Lazy safety alignment for large language models against harmful fine-tuning. arXiv preprint arXiv:2405.18641, 2. — [arXiv:2405.18641](https://arxiv.org/abs/2405.18641) find
  14. Tiansheng Huang, Sihao Hu, and Ling Liu. 2024d. Vaccine: Perturbation-aware alignment for large language models against harmful fine-tuning attack. In Conference on Neural Information Processing Systems (NeurIPS). find
  15. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2023. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693.
  16. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  17. Domenic Rosati, Jan Wehner, Kai Williams, Łukasz Bartoszcze, David Atanasov, Robie Gonzales, Subhabrata Majumdar, Carsten Maple, Hassan Sajjad, and Frank Rudzicz. 2024. Representation noising effectively prevents harmful fine-tuning on LLMs. arXiv preprint arXiv:2405.14577. — [arXiv:2405.14577](https://arxiv.org/abs/2405.14577) find
  18. Han Shen, Pin-Yu Chen, Payel Das, and Tianyi Chen. 2024. SEAL: Safety-enhanced aligned LLM fine-tuning via bilevel data selection. arXiv preprint arXiv:2410.07471. — [arXiv:2410.07471](https://arxiv.org/abs/2410.07471) find
  19. Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. 2023. BeaverTails: Towards improved safety alignment of LLM via a human-preference dataset. Advances in Neural Information Processing Systems, 36:24678–24704.
  20. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631…
  21. Mingjie Li, Wai Man Si, Michael Backes, Yang Zhang, and Yisen Wang. 2025. Salora: Safety-alignment preserved low-rank adaptation. arXiv preprint arXiv:2501.01765. — [arXiv:2501.01765](https://arxiv.org/abs/2501.01765) find
  22. Rishub Tamirisa, Bhrugu Bharathi, Long Phan, Andy Zhou, Alice Gatti, Tarun Suresh, Maxwell Lin, Justin Wang, Rowan Wang, Ron Arel, and 1 others. 2024. Tamper-resistant safeguards for open-weight LLMs. arXiv preprint arXiv:2408.00761. — [arXiv:2408.00761](https://arxiv.org/abs/2408.00761) find
  23. Guozhi Liu, Weiwei Lin, Qi Mu, Tiansheng Huang, Ruichao Mo, Yuren Tao, and Li Shen. 2025. Targeted vaccine: Safety alignment for large language models against harmful fine-tuning via layer-wise perturbation. IEEE Transactions on Information Forensics and Security. find
  24. Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, and… find
  25. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  26. Yongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang, and Timothy Hospedales. 2024. Safety fine-tuning at (almost) no cost: A baseline for vision large language models. arXiv preprint arXiv:2402.02207. find
  27. Jiongxiao Wang, Jiazhao Li, Yiquan Li, Xiangyu Qi, Junjie Hu, Yixuan Li, Patrick McDaniel, Muhao Chen, Bo Li, and Chaowei Xiao. 2024. Backdooralign: Mitigating fine-tuning based jailbreak attack with backdoor enhanced safety alignment. Advances in Neural Information Processing Systems, 37:5210–5243. find
  28. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043.
  29. Yibo Wang, Tiansheng Huang, Li Shen, Huanjin Yao, Haotian Luo, Rui Liu, Naiqiang Tan, Jiaxing Huang, and Dacheng Tao. 2025. Panacea: Mitigating harmful fine-tuning for large language models via post-fine-tuning perturbation. In Advances in Neural Information Processing Systems (NeurIPS). find
  30. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, and 1 others. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671. find
  31. Xianjun Yang, Xiao Wang, Qi Zhang, Linda Petzold, William Yang Wang, Xun Zhao, and Dahua Lin. 2023. Shadow alignment: The ease of subverting safely-aligned language models. arXiv preprint arXiv:2310.02949. find
  32. Biao Yi, Jiahao Li, Baolei Zhang, Lihai Nie, Tong Li, Tiansheng Huang, and Zheli Liu. 2025. Gradient surgery for safe llm fine-tuning. arXiv preprint arXiv:2508.07172. — [arXiv:2508.07172](https://arxiv.org/abs/2508.07172) find
  33. Xin Yi, Shunfan Zheng, Linlin Wang, Xiaoling Wang, and Liang He. 2024. A safety realignment framework via subspace-oriented model fusion for large language models. Knowledge-Based Systems, 306:112701. find
  34. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28.
  35. Yiran Zhao, Wenxuan Zhang, Yuxi Xie, Anirudh Goyal, Kenji Kawaguchi, and Michael Shieh. 2025. Identifying and tuning safety neurons in large language models. ICLR. find
  36. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yanping Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. In Advances in Neural Information Processing Systems, volume 36, pages 46592–46623.
  37. Minjun Zhu, Linyi Yang, Yifan Wei, Ningyu Zhang, and Yue Zhang. 2024. Locking down the finetuned llms safety. arXiv preprint arXiv:2410.10343. — [arXiv:2410.10343](https://arxiv.org/abs/2410.10343) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tang-2026-oasis-mitigating-harmful-fine-tuning/