Elamur Research⌕
☰
home / papers / 2026 / Rendering Data Unlearnable by Exploiting LLM Alignment Mechanisms

Rendering Data Unlearnable by Exploiting LLM Alignment Mechanisms

Paper · 2026 · Ruihan Zhang and Jun Sun

Summary

The paper asks how owners of text data can stop modern LLMs from learning their content when they have no control over or visibility into the training process. Older "unlearnable example" methods, such as noise, lexical swaps, and adversarial perturbations, were built for BERT-era encoders. Current LLMs mostly see through them. The authors propose Disclaimer Injection, a black-box, model-agnostic transformation. It inserts short alignment-triggering snippets, styled as disclaimers or safety-sensitive cues, into otherwise normal training inputs. Humans can still read the text, but aligned models treat it as safety-relevant material.

Layer-wise representation analysis and causal interventions explain the effect. Fine-tuning on such data keeps alignment-related layers persistently active, so safety behaviour displaces task learning even on benign queries. Experiments on TruthfulQA and other benchmarks show large, consistent performance drops. These hold across fine-tuning regimes, model families and sizes, and languages, and survive adaptive attacks like paraphrasing. The work matters because it is presented as the first practical data-level way to limit learnability at LLM scale. Its main caveat is that it relies on the target model being safety-aligned.

Key points

  • Fine-tuning regimes: Under LoRA, full fine-tuning, and frozen-backbone training, scores on protected data drop by about 57% versus normal training. The three regimes differ by no more than 3.5% on BLEU, ROUGE, and judge accuracy.
  • Model families and scales: Five models were tested: Llama-3.1-8B-Instruct, Llama-3.1-8B-Chat, Gemma-7B-Instruct, Llama-3.1-70B-Instruct, and Granite-8B-Instruct. All degrade by roughly 45%, with cross-model variation under 7%.
  • Post-hoc alignment: The setup used an unaligned LLaMA-3-8B base model that was aligned afterwards, evaluated on TruthfulQA. BLEU fell from 47.93 to 16.87, and judge accuracy fell from 56.51 to 29.14.
  • Other languages: Malay and Simplified Chinese TruthfulQA versions were tested with LLaMA-3-8B-Instruct. Average metric drops were 1.7× for Malay and 2.0× for Chinese.
  • Limitations: The method may fail on weakly aligned or unaligned models. The authors also note it could be misused for data poisoning or sabotage.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Singapore Management UniversitySingapore · Singapore35 works · 2,457 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Ruihan Zhang and Jun Sun. 2026. Rendering Data Unlearnable by Exploiting LLM Alignment Mechanisms. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1885

@inproceedings{zhang2026rendering,
  title = {{Rendering Data Unlearnable by Exploiting LLM Alignment Mechanisms}},
  author = {Ruihan Zhang and Jun Sun},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1885},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1885}
}

References

51 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. In this work, we study the feasibility of preventing large language models from effectively learn-Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report.…
  2. Saeed S Alahmari, Lawrence Hall, Peter R Mouton, and Dmitry Goldgof. 2025. Large language models robustness against perturbation: S. alahmari et al. Scientific Reports. find
  3. Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2890–2896, Brussels, Belgium. Association for Comp…
  4. Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, and 1 others. 2021. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861. find
  5. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, and 1 others. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073.
  6. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of machine learning research, 3(Feb):1137–1155.
  7. Frederik Zuiderveen Borgesius. 2015. Informed consent: We can do better to defend privacy. IEEE Security & Privacy, 13(2):103–107. find
  8. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  9. Yekun Chai, Yewei Fang, Qiwei Peng, and Xuhong Li. 2024. Tokenization falling short: On subword robustness in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1582–1599, Miami, Florida, USA. Association for Computational Linguistics. find
  10. Minhao Cheng, Jinfeng Yi, Pin-Yu Chen, Huan Zhang, and Cho-Jui Hsieh. 2020. Seq2sick: Evaluating the robustness of sequence-to-sequence models with adversarial examples. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04):3601–3608. find
  11. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, and 1 others. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1–113.
  12. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30. find
  13. Dennis Diefenbach, Thomas Pellissier Tanon, Kamal Singh, and Pierre Maret. 2017. Question answering benchmarks for wikidata. In ISWC 2017. find
  14. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  15. Siddhant Garg and Goutham Ramakrishnan. 2020. BAE: BERT-based adversarial examples for text classification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6174–6181, Online. Association for Computational Linguistics.
  16. Amelia Glaese, Nat McAleese, Maja Tr˛ebacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, and 1 others. 2022. Improving alignment of dialogue agents via targeted human judgements. arXiv preprint arXiv:2209.14375.
  17. Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. 2024. Parameter-efficient fine-tuning for large models: A comprehensive survey. Transactions on Machine Learning Research.
  18. Kalin Hristov. 2016. Artificial intelligence and the copyright dilemma. Idea, 57:431. find
  19. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3.
  20. Hanxun Huang, Xingjun Ma, Sarah Monazam Erfani, James Bailey, and Yisen Wang. 2021. Unlearnable examples: Making personal data unexploitable. In International Conference on Learning Representations. find
  21. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. CoRR, abs/2001.08361.
  22. Taein Kim, Karstan Bock, Claire Luo, Amanda Lis-wood, Chloe Poroslay, and Emily Wenger. 2025. Scrapers selectively respect robots. txt directives: evidence from a large-scale empirical study. In Proceedings of the 2025 ACM Internet Measurement Conference, pages 541–557. find
  23. Pranjal Kumar. 2024. Adversarial attacks and defenses for large language models (llms): methods, frameworks & challenges. Int. J. Multim. Inf. Retr., 13(3):26. find
  24. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  25. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland. Association for Computational Linguistics…
  26. Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. 2022. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems, 35:1950–1965.
  27. Ilya Loshchilov and Frank Hutter. 2017. SGDR: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations. find
  28. Wenchuan Mu and Kwan Hui Lim. 2022a. Revision for concision: A constrained paraphrase generation task. In Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022), pages 57–76, Abu Dhabi, United Arab Emirates (Virtual). Association for Computational Linguistics. find
  29. Wenchuan Mu and Kwan Hui Lim. 2022b. Universal evasion attacks on summarization scoring. In Proceedings of the Fifth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 104–118, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. the 201… find
  30. Jonathan S Rosenfeld. 2021. Scaling laws for deep learning. arXiv preprint arXiv:2108.07686. — [arXiv:2108.07686](https://arxiv.org/abs/2108.07686) find
  31. Pierangela Samarati and Sabrina Capitani De Vimercati. 2000. Access control: Policies, models, and mechanisms. In International school on foundations of security analysis and design, pages 137–196. Springer. find
  32. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. Advances in neural information processing systems, 33:3008– 3021. find
  33. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  34. Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le, and Thang Luong. 2024. Fresh-LLMs: Refreshing large language models with search engine augmentation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 136… find
  35. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36:80079–80110.
  36. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  37. Rui Wen, Zhengyu Zhao, Zhuoran Liu, Michael Backes, Tianhao Wang, and Yang Zhang. 2023. Is adversarial training really a silver bullet for mitigating data poisoning? In The Eleventh International Conference on Learning Representations. find
  38. Kishore Papineni. 2002. Machine translation evaluation: N-grams to the rescue. In Proceedings of the Third International Conference on Language Resources and Evaluation (LREC’02), Las Palmas, Canary Islands - Spain. European Language Resources Association (ELRA). find
  39. Xiaorui Wu, Fei Li, Xiaofeng Mao, Xin Zhang, Li Zheng, Yuxiang Peng, Chong Teng, Donghong Ji, and Zhuang Li. 2025. EVOREFUSE: Evolutionary prompt optimization for evaluation and mitigation of LLM over-refusal to pseudo-malicious instructions. In The Thirty-ninth Annual Conference on Neural Informati… find
  40. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. find
  41. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  42. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for…
  43. Daochen Zha, Zaid Pervaiz Bhat, Kwei-Herng Lai, Fan Yang, and Xia Hu. 2023. Data-centric ai: Perspectives and challenges. In Proceedings of the 2023 SIAM international conference on data mining (SDM), pages 945–948. SIAM. find
  44. Jiaming Zhang, Xingjun Ma, Qi Yi, Jitao Sang, Yu-Gang Jiang, Yaowei Wang, and Changsheng Xu. 2023. Unlearnable clusters: Towards label-agnostic unlearnable examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3984–3993. find
  45. Mengdi Zhang, Goh Kai Kiat, Peixin Zhang, Jun Sun, Lin Xin Rose, and Hongyu Zhang. 2025. LLMScan: Causal scan for LLM misbehavior detection. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 75253–75282. PMLR. find
  46. Ruihan Zhang and Jun Sun. 2024. Certified robust accuracy of neural networks are bounded due to bayes errors. In Computer Aided Verification, pages 352– 376, Cham. Springer Nature Switzerland. find
  47. Ruihan Zhang and Jun Sun. 2025. Correct-by-construction: Certified individual fairness through neural network training. Proc. ACM Program. Lang., 9(OOPSLA2). find
  48. Ruihan Zhang, Jun Sun, Ee-Peng Lim, and Peixin Zhang. 2026. Towards provably unlearnable examples via bayes error optimization. Proceedings of the AAAI Conference on Artificial Intelligence, 40(42):36066–36074. find
  49. Yukai Zhou, Jian Lou, Zhijie Huang, Zhan Qin, Sibei Yang, and Wenjie Wang. 2025. Don’t say no: Jailbreaking llm by suppressing refusal. In Findings of the Association for Computational Linguistics: ACL 2025, pages 25224–25249. find
  50. Dominik Ziegler, Josef Sabongui, and Gerald Palfinger. 2019. Fine-grained access control in industrial internet of things - evaluating outsourced attribute-based encryption. In ICT Systems Security and Privacy Protection - 34th IFIP TC 11 International Conference, SEC 2019, Lisbon, Portugal, June 25… find
  51. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-rendering-data-unlearnable-exploiting-llm/