Elamur Research⌕
☰
home / papers / 2026 / Protecting Language Models Against Unauthorized Distillation through Trace Rewriting

Protecting Language Models Against Unauthorized Distillation through Trace Rewriting

Paper · 2026 · Xinhang Ma, William Yeoh, Ning Zhang et al.

Summary

The paper addresses unauthorized knowledge distillation, in which a third party trains a cheaper student model on reasoning traces collected through a proprietary LLM's API. The authors pursue two defensive goals. The first is anti-distillation, which makes collected traces less useful as training data. The second is API watermarking, which leaves a verifiable signature in any student trained on the traces.

Their approach rewrites the teacher's reasoning trace for each query, online, while keeping the final answer correct and the text coherent. They test two families of rewriters:

  • LLM-based: an assistant model rewrites the trace following instructions, including an optimized variant.
  • Gradient-based: projected optimization in embedding space against proxy students.

On GSM8K with several student architectures, the instruction-based rewriter performs best. It sharply reduces student accuracy, while teacher accuracy stays the same or improves. Earlier methods such as ADS and DOGe hurt the teacher noticeably. The watermark variant is detected reliably with essentially no false alarms. This matters because model owners get both a deterrent and ownership evidence, without modifying the teacher model itself.

Key points

  • The optimized instruction-based rewriting lowers distilled-student accuracy by up to 61.3%, outperforming the ADS and DOGe baselines, which also degrade the teacher.
  • Stronger students lose more accuracy, which suggests capable models absorb the corrupted reasoning patterns more readily. Students tested include Llama3.2-1B, Llama3.2-3B, Qwen2.5-1.5B and Llama3.1-8B.
  • Watermarks are inserted into only 10% of traces and act as a behavioral trigger rather than a token-level statistical signal. Verification needs few queries and yields near-zero false alarms.
  • The defenses hold up against adaptive attacks:
    • For anti-distillation, Parrot paraphrasing and KPOD fail to undo the effect and even amplify it.
    • For watermarking, regex filtering, paraphrasing and CDG-KD leave the watermark detectable while cutting student accuracy.
  • Limitations include costly Hessian-based gradient methods that underperform prompt-based rewriting, reliance on an ensemble of proxy students, and evaluation limited to SFT distillation. Code is available at github.com/xhOwenMa/trace-rewriting.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Washington University in St. LouisSt Louis · United States12 works · 1,028 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 worksHardware and Architecture · 81 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Xinhang Ma, William Yeoh, Ning Zhang and Yevgeniy Vorobeychik. 2026. Protecting Language Models Against Unauthorized Distillation through Trace Rewriting. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.519

@inproceedings{ma2026protecting,
  title = {{Protecting Language Models Against Unauthorized Distillation through Trace Rewriting}},
  author = {Xinhang Ma, William Yeoh, Ning Zhang and Yevgeniy Vorobeychik},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.519},
  url = {https://doi.org/10.18653/v1/2026.acl-long.519}
}

References

56 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Xinghao Chen, Zhijing Sun, Guo Wenjin, Miaoran Zhang, Yanjun Chen, Yirong Sun, Hui Su, Yijie Pan, Dietrich Klakow, Wenjie Li, and 1 others. 2025. Unveiling the key factors for distilling chain-of-thought reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, pages 15094–1… find
  2. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
  3. Amirhossein Dabiriaghdam and Lele Wang. 2025. Simmark: A robust sentence-level similarity-based watermarking algorithm for large language models. arXiv preprint arXiv:2502.02787. find
  4. Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 31–36. find
  5. Kaituo Feng, Changsheng Li, Xiaolu Zhang, Jun Zhou, Ye Yuan, and Guoren Wang. 2024. Keypoint-based progressive chain-of-thought distillation for llms. arXiv preprint arXiv:2405.16064. — [arXiv:2405.16064](https://arxiv.org/abs/2405.16064) find
  6. Micah Goldblum, Dimitris Tsipras, Chulin Xie, Xinyun Chen, Avi Schwarzschild, Dawn Song, Aleksander M adry, Bo Li, and Tom Goldstein. 2022. Dataset security for machine learning: Data poisoning, backdoor attacks, and defenses. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2):15… find
  7. Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. 2021. Knowledge distillation: A survey. International journal of computer vision, 129(6):1789–1819. find
  8. Chenxi Gu, Chengsong Huang, Xiaoqing Zheng, Kai-Wei Chang, and Cho-Jui Hsieh. 2022. Watermarking pre-trained language models with backdooring. arXiv preprint arXiv:2210.07543. — [arXiv:2210.07543](https://arxiv.org/abs/2210.07543) find
  9. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  10. Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. 2023. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532. find
  11. Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, and Chenguang Wang. 2022a. Protecting intellectual property of language generation apis with lexical watermark. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10758– 10766. find
  12. Prithiviraj Damodaran. 2021. Parrot: Paraphrase generation for nlu. find
  13. Xuanli He, Qiongkai Xu, Yi Zeng, Lingjuan Lyu, Fangzhao Wu, Jiwei Li, and Ruoxi Jia. 2022b. Cater: Intellectual property protection on text generation apis via conditional watermarks. Advances in Neural Information Processing Systems, 35:5431–5445. find
  14. Badhan Chandra Das, M Hadi Amini, and Yanzhao Wu. 2025. Security and privacy challenges of large language models: A survey. ACM Computing Surveys, 57(6):1–39.
  15. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300.
  16. Jiayu Ding, Lei Cui, Li Dong, Nanning Zheng, and Furu Wei. 2025. Information-preserving reformulation of reasoning traces for antidistillation. arXiv preprint arXiv:2510.11545. find
  17. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. NeurIPS.
  18. Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018. Hotflip: White-box adversarial examples for text classification. In Proceedings of the 56th Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. 11316 10
  19. Abe Hou, Jingyu Zhang, Tianxing He, Yichen Wang, Yung-Sung Chuang, Hongwei Wang, Lingfeng Shen, Benjamin Van Durme, Daniel Khashabi, and Yulia Tsvetkov. 2024. Semstamp: A semantic watermark with paraphrastic robustness for text generation. In Proceedings of the 2024 Conference of the North American… find
  20. Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. In Findings of the Association for Computati…
  21. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3.
  22. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720.
  23. Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858.
  24. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. A watermark for large language models. In International Conference on Machine Learning, pages 17061–17084. PMLR.
  25. Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. 2021. Gedi: Generative discriminator guided sequence generation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4929–4952.
  26. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  27. Hynek Kydlíček. Math-Verify: Math Verification Library. find
  28. Peixuan Li, Pengzhou Cheng, Fangqi Li, Wei Du, Haodong Zhao, and Gongshen Liu. 2023a. Plmmark: a secure and robust black-box watermarking framework for pre-trained language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 14991–14999. find
  29. Pingzhi Li, Zhen Tan, Huaizhi Qu, Huan Liu, and Tianlong Chen. 2025. Doge: Defensive output generation for llm protection against knowledge distillation. arXiv preprint arXiv:2505.19504. find
  30. Zekun Li, Baolin Peng, Pengcheng He, Michel Galley, Jianfeng Gao, and Xifeng Yan. 2023b. Guiding large language models via directional stimulus prompting. Advances in Neural Information Processing Systems, 36:62630–62656. find
  31. Xun Liang, Hanyu Wang, Yezhaohui Wang, Shichao Song, Jiawei Yang, Simin Niu, Jie Hu, Dan Liu, Shunyu Yao, Feiyu Xiong, and 1 others. 2024a. Controllable text generation for large language models: A survey. arXiv preprint arXiv:2408.12599.
  32. Yuqing Liang, Jiancheng Xiao, Wensheng Gan, and Philip S Yu. 2024b. Watermarking techniques for large language models: A survey. arXiv preprint arXiv:2409.00089. — [arXiv:2409.00089](https://arxiv.org/abs/2409.00089)
  33. Zi Liang, Qingqing Ye, Xuan Liu, Yanyun Wang, Jianliang Xu, and Haibo Hu. 2025. Virus infection attack on llms: Your poisoning can spread" via" synthetic data. arXiv preprint arXiv:2509.23041. — [arXiv:2509.23041](https://arxiv.org/abs/2509.23041) find
  34. Yixin Liu, Hongsheng Hu, Xun Chen, Xuyun Zhang, and Lichao Sun. 2023. Watermarking text data on large language models for dataset copyright. arXiv preprint arXiv:2305.13257. — [arXiv:2305.13257](https://arxiv.org/abs/2305.13257) find
  35. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, and 1 others. 2023. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534–46594. find
  36. Andrea Madotto, Etsuko Ishii, Zhaojiang Lin, Sumanth Dathathri, and Pascale Fung. 2020. Plug-and-play conversational models. arXiv preprint arXiv:2010.04344. find
  37. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. In NeurIPS EMC 2 Workshop. find
  38. Yash Savani, Asher Trockman, Zhili Feng, Avi Schwarzschild, Alexander Robey, Marc Finzi, and J Zico Kolter. 2025. Antidistillation sampling. arXiv preprint arXiv:2504.13146. find
  39. Zhiyi Tian, Lei Cui, Jie Liang, and Shui Yu. 2022. A comprehensive survey on poisoning attacks and countermeasures in machine learning. ACM Computing Surveys, 55(8):1–35. find
  40. Brandon Tran, Jerry Li, and Aleksander Madry. 2018. Spectral signatures in backdoor attacks. Advances in neural information processing systems, 31. find
  41. Joshua Vendrow, Edward Vendrow, Sara Beery, and Aleksander Madry. 2025. Do large language model benchmarks test reliability? arXiv preprint arXiv:2502.03461. 11317 11 find
  42. Yevgeniy Vorobeychik and Murat Kantarcioglu. 2018. Adversarial machine learning. Morgan & Claypool Publishers. find
  43. Alexander Wan, Eric Wallace, Sheng Shen, and Dan Klein. 2023. Poisoning language models during instruction tuning. In International Conference on Machine Learning, pages 35413–35425. PMLR. find
  44. Wenbo Wan, Jun Wang, Yunming Zhang, Jing Li, Hui Yu, and Jiande Sun. 2022. A comprehensive survey on robust image watermarking. Neurocomputing, 488:226–247. find
  45. Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo, Jiayou Zhang, Nebojsa Jojic, Eric P Xing, and Zhiting Hu. 2023. Promptagent: Strategic planning with language models enables expert-level prompt optimization. arXiv preprint arXiv:2310.16427. find
  46. Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. 2023a. Provable robust watermarking for ai-generated text. arXiv preprint arXiv:2306.17439. find
  47. Xuandong Zhao, Lei Li, and Yu-Xiang Wang. 2022. Distillation-resistant watermarking for model protection in nlp. arXiv preprint arXiv:2210.03312. find
  48. Xuandong Zhao, Yu-Xiang Wang, and Lei Li. 2023b. Protecting language generation models via invisible watermarking. In International Conference on Machine Learning, pages 42187–42199. PMLR. find
  49. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022. Large language models are human-level prompt engineers. In The eleventh international conference on learning representations.
  50. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, and 1 others. 2024. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266–95290…
  51. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  52. Jiashu Xu, Fei Wang, Mingyu Ma, Pang Wei Koh, Chaowei Xiao, and Muhao Chen. 2024a. Instructional fingerprinting of large language models. In North American Chapter of the Association for Computational Linguistics, pages 3277–3306. find
  53. Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. 2024b. A survey on knowledge distillation of large language models. arXiv preprint arXiv:2402.13116. — [arXiv:2402.13116](https://arxiv.org/abs/2402.13116) find
  54. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2023. Large language models as optimizers. In The Twelfth International Conference on Learning Representations.
  55. Kevin Yang and Dan Klein. 2021. Fudge: Controlled text generation with future discriminators. arXiv preprint arXiv:2104.05218.
  56. Xin Yi, Yue Li, Shunfan Zheng, Linlin Wang, Xiaoling Wang, and Liang He. 2025. Unified attacks to large language model watermarks: spoofing and scrubbing in unauthorized knowledge distillation. Knowledge-Based Systems, page 114295. 11318 12 find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ma-2026-protecting-language-models-against-unauthorized/