Elamur Research⌕
☰
home / papers / 2025 / DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillation

DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillation

Paper · 2025 · Pingzhi Li, Zhen Tan, Mohan Zhang et al.

Summary

Proprietary LLMs can be cheaply copied when competitors fine-tune smaller models on text collected from public APIs. Watermarks and fingerprints only reveal such copying afterwards. Earlier active defenses assume the attacker matches the teacher's logits, which does not hold when only generated text is available. DOGe (Defensive Output Generation) is a preventive defense against this output-only form of knowledge distillation. The authors frame the problem as balancing two goals: the teacher must stay useful while students trained on its outputs should learn poorly. To do this, they adversarially fine-tune only the LM head (the final linear layer). The objective pushes the teacher's outputs away from what proxy students would learn, and it uses reasoning-aware masking. The defended teacher keeps correct final answers, but its reasoning traces become harder to imitate, for example through hedging, redundant re-checks, or occasionally garbled steps. Teacher accuracy stays the same or rises slightly, while distilled students lose substantial accuracy. Because the method needs no full retraining and adds no inference cost, it suits API providers and also developers releasing open-weight models.

Key points

  • Main setup: Qwen3-8B serves as the teacher and Llama-3.2-1B as the student, evaluated on GSM8K, MATH, ARC and CSQA. Teacher accuracy changes by +0.4% to +1.4%, while the misled student loses about 9% to 21%.
  • Larger students suffer more. With Llama-3.1-8B as the student, accuracy drops by 20–50%, compared with roughly 8–18% for the 1B student.
  • The default adversarial coefficient is λ = 3×10⁻⁵, which converges smoothly within 100 training steps. Training already becomes unstable at λ = 10⁻⁴, and at 10⁻³ the loss landscape becomes much more rugged.
  • Case studies use a DeepSeek-R1-7B teacher trained on GSM8K. Typical defended outputs are coherent but filled with low-confidence wording and repeated verification. Rare failure cases show language mixing, yet still reach the right answer.
  • Models are released at huggingface.co/unites-lab/doge-anti-distillation, and code at github.com/unites-lab/doge.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of North Carolina at Chapel HillChapel Hill · United States77 works · 23,365 citations🎓Arizona State UniversityTempe · United States58 works · 5,334 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksHardware and Architecture · 183 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 4,145 worksEfficient models & compression · 328 works

Cite

Cite as Pingzhi Li, Zhen Tan, Mohan Zhang, Huaizhi Qu, Huan Liu and Tianlong Chen. 2025. DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillation. arXiv:2505.19504.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2505.19504 · arXiv 2505.19504

@misc{li2025doge,
  title = {{DOGe: Defensive Output Generation for LLM Protection Against Knowledge Distillation}},
  author = {Pingzhi Li, Zhen Tan, Mohan Zhang, Huaizhi Qu, Huan Liu and Tianlong Chen},
  year = {2025},
  doi = {10.48550/arxiv.2505.19504},
  eprint = {2505.19504},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2505.19504}
}

References

71 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ba, J. and Caruana, R. Do deep nets really need to be deep? Advances in neural information processing systems, 27, 2014. 3 find
  2. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. 1
  3. Chen, P., Liu, S., Zhao, H., and Jia, J. Distilling knowledge via knowledge review. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5008–5017, 2021. 3 find
  4. Chen, Y., Guan, R., Gong, X., Dong, J., and Xue, M. D-dae: Defense-penetrating model extraction attacks. In 2023 IEEE Symposium on Security and Privacy (SP), pp. 382–399. find
  5. Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., Stoica, I., and Xing, E. P. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL https://lmsys.org/blog/2023-03-30-vicuna/. 3 ↗
  6. Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018. URL https://arxiv.org/abs/1803.05457. 7 ↗
  7. Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems, 2021. URL https://arxiv.org/abs/2110.14168. 6, 7 ↗
  8. Cui, W., Li, X., Huang, J., Wang, W., Wang, S., and Chen, J. Substitute model generation for black-box adversarial attack based on knowledge distillation. In 2020 IEEE International Conference on Image Processing (ICIP), pp. 648–652. find
  9. Cui, X., Zhu, M., Qin, Y., Xie, L., Zhou, W., and Li, H. Multi-level optimal transport for universal cross-tokenizer knowledge distillation on language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 23724–23732, 2025. 6 find
  10. Ge, Y., Wang, Q., Zheng, B., Zhuang, X., Li, Q., Shen, C., and Wang, C. Anti-distillation backdoor attacks: Backdoors can really survive in knowledge distillation. In Proceedings of the 29th ACM International Conference on Multimedia, pp. 826–834, 2021. 3 find
  11. Gong, X., Wang, Q., Chen, Y., Yang, W., and Jiang, X. Model extraction attacks and defenses on cloud-based machine learning models. IEEE Communications Magazine, 58(12):83–89, 2021. 3 find
  12. Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014. 2
  13. Gou, J., Yu, B., Maybank, S. J., and Tao, D. Knowledge distillation: A survey. International Journal of Computer Vision, 129(6):1789–1819, 2021. 3 find
  14. Guan, J., Liang, J., and He, R. Are you stealing my model? sample correlation for fingerprinting deep neural networks. Advances in Neural Information Processing Systems, 35:36571–36584, 2022. 3 find
  15. He, J., Zhang, J., Chen, Z., Chen, S., Zhang, M., and Liu, Y. Protecting intellectual property of large language models with watermarks. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 10711–10721, 2022. 1 find
  16. Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. Measuring mathematical problem solving with the math dataset, 2021. URL https://arxiv.org/ abs/2103.03874. 7 ↗
  17. Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. 1, 2, 3
  18. Hong, I. and Choi, C. Knowledge distillation vulnerability of deit through cnn adversarial attack. Neural Computing and Applications, pp. 1–11, 2023. 3 find
  19. Hosny, K. M., Magdi, A., ElKomy, O., and Hamza, H. M. Digital image watermarking using deep learning: A survey. Computer Science Review, 53:100662, 2024. 3 find
  20. Hsu, Y.-S., Feldhus, N., and Hakimov, S. Free-text rationale generation under readability level control. ArXiv, abs/2407.01384, 2024. URL https://api.semanticscholar.org/CorpusId:270870139. 3 ↗
  21. Huang, J., Shao, H., and Chang, K. C.-C. Are large pre-trained language models leaking your personal information? In Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 2038–2047, 2022. 1
  22. Huang, L., Joseph, A. D., Nelson, B., Rubinstein, B. I., and Tygar, J. D. Adversarial machine learning. In Proceedings of the 4th ACM workshop on Security and artificial intelligence, pp. 43–58, 2011. 3 find
  23. Huang, Z. and Wang, N. Like what you like: Knowledge distill via neuron selectivity transfer. arXiv preprint arXiv:1707.01219, 2017. 3 — [arXiv:1707.01219](https://arxiv.org/abs/1707.01219) find
  24. Jiang, W., Li, H., Xu, G., Zhang, T., and Lu, R. A comprehensive defense framework against model extraction attacks. IEEE Transactions on Dependable and Secure Computing, 21(2):685–700, 2023. 3 find
  25. Jiang, Y., Gao, Y., Zhou, C., Hu, H., Fu, A., and Susilo, W. Intellectual property protection for deep learning model and dataset intelligence. arXiv preprint arXiv:2411.05051, 2024. 3 — [arXiv:2411.05051](https://arxiv.org/abs/2411.05051) find
  26. Kim, H. J., Kim, Y., Park, C., Kim, J., Park, C., Yoo, K. M., goo Lee, S., and Kim, T. Aligning language models to explicitly handle ambiguity. In Conference on Empirical Methods in Natural Language Processing, 2024. URL https://api.semanticscholar.org/CorpusId:269214521. 3 ↗
  27. Kim, J., Park, S., and Kwak, N. Paraphrasing complex network: Network compression via factor transfer. Advances in neural information processing systems, 31, 2018. 3 find
  28. Kim, Y. and Rush, A. M. Sequence-level knowledge distillation. In Proceedings of the 2016 conference on empirical methods in natural language processing, pp. 1317–1327, 2016. 3
  29. Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., and Goldstein, T. A watermark for large language models. In International Conference on Machine Learning, pp. 17061–17089. PMLR, 2023. 1
  30. Kumar, R. S. S., Nyström, M., Lambert, J., Marshall, A., Goertzel, M., Comissoneru, A., Swann, M., and Xia, S. Adversarial machine learning-industry perspectives. In 2020 IEEE security and privacy workshops (SPW), pp. 69–75. find
  31. Kurakin, A., Goodfellow, I., and Bengio, S. Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236, 2016. 3 — [arXiv:1611.01236](https://arxiv.org/abs/1611.01236) find
  32. Lambert, N., Morrison, J., Pyatkin, V., Huang, S., Ivison, H., Brahman, F., Miranda, L. J. V., Liu, A., Dziri, N., Lyu, S., Gu, Y., Malik, S., Graf, V., Hwang, J. D., Yang, J., Bras, R. L., Tafjord, O., Wilhelm, C., Soldaini, L., Smith, N. A., Wang, Y., Dasigi, P., and Hajishirzi, H. Tülu 3: Pushing… — Tulu 3: Pushing Frontiers in Open Language Model Post-Training
  33. Li, B., Wang, Y., Meng, T., Chang, K.-W., and Peng, N. Control large language models via divide and conquer. In Conference on Empirical Methods in Natural Language Processing, 2024a. URL https://api.semanticscholar.org/CorpusId:273185893. 3 ↗
  34. Li, D., Jiang, B., Huang, L., Beigi, A., Zhao, C., Tan, Z., Bhattacharjee, A., Jiang, Y., Chen, C., Wu, T., et al. From generation to judgment: Opportunities and challenges of llm-as-a-judge. arXiv preprint arXiv:2411.16594, 2024b. 10 find
  35. Li, D., Jiang, B., Huang, L., Beigi, A., Zhao, C., Tan, Z., Bhattacharjee, A., Jiang, Y., Chen, C., Wu, T., Shu, K., Cheng, L., and Liu, H. From generation to judgment: Opportunities and challenges of llm-as-a-judge, 2025a. URL https://arxiv.org/abs/2411.16594. 19 ↗
  36. Li, D., Sun, R., Huang, Y., Zhong, M., Jiang, B., Han, J., Zhang, X., Wang, W., and Liu, H. Preference leakage: A contamination problem in llm-as-a-judge, 2025b. URL https://arxiv.org/abs/2502. 01534. 19 ↗
  37. Li, G., Zhu, P., Li, J., Yang, Z., Cao, N., and Chen, Z. Security matters: A survey on adversarial machine learning. arXiv preprint arXiv:1810.07339, 2018. 3 — [arXiv:1810.07339](https://arxiv.org/abs/1810.07339) find
  38. Liang, J., Li, G., and Yu, Y. Universal and context-independent triggers for precise control of llm outputs. ArXiv, abs/2411.14738, 2024a. URL https://api.semanticscholar.org/CorpusId:274192480. 3 — [arXiv:2411.14738](https://arxiv.org/abs/2411.14738) ↗
  39. Liang, J., Pang, R., Li, C., and Wang, T. Model extraction attacks revisited. In Proceedings of the 19th ACM Asia Conference on Computer and Communications Security, pp. 1231–1245, 2024b. 3 find
  40. Liang, Y., Xiao, J., Gan, W., and Yu, P. S. Watermarking techniques for large language models: A survey. arXiv preprint arXiv:2409.00089, 2024c. 1, 3 — [arXiv:2409.00089](https://arxiv.org/abs/2409.00089)
  41. Liu, P., Wu, L., Wang, L., Guo, S., and Liu, Y. Step-by-step: Controlling arbitrary style in text with large language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pp. 15285–15295, 2024. 3 find
  42. Ma, H., Chen, T., Hu, T.-K., You, C., Xie, X., and Wang, Z. Undistillable: Making a nasty teacher that cannot teach students. arXiv preprint arXiv:2105.07381, 2021. 1 — [arXiv:2105.07381](https://arxiv.org/abs/2105.07381) find
  43. Minixhofer, B., Ponti, E. M., and Vulić, I. Cross-tokenizer distillation via approximate likelihood matching. arXiv preprint arXiv:2503.20083, 2025. 6 find
  44. Mirzadeh, S. I., Farajtabar, M., Li, A., Levine, N., Matsukawa, A., and Ghasemzadeh, H. Improved knowledge distillation via teacher assistant. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 5191–5198, 2020. 3
  45. Nguyen, D., Chen, J., and Zhou, T. Multi-objective linguistic control of large language models. In Annual Meeting of the Association for Computational Linguistics, 2024. URL https://api.semanticscholar.org/CorpusId:270702436. 3 ↗
  46. Peng, X. and Geng, X. Self-controller: Controlling llms with multi-round step-by-step self-awareness. ArXiv, abs/2410.00359, 2024. URL https://api.semanticscholar.org/CorpusId:273022522. 3 — [arXiv:2410.00359](https://arxiv.org/abs/2410.00359) ↗
  47. Peng, Z., Li, S., Chen, G., Zhang, C., Zhu, H., and Xue, M. Fingerprinting deep neural networks globally via universal adversarial perturbations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13430–13439, 2022. 3 find
  48. Romero, A., Ballas, N., Kahou, S. E., Chassang, A., Gatta, C., and Bengio, Y. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550, 2014. 3 find
  49. Šarčević, T., Karlowicz, A., Mayer, R., Baeza-Yates, R., and Rauber, A. U can’t gen this? a survey of intellectual property protection methods for data in generative ai. arXiv preprint arXiv:2406.15386, 2024. 3 — [arXiv:2406.15386](https://arxiv.org/abs/2406.15386) find
  50. Savani, Y., Trockman, A., Feng, Z., Schwarzschild, A., Robey, A., Finzi, M., and Kolter, J. Z. Antidistillation sampling. arXiv preprint arXiv:2504.13146, 2025a. 2, 6
  51. Savani, Y., Trockman, A., Feng, Z., Schwarzschild, A., Robey, A., Finzi, M., and Kolter, J. Z. Antidistillation sampling, 2025b. URL https://arxiv.org/abs/2504.13146. 3 ↗
  52. Sun, Y., Liu, T., Hu, P., Liao, Q., Fu, S., Yu, N., Guo, D., Liu, Y., and Liu, L. Deep intellectual property protection: A survey. arXiv preprint arXiv:2304.14613, 2023. 3 — [arXiv:2304.14613](https://arxiv.org/abs/2304.14613) find
  53. Takemura, T., Yanai, N., and Fujiwara, T. Model extraction attacks on recurrent neural networks. Journal of Information Processing, 28:1010–1024, 2020. 3 find
  54. Talmor, A., Herzig, J., Lourie, N., and Berant, J. Commonsenseqa: A question answering challenge targeting commonsense knowledge, 2019. URL https://arxiv.org/abs/1811.00937. 7 ↗
  55. Tang, M., Dai, A., DiValentin, L., Ding, A., Hass, A., Gong, N. Z., Chen, Y., et al. {ModelGuard}:{Information-Theoretic} defense against model extraction attacks. In 33rd USENIX Security Symposium (USENIX Security 24), pp. 5305–5322, 2024. 3 find
  56. Tao, Z., Xi, D., Li, Z., Tang, L., and Xu, W. Cat-llm: prompting large language models with text style definition for chinese article-style transfer. arXiv preprint arXiv:2401.05707, 2024. 3 find
  57. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_ alpaca, 2023. 3 ↗
  58. Team, G. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context, 2024. URL https://arxiv.org/abs/2403.05530. 19 ↗
  59. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023. 1
  60. Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., and Ristenpart, T. Stealing machine learning models via prediction apis. In 25th USENIX Security Symposium (USENIX Security 16), pp. 601–618, 2016. 1 find
  61. Vorobeychik, Y. and Kantarcioglu, M. Adversarial machine learning. Morgan & Claypool Publishers, 2018. 3 find
  62. Wan, W., Wang, J., Zhang, Y., Li, J., Yu, H., and Sun, J. A comprehensive survey on robust image watermarking. Neurocomputing, 488:226–247, 2022. 3 find
  63. West, P., Bhagavatula, C., Hessel, J., Hwang, J. D., Jiang, L., Bras, R. L., Lu, X., Welleck, S., and Choi, Y. Symbolic knowledge distillation: from general language models to commonsense models. arXiv preprint arXiv:2110.07178, 2021. 3
  64. Xu, J., Wang, F., Ma, M. D., Koh, P. W., Xiao, C., and Chen, M. Instructional fingerprinting of large language models. arXiv preprint arXiv:2401.12255, 2024a. 1 find
  65. Xu, X., Li, M., Tao, C., Shen, T., Cheng, R., Li, J., Xu, C., Tao, D., and Zhou, T. A survey on knowledge distillation of large language models. arXiv preprint arXiv:2402.13116, 2024b. 3 — [arXiv:2402.13116](https://arxiv.org/abs/2402.13116) find
  66. Yu, N., Skripniuk, V., Abdelnabi, S., and Fritz, M. Artificial fingerprinting for generative models: Rooting deepfake attribution in training data. In Proceedings of the IEEE/CVF International conference on computer vision, pp. 14448–14457, 2021. 3 find
  67. Zelikman, E., Wu, Y., Mu, J., and Goodman, N. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35:15476–15488, 2022. 3
  68. Zhang, X., Fang, C., and Shi, J. Thief, beware of what get you there: Towards understanding model extraction attack. arXiv preprint arXiv:2104.05921, 2021. 3 — [arXiv:2104.05921](https://arxiv.org/abs/2104.05921) find
  69. Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., and Stoica, I. Judging llm-as-a-judge with mt-bench and chatbot arena, 2023. URL https://arxiv.org/abs/2306.05685. 19 ↗
  70. Zhong, X., Das, A., Alrasheedi, F., and Tanvir, A. A brief, in-depth survey of deep learning-based image watermarking. Applied Sciences, 13(21):11852, 2023. 3 find
  71. Zhou, G., Fan, Y., Cui, R., Bian, W., Zhu, X., and Gai, K. Rocket launching: A universal and efficient framework for training well-performing light net. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018. 3 find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2025-doge-defensive-output-generation-llm/