Elamur Research⌕
☰
home / papers / 2024 / Who Wrote this Code? Watermarking for Code Generation

Who Wrote this Code? Watermarking for Code Generation

Paper · 2024 · Taehyun Lee, Seokhee Hong, Jaewoo Ahn et al.

Summary

Detecting code produced by large language models matters because of licensing disputes, plagiarism and AI-assisted malware. The authors show that existing detectors work poorly on code. This includes logit-based watermarking such as WLLM (Kirchenbauer et al., 2023a) and post-hoc classifiers. The root cause is low entropy: a program usually allows only a few valid tokens at each step. A strong watermark therefore pushes the model toward wrong tokens and breaks the code. A weak watermark leaves too few green-list tokens for a reliable statistical signal.

Their method, SWEET (Selective WatErmarking via Entropy Thresholding), extends WLLM. It biases toward green-list tokens, and counts them at detection, only at positions where next-token entropy exceeds a threshold. Low-entropy positions are skipped during both generation and detection. On code-generation benchmarks such as HumanEval and MBPP, SWEET detects machine-written code better than all baselines, post-hoc detectors included, and harms correctness less than WLLM. It also keeps working without prompts, with a smaller surrogate detector, and under paraphrasing attacks. This makes watermarking of Code LLMs more practical.

Key points

  • Illustrative case (HumanEval/4): WLLM watermarks every token and yields incorrect code with a z-score of 2.45. SWEET with a moderate threshold watermarks about 20% of tokens and produces correct code with a z-score of 4.96.
  • Surrogate detection: code generated by LLaMA2-13B was checked using LLaMA2-7B. Detection dropped only slightly and still beat the baseline, likely because both models share a training corpus.
  • Paraphrasing robustness: the authors tested 273 MBPP solutions with two attacks. One renamed variables to random 2–5 character strings (five seeds); the other used a commercial refactoring service (CodePal). SWEET stayed ahead of the baselines, but its AUROC fell to roughly 0.8 when every variable was renamed, because names account for many high-entropy tokens.
  • Limitations: SWEET is white-box and needs the source model (or a surrogate) at detection time. The entropy threshold requires tuning. Watermark forgery remains a risk.
  • Code: released at github.com/hongcheki/sweet-watermark.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Georgia Institute of TechnologyAtlanta · United States69 works · 22,788 citations🎓Seoul National UniversitySeoul · South Korea20 works · 45,282 citations🏢NaverSeongnam-si · South Korea15 works · 11,528 citations🏢NAVER CloudSeongnam-si · South Korea13 works · 10,602 citations

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksComputer Vision and Pattern Recognition · 1,282 worksInformation Systems · 523 works

🧭 Topics 1Research subjects and languages the work is about.Code & programming languages · 55 works

Cite

Cite as Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin and Gunhee Kim. 2024. Who Wrote this Code? Watermarking for Code Generation. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.acl-long.268

@inproceedings{lee2024who,
  title = {{Who Wrote this Code? Watermarking for Code Generation}},
  author = {Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin and Gunhee Kim},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.268},
  url = {https://doi.org/10.18653/v1/2024.acl-long.268}
}

References

69 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sahar Abdelnabi and Mario Fritz. 2021. Adversarial watermarking transformer: Towards tracing text provenance with data hiding. In 2021 IEEE Symposium on Security and Privacy (SP), pages 121–140. IEEE, IEEE. find
  2. Mikhail J. Atallah, Victor Raskin, Michael Crogan, Christian Hempelmann, Florian Kerschbaum, Dina Mohamed, and Sanket Naik. 2001. Natural language watermarking: Design, analysis, and a proof-of-concept implementation. In Information Hiding, pages 185–200. Springer, Springer Berlin Heidelberg. find
  3. Mikhail J. Atallah, Victor Raskin, Christian F. Hempelmann, Mercan Karahan, Radu Sion, Umut Topkara, and Katrina E. Triezenberg. 2002. Natural language watermarking and tamperproofing. In Information Hiding, pages 196–212. Springer, Springer Berlin Heidelberg. find
  4. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732.
  5. Mohammad Bavarian, Heewoo Jun, Nikolas Tezak, John Schulman, Christine McLeavey, Jerry Tworek, and Mark Chen. 2022. Efficient training of language models to fill in the middle. arXiv preprint arXiv:2207.14255.
  6. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. In USENIX Security Symposium, volume 6.
  7. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
  8. Miranda Christ, Sam Gunn, and Or Zamir. 2023. Undetectable watermarks for language models. arXiv preprint arXiv:2306.09194. — [arXiv:2306.09194](https://arxiv.org/abs/2306.09194) find
  9. Ayan Dey, Sukriti Bhattacharya, and Nabendu Chaki. 2018. Software watermarking: Progress and challenges. INAE Letters, 4(1):65–75. find
  10. Xueying Du, Mingwei Liu, Kaixin Wang, Hanlin Wang, Junwei Liu, Yixuan Chen, Jiayi Feng, Chaofeng Sha, Xin Peng, and Yiling Lou. 2023. Classeval: A manually-crafted benchmark for evaluating llms on class-level code generation. find
  11. Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, and Mike Lewis. 2023. Incoder: A generative model for code infilling and synthesis. In International Conference on Learning Representations. find
  12. Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. GLTR: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111–116, Florence, Italy. Association for Computat…
  13. GeorgiaTechResearchInstitute. 2023. gpteacher-code-instruct. starcoder-Chenchen Gu, Xiang Lisa Li, Percy Liang, and Tatsunori Hashimoto. 2023. On the learnability of watermarks for language models. arXiv preprint arXiv:2312.04469. — [arXiv:2312.04469](https://arxiv.org/abs/2312.04469) find
  14. Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. 2023. Textbooks are all you need. arXiv preprint arXiv:2306.11644. find
  15. Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. arXiv preprint arXiv:2301.07597.
  16. James Hamilton and Sebastian Danicic. 2011. A survey of static software watermarking. In 2011 World Congress on Internet Security (WorldCIS-2011), pages 100–107. IEEE, IEEE. find
  17. Julian Hazell. 2023. Large language models can be used to effectively scale spear phishing campaigns. arXiv preprint arXiv:2305.06972. — [arXiv:2305.06972](https://arxiv.org/abs/2305.06972) find
  18. Jingxuan He and Martin Vechev. 2023. Large language models for code: Security hardening and adversarial testing. arXiv preprint arXiv:2302.05319. — [arXiv:2302.05319](https://arxiv.org/abs/2302.05319) find
  19. Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, and Chenguang Wang. 2022a. Protecting intellectual property of language generation apis with lexical watermark. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10758– 10766. find
  20. Xuanli He, Qiongkai Xu, Yi Zeng, Lingjuan Lyu, Fangzhao Wu, Jiwei Li, and Ruoxi Jia. 2022b. Cater: Intellectual property protection on text generation apis via conditional watermarks. Advances in Neural Information Processing Systems, 35:5431–5445. find
  21. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In International Conference on Learning Representations. find
  22. Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. 2023. Unbiased watermark for large language models. arXiv preprint arXiv:2310.10669. — [arXiv:2310.10669](https://arxiv.org/abs/2310.10669) find
  23. Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. 2019. Codesearchnet challenge: Evaluating the state of semantic code search. arXiv preprint arXiv:1909.09436.
  24. Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. 2020. Automatic detection of generated text is easiest when humans are fooled. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1808–1822, Online. Association for Computational…
  25. Zunera Jalil and Anwar M. Mirza. 2009. A review of digital watermarking techniques for text documents. In 2009 International Conference on Information and Multimedia Technology, pages 230–234. IEEE, IEEE. find
  26. Young-Won Kim, Kyung-Ae Moon, and Il-Seok Oh. 2003. A text watermarking algorithm based on word classification and inter-word space statistics. In Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings., pages 775–779. Citeseer, IEEE Comput. Soc. find
  27. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023a. A watermark for large language models. The Fortieth International Conference on Machine Learning.
  28. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. 2023b. On the reliability of watermarks for large language models. arXiv preprint arXiv:2306.04634. find
  29. Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2023. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. arXiv preprint arXiv:2303.13408.
  30. Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. 2023. Robust distortion-free watermarks for language models. arXiv preprint arXiv:2307.15593. find
  31. Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Wen-Tau Yih, Daniel Fried, Sida Wang, and Tao Yu. 2023. DS-1000: A natural and reliable benchmark for data science code generation. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of… find
  32. Jun Li and Quan Liu. 2010. Design of a software watermarking algorithm based on register allocation. In 2010 2nd International Conference on E-business and Information System Security, pages 1–4. IEEE, IEEE. find
  33. Linyang Li, Pengyu Wang, Ke Ren, Tianxiang Sun, and Xipeng Qiu. 2023a. Origin tracing and detecting of llms. arXiv preprint arXiv:2304.14072. — [arXiv:2304.14072](https://arxiv.org/abs/2304.14072) find
  34. Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. 2023b. Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161. find
  35. Zongjie Li, Chaozheng Wang, Shuai Wang, and Cuiyun Gao. 2023c. Protecting intellectual property of large language model-based code generation apis via watermarks. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pages 2336–2350. find
  36. Aiwei Liu, Leyi Pan, Xuming Hu, Shu’ang Li, Lijie Wen, Irwin King, and Philip S. Yu. 2023a. An unforgeable publicly verifiable watermark for large language models. find
  37. Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijie Wen. 2023b. A semantic invariant robust watermark for large language models. arXiv preprint arXiv:2310.06356. — [arXiv:2310.06356](https://arxiv.org/abs/2310.06356) find
  38. Hasan Mesut Meral, Bülent Sankur, A. Sumru Özsoy, Tunga Güngör, and Emre Sevinç. 2009. Natural language watermarking via morphosyntactic alterations. Computer Speech & Language, 23(1):107–125. find
  39. Yisroel Mirsky, Ambra Demontis, Jaidip Kotak, Ram Shankar, Deng Gelei, Liu Yang, Xiangyu Zhang, Maura Pintor, Wenke Lee, Yuval Elovici, and Battista Biggio. 2023. The threat of offensive AI to organizations. Computers & Security, 124:103006. find
  40. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detection using probability curvature. The Fortieth International Conference on Machine Learning.
  41. Sandra Mitrović, Davide Andreoletti, and Omran Ayoub. 2023. Chatgpt or human? detect and explain. explaining decisions of machine learning model for detecting short chatgpt-generated text. find
  42. Niklas Muennighoff, Qian Liu, Armel Randy Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro Von Werra, and Shayne Longpre. 2024. Octopack: Instruction tuning code large language models. In The Twelfth International Conference on Learning Representations. find
  43. Ginger Myles, Christian Collberg, Zachary Heidepriem, and Armand Navabi. 2005. The evaluation of two software watermarking algorithms. Software: Practice and Experience, 35(10):923–938. find
  44. Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2023. Codegen: An open large language model for code with multi-turn program synthesis. In The Eleventh International Conference on Learning Representations. find
  45. Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Lijie Wen, Irwin King, and Philip S. Yu. 2024. A survey of text watermarking in the era of large language models. find
  46. OpenAI. 2023a. Gpt-4 technical report. OpenAI Blog.
  47. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. find
  48. Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. 2022. Asleep at the keyboard? assessing the security of GitHub copilot’s code contributions. In 2022 IEEE Symposium on Security and Privacy (SP), pages 754– 768. IEEE, IEEE. find
  49. Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. Wizardcoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv:2306.08568. find
  50. Haoyu Ma, Chunfu Jia, Shijia Li, Wantong Zheng, and Dinghao Wu. 2019. Xmark: Dynamic software watermarking using collatz conjecture. IEEE Transactions on Information Forensics and Security, 14(11):2859–2874. find
  51. OpenAI. 2023b. New ai classifier for indicating ai-written text. OpenAI Blog. find
  52. Jie Ren, Han Xu, Yiding Liu, Yingqian Cui, Shuaiqiang Wang, Dawei Yin, and Jiliang Tang. 2023. A robust semantics-based watermark for large language model against paraphrasing. arXiv preprint arXiv:2311.08721. find
  53. Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. 2023. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303.11156.
  54. Gustavo Sandoval, Hammond A. Pearce, Teo Nys, Ramesh Karri, Siddharth Garg, and Brendan Dolan-Gavitt. 2023. Lost at c: A user study on the security implications of large language model code assistants. In 32nd USENIX Security Symposium (USENIX Security 23). find
  55. Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, et al. 2019. Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203.
  56. Yuki Takezawa, Ryoma Sato, Han Bao, Kenta Niwa, and Makoto Yamada. 2023. Necessary and sufficient watermark for large language models. arXiv preprint arXiv:2310.00833. — [arXiv:2310.00833](https://arxiv.org/abs/2310.00833) find
  57. Edward Tian and Alexander Cui. 2023. Gptzero: Towards detection of ai-generated text using zero-shot and supervised methods. find
  58. Umut Topkara, Mercan Topkara, and Mikhail J. Atallah. 2006. The hiding virtues of ambiguity. In Proceedings of the 8th workshop on Multimedia and security, pages 164–174. ACM. find
  59. Xianjun Yang, Wei Cheng, Linda Petzold, William Yang Wang, and Haifeng Chen. 2023. Dna-gpt: Divergent n-gram analysis for training-free detection of gpt-generated text. arXiv preprint arXiv:2305.17359. — [arXiv:2305.17359](https://arxiv.org/abs/2305.17359) find
  60. KiYoon Yoo, Wonhyuk Ahn, Jiho Jang, and Nojun Kwak. 2023. Robust natural language watermarking through invariant features. In Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics. find
  61. Xiao Yu, Yuang Qi, Kejiang Chen, Guoqiang Chen, Xi Yang, Pengyuan Zhu, Weiming Zhang, and Nenghai Yu. 2023. Gpt paternity test: Gpt generated text detection with gpt genetic inheritance. arXiv preprint arXiv:2305.12519. — [arXiv:2305.12519](https://arxiv.org/abs/2305.12519) find
  62. Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. 2023. Provable robust watermarking for ai-generated text. arXiv preprint arXiv:2306.17439. find
  63. Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, Yang Li, et al. 2023. Codegeex: A pre-trained model for code generation with multilingual evaluations on humaneval-x. arXiv preprint arXiv:2303.17568.
  64. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. find
  65. Priyan Vaithilingam, Tianyi Zhang, and Elena L Glassman. 2022. Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models. In Chi conference on human factors in computing systems extended abstracts, pages 1–7. find
  66. Lean Wang, Wenkai Yang, Deli Chen, Hao Zhou, Yankai Lin, Fandong Meng, Jie Zhou, and Xu Sun. 2023. Towards codable text watermarking for large language models. arXiv preprint arXiv:2307.15992. — [arXiv:2307.15992](https://arxiv.org/abs/2307.15992) find
  67. Yilong Wang, Daofu Gong, Bin Lu, Fei Xiang, and Fenlin Liu. 2018. Exception handling-based dynamic software watermarking. IEEE Access, 6:8882–8889. find
  68. Kangxi Wu, Liang Pang, Huawei Shen, Xueqi Cheng, and Tat-Seng Chua. 2023. LLMDet: A third party large language models generated text detection tool. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2113–2133, Singapore. Association for Computational Linguistics. find
  69. Xi Yang, Jie Zhang, Kejiang Chen, Weiming Zhang, Zehua Ma, Feng Wang, and Nenghai Yu. 2022. Tracing text provenance via context-aware lexical substitution. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11613– 11621. Association for the Advancement of Artificial I… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lee-2024-who-wrote-this-code-watermarking/