Elamur Research
☰
home / papers / 2023 / A Watermark for Large Language Models

A Watermark for Large Language Models

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2301.10226

Summary

The paper tackles a practical question: how can text produced by a large language model be reliably distinguished from human writing? This matters because of misuse such as disinformation, social-media bots and academic cheating, and because synthetic content pollutes corpora used to train future models. Rather than building a post-hoc classifier, the authors hide a statistical signal in the text while it is generated. At every step, a hash of the preceding token seeds a random generator that splits the vocabulary into a "green" and a "red" list. In the soft variant, a constant δ is added to the logits of green tokens. This nudge changes the choice of tokens when many plausible options exist and barely affects nearly forced ones. To detect the watermark, one only needs the hash function: count the green tokens and run a one-proportion z-test, which yields interpretable p-values without loading the model or calling its API. The method is demonstrated on the OPT family. It requires no retraining, works on short spans of text, and makes false accusations against human authors statistically unlikely. That last property is important for groups like non-native speakers, whom existing detectors often wrongly flag.

Key points

  • Example result: On OPT-6.7B with γ = 0.25 and δ = 2, a sample contained 28 green tokens where about 9 would be expected from a human. That corresponds to a chance probability of roughly 6×10⁻¹⁴. The watermark can be detected from spans as short as about 25 tokens.
  • Hard red-list baseline: Using a z > 4 threshold gives a false-positive rate of about 3×10⁻⁵. Any fully watermarked sequence of 16 or more tokens is caught.
  • Robustness: In a 1000-token text, an attacker who knows the scheme and adversarially changes 200 tokens still leaves z ≈ 6.3 (p ≈ 10⁻¹⁰). Erasing the mark generally requires editing roughly a quarter of the tokens.
  • Detector independence: The detector depends only on γ and the hashing scheme, not on δ. Generation settings can therefore be tuned per context (e.g., prose vs. code) while keeping a single detector.
  • Code and demo: Available at github.com/jwkirchenbauer/lm-watermarking. The detector can be released publicly or kept private behind an API.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

263 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 253 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2301.10226 · arXiv 2301.10226

@misc{kirchenbauer2023watermark,
  title = {{A Watermark for Large Language Models}},
  author = {Kirchenbauer, John and Geiping, Jonas and Wen, Yuxin and Katz, Jonathan and Miers, Ian and Goldstein, Tom},
  year = {2023},
  doi = {10.48550/arxiv.2301.10226},
  eprint = {2301.10226},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2301.10226}
}

References

10 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aaronson, S. My AI Safety Lecture for UT Effective Altruism, November 2022. URL https://scottaaronson.blog/?p=6823. ↗
  2. Abdelnabi, S. and Fritz, M. Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hiding. In 2021 IEEE Symposium on Security and Privacy (SP), pp. 121–140, May 2021. doi: 10.1109/SP40001.2021.00083. — [doi:10.1109/sp40001.2021.00083](https://doi.org/10.1109/sp40001.2021.00083) find
  3. Adi, Y., Baum, C., Cisse, M., Pinkas, B., and Keshet, J. Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In Proceedings of the 27th USENIX Conference on Security Symposium, SEC’18, pp. 1615–1631, USA, August 2018. USENIX Association. ISBN 978-1-931971-46-1. find
  4. Atallah, M. J., Raskin, V., Crogan, M., Hempelmann, C., Kerschbaum, F., Mohamed, D., and Naik, S. Natural Language Watermarking: Design, Analysis, and a Proof-of-Concept Implementation. In Moskowitz, I. S. (ed.), Information Hiding, Lecture Notes in Computer Science, pp. 185–200, Berlin, Heidelberg,… — [doi:10.1007/3-540-45496-9](https://doi.org/10.1007/3-540-45496-9) find
  5. Atallah, M. J., Raskin, V., Hempelmann, C. F., Karahan, M., Sion, R., Topkara, U., and Triezenberg, K. E. Natural Language Watermarking and Tamperproofing. In Petitcolas, F. A. P. (ed.), Information Hiding, Lecture Notes in Computer Science, pp. 196–212, Berlin, Heidelberg, 2003. Springer. ISBN 978-… — [doi:10.1007/3-540-36415-3](https://doi.org/10.1007/3-540-36415-3) find
  6. Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., Chen, C., Olsson, C., Olah, C., Hernandez, D., Drain, D., Ganguli, D., Li, D., Tran-Johnson, E., Perez, E., Kerr, J., Mueller, J., Ladish, J., Landau, J., Ndousse, K., Lukosiute,… find our card
  7. Bender, E. M., Gebru, T., McMillan-Major, A., and Shmitchell, S. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, pp. 610– 623, New York, NY, USA, March 2021. Association for Comp… — [doi:10.1145/3442188.3445922](https://doi.org/10.1145/3442188.3445922) find
  8. Bergman, A. S., Abercrombie, G., Spruit, S., Hovy, D., Dinan, E., Boureau, Y.-L., and Rieser, V. Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 39–52, Edinburgh,… find
  9. Boenisch, F. A Systematic Review on Model Watermarking for Neural Networks. Frontiers in Big Data, 4, 2021. ISSN 2624-909X. doi: 10.3389/fdata.2021.729663. URL https://www.frontiersin.org/articles/10.3389/fdata.2021.729663. — [doi:10.3389/fdata.2021.729663](https://doi.org/10.3389/fdata.2021.729663) ↗
  10. Boucher, N., Shumailov, I., Anderson, R., and Papernot, N. Bad Characters: Imperceptible NLP Attacks. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1987– 2004, May 2022. doi: 10.1109/SP46214.2022.9833641. — [doi:10.1109/sp46214.2022.9833641](https://doi.org/10.1109/sp46214.2022.9833641) find