Elamur Research⌕
☰
home / papers / 2026 / The Mark Fades: Adaptive Evolutionary Paraphrase-based Attack against LLM Watermarks

The Mark Fades: Adaptive Evolutionary Paraphrase-based Attack against LLM Watermarks

Summary

The paper looks at how fragile statistical watermarks in LLM-generated text are when an attacker can only rewrite the text. Existing paraphrase attacks force a trade-off: light rewriting leaves the watermark detectable, while heavy rewriting damages meaning and fluency.

The authors first run a graded stress test. It starts with random word deletions, insertions and substitutions, then moves to LLM-driven polishing, expansion, summarization and back-translation at adjustable strengths. They then propose TSAPA, a training-free, black-box attack that treats watermark removal as constrained multi-objective optimization. A genetic algorithm evolves paraphrase candidates chunk by chunk and searches the Pareto front over three goals: naturalness, semantic similarity and lexical diversity. Mutations are guided by token-level Pseudo-Log-Likelihood (PLL) scores, which single out tokens likely to carry the watermark.

On Qwen3 models (1.7B/8B/32B) and several KGW- and Christ-family schemes, TSAPA removes watermarks in over 90% of cases while preserving meaning better than baselines such as SIRA. The results show that current watermarks can be stripped without access to the key or the detector.

Key points

  • Threat model: The attacker holds only a corpus of watermarked texts and a paraphrasing LLM (Llama3-70B or gpt-4o-mini in the experiments). It has no knowledge of the scheme, no secret key and no queries to the detector.
  • Main result: On article continuation with Qwen3-32B under the EXP scheme, TSAPA reaches near-100% ASR with BERTScore around 0.73–0.77, beating SIRA and simple edit or rewrite attacks.
  • Model size: Across eight watermarking schemes and three Qwen3 sizes, ASR stays above 97% and is often 100%. Larger source models therefore give no added robustness.
  • Ablation: Optimizing only semantic similarity yields 2.97% ASR. Optimizing only diversity yields 100% ASR but PPL 537.26. The full objective reaches 93.07% ASR with BERTS 0.7629 and PPL 7.03.
  • Generations and limitations: Going from 1 to 9 generations cuts TPR@1%FPR from about 0.47 to 0.10. The many LLM calls make the method suited to offline use. Optimizing chunks in isolation may hurt global coherence.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.459

@inproceedings{zhao2026mark,
  title = {{The Mark Fades: Adaptive Evolutionary Paraphrase-based Attack against LLM Watermarks}},
  author = {Zhao, Yusheng and Zhao, Jian and Zhang, Tianle and Wei, Feng and Li, Xuelong},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.459},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.459}
}

References

29 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. 2023. Unbiased watermark for large language models. arXiv preprint arXiv:2310.10669. ↗
  2. Scott Aaronson and H. Kirchner. 2022. Watermarking GPT outputs. https://www.scottaaronson.com/ talks/watermark.ppt. PowerPoint presentation. ↗
  3. Ruibo Chen, Yihan Wu, Junfeng Guo, and Heng Huang. 2024. De-mark: Watermark removal in large language models. arXiv preprint arXiv:2410.13808. ↗
  4. Yixin Cheng, Hongcheng Guo, Yangming Li, and Leonid Sigal. 2025. Revealing weaknesses in text watermarking through self-information rewrite attacks. In Forty-second International Conference on Machine Learning (ICML’25). ↗
  5. Miranda Christ, Sam Gunn, and Or Zamir. 2024. Undetectable watermarks for language models. In The Thirty Seventh Annual Conference on Learning Theory, pages 1125–1139. PMLR. ↗
  6. Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, and 1 others. 2024. Scalable watermarking for identifying large language model outputs. Nature, 634(8035):818–823. find
  7. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023a. A watermark for large language models. In International Conference on Machine Learning (ICML’23), pages 17061–17084. ↗ our card
  8. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. 2023b. On the reliability of watermarks for large language models. arXiv preprint arXiv:2306.04634. ↗
  9. Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2023. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. Advances in Neural Information Processing Systems (NeurIPS’23), 36:27469–27500. find our card
  10. Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. 2023. Robust distortion-free watermarks for language models. arXiv preprint arXiv:2307.15593. 9436 9 ↗
  11. Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. 2023. Who wrote this code? watermarking for code generation. arXiv preprint arXiv:2305.15060. ↗
  12. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81. find
  13. Aiwei Liu, Leyi Pan, Xuming Hu, Shu’ang Li, Lijie Wen, Irwin King, and Philip S Yu. 2023a. An unforgeable publicly verifiable watermark for large language models. arXiv preprint arXiv:2307.16230. ↗
  14. Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijie Wen. 2023b. A semantic invariant robust watermark for large language models. arXiv preprint arXiv:2310.06356. ↗
  15. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. ↗
  16. Yijian Lu, Aiwei Liu, Dianzhi Yu, Jingjing Li, and Irwin King. 2024. An entropy-based text watermarking detection method. arXiv preprint arXiv:2403.13485. ↗
  17. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In International conference on machine learning, pages 24950–24962. PMLR. find our card
  18. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67. ↗
  19. Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. 2023. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303.11156. ↗ our card
  20. Julian Salazar, Davis Liang, Toan Q Nguyen, and Katrin Kirchhoff. 2019. Masked language model scoring. arXiv preprint arXiv:1910.14659. ↗ our card
  21. Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2022. Asqa: Factoid questions meet long-form answers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8273–8288. find our card
  22. Yihan Wu, Zhengmian Hu, Junfeng Guo, Hongyang Zhang, and Heng Huang. 2023. A resilient and accessible distribution-preserving watermark for large language models. arXiv preprint arXiv:2310.07710. ↗
  23. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388. ↗ our card
  24. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. ↗
  25. OpenAI. 2024. Gpt-4o mini: advancing cost-efficient Zhaoxi Zhang, Xiaomei Zhang, Yanjun Zhang, intelligence. https://openai.com/index/ He Zhang, Shirui Pan, Bo Liu, Asif Qumer Gill, gpt-4o-mini-advancing-cost-efficient-intelligence/. and Leo Yu Zhang. 2025. Character-level pertur-Accessed: 2024-07-18. bations disrupt llm watermarks. arXiv preprint arXiv:2509.09112. ↗
  26. Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Xuandong Zhao, Prabhanjan Vijendra Ananth, Lei Li, Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xum-and Yu-Xiang Wang. 2024. Provable robust watering Hu, Lijie Wen, and 1 others. 2024. Markllm: marking for AI-generated text. In The Twelfth Inter-An open-source toolkit for llm watermarking. arXiv national Conference on Learning Representations. preprint arXiv:2405.10051. ↗
  27. Qi Pang, Shengyuan Hu, Wenting Zheng, and Virginia Smith. 2024. Attacking llm watermarks by exploiting their strengths. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models. find
  28. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. ↗ our card
  29. Julien Piet, Chawin Sitawarin, Vivian Fang, Norman Mu, and David Wagner. 2025. Markmywords: Analyzing and evaluating language model watermarks. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML’25), pages 68–91. 9437 10 find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/zhao-2026-mark-fades-adaptive-evolutionary-paraphrase/