Elamur Research⌕
☰
home / papers / 2026 / You Can Have a Second Chance: Unbiased and Multi-bit Watermarking for Diffusion Language Models with Regret-based Remasking

You Can Have a Second Chance: Unbiased and Multi-bit Watermarking for Diffusion Language Models with Regret-based Remasking

Paper · 2026 · Ke Yang, Dongyang Liang, Jing Yu et al.

Summary

The paper addresses watermarking text produced by Diffusion Language Models (DLMs). Classic LLM watermarks assume tokens are generated left to right, so the preceding token can seed the watermark. DLMs instead unmask positions in arbitrary order. Prior adaptations of watermarking to DLMs, by Wu et al. and by Gloaguen et al. (both 2025), substitute guessed tokens for masked neighbours and then apply a biased Green-Red scheme. This causes three problems: unreliable signals early in denoising, quality loss as bias accumulates, and support only for zero-bit (yes/no) detection.

The authors propose UMR, which combines three components:

  • Stability-aware constraint: a token is watermarked only when at least one adjacent token is already unmasked.
  • Bit-controlled unbiased modulation: message bits are mapped to vocabulary partitions, and token probabilities are reweighted without changing the expected distribution.
  • Regret-based Remasking: unwatermarked tokens are remasked and regenerated later, giving them a second chance.

Across several DLMs and tasks, UMR detects watermarks with about 0.95 accuracy. Its output quality stays close to the unwatermarked model and beats the baselines. This makes multi-bit provenance tracing practical for DLMs, for example embedding user IDs or copyright information.

Key points

  • UMR is presented as the first watermarking scheme for DLMs that is both unbiased and multi-bit. Previous methods were biased and zero-bit.
  • Watermarking uses a bidirectional constraint: the left neighbour acts as the prefix, and the chosen token must also act as a valid prefix for the right neighbour. Masked neighbours are temporarily replaced by the argmax prediction.
  • A precomputed |V|×|V| prefix-vocabulary matrix M_W stores the watermark regions, so per-step hashing is avoided.
  • Regret-based Remasking unmasks one extra token for each regretted token. As a result, it adds no diffusion steps and no latency.
  • Stated limitations:
    • Unbiasedness holds only approximately, because it relies on a large-vocabulary assumption.
    • In the zero-bit setting, detection on short texts is slightly weaker than standard bidirectional schemes.
    • Robustness to attacks is reduced, as a trade-off for preserving quality.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Vision and Pattern Recognition · 3,486 worksHardware and Architecture · 167 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Ke Yang, Dongyang Liang, Jing Yu, Shuguang Yuan and Chi Chen. 2026. You Can Have a Second Chance: Unbiased and Multi-bit Watermarking for Diffusion Language Models with Regret-based Remasking. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1297

@inproceedings{yang2026you,
  title = {{You Can Have a Second Chance: Unbiased and Multi-bit Watermarking for Diffusion Language Models with Regret-based Remasking}},
  author = {Ke Yang, Dongyang Liang, Jing Yu, Shuguang Yuan and Chi Chen},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1297},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1297}
}

References

30 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ishaan Gulrajani and Tatsunori B Hashimoto. 2023. Likelihood-based diffusion language models. Advances in Neural Information Processing Systems, 36:16693–16715. find
  2. Zhiwei He, Binglin Zhou, Hongkun Hao, Aiwei Liu, Xing Wang, Zhaopeng Tu, Zhuosheng Zhang, and Rui Wang. 2024. Can watermarks survive translation? on the cross-lingual consistency of text watermark for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computation… find
  3. Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. 2023. Unbiased watermark for large language models. Preprint, arXiv:2310.10669. — [arXiv:2310.10669](https://arxiv.org/abs/2310.10669) find
  4. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. A watermark for large language models. In International Conference on Machine Learning, pages 17061–17084. PMLR.
  5. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. 2024. On the reliability of watermarks for large language models. Preprint, arXiv:2306.04634.
  6. Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2023. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. Preprint, arXiv:2303.13408.
  7. Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. 2024. Robust distortion-free watermarks for language models. Preprint, arXiv:2307.15593.
  8. Inception Labs. 2025. “mercury”. https://www.inceptionlabs.ai/introducing-mercury. Accessed: 2025-12-29. ↗
  9. Tianyi Li, Mingda Chen, Bowei Guo, and Zhiqiang Shen. 2025. A survey on diffusion language models. Preprint, arXiv:2508.10875. — [arXiv:2508.10875](https://arxiv.org/abs/2508.10875) find
  10. Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. 2022. Diffusion-lm improves controllable text generation. Advances in neural information processing systems, 35:4328– 4343.
  11. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
  12. Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. 2025. Large language diffusion models. Preprint, arXiv:2502.09992.
  13. Radical Numerics. 2025. Training diffusion language models at scale using autoregressive models. find
  14. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
  15. Wenjie Qu, Wengrui Zheng, Tianyang Tao, Dong Yin, Yanze Jiang, Zhihua Tian, Wei Zou, Jinyuan Jia, and Jiaheng Zhang. 2025. Provably robust multi-bit watermarking for ai-generated text. In 34th USENIX Security Symposium (USENIX Security 25), pages 201–220. find
  16. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  17. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2023. Exploring the limits of transfer learning with a unified text-to-text transformer. Preprint, arXiv:1910.10683.
  18. Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, and Volodymyr Kuleshov. 2024. Simple and effective masked diffusion language models. Preprint, arXiv:2406.07524.
  19. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073– 1083, Vancouver, Canada. Association for Compu… find
  20. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca. ↗
  21. Linyu Wu, Linhao Zhong, Wenjie Qu, Yuexin Li, Yue Liu, Shengfang Zhai, Chunhua Shen, and Jiaheng Zhang. 2025. Dmark: Order-agnostic watermarking for diffusion large language models. Preprint, arXiv:2510.02902. — [arXiv:2510.02902](https://arxiv.org/abs/2510.02902) find
  22. Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. 2025. Dream 7b: Diffusion large language models. Preprint, arXiv:2508.15487.
  23. KiYoon Yoo, Wonhyuk Ahn, and Nojun Kwak. 2024a. Advancing beyond identification: Multi-bit watermark for large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p… find
  24. KiYoon Yoo, Wonhyuk Ahn, and Nojun Kwak. 2024b. Advancing beyond identification: Multi-bit watermark for large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p… find
  25. Runpeng Yu, Qi Li, and Xinchao Wang. 2025. Discrete diffusion in large language and multimodal models: A survey. Preprint, arXiv:2506.13759. — [arXiv:2506.13759](https://arxiv.org/abs/2506.13759) find
  26. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. Opt: Open pre-trai…
  27. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. Preprint, arXiv:1904.09675. find
  28. Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. 2023. Provable robust watermarking for ai-generated text. Preprint, arXiv:2306.17439.
  29. Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. 2025. Llada 1.5: Variance-reduced preference optimization for large language diffusion models. Preprint, arXiv:2505.19223.
  30. İsmail Tarım and Aytuğ Onan. 2025. Can you detect the difference? Preprint, arXiv:2507.10475. — [arXiv:2507.10475](https://arxiv.org/abs/2507.10475) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2026-you-can-have-second-chance/