Elamur Research⌕
☰
home / papers / 2026 / Synthetic Text Detection in the Age of Large Language Models: Watermark vs. Automatic Detection

Synthetic Text Detection in the Age of Large Language Models: Watermark vs. Automatic Detection

Paper · 2026 · Adaku Uchendu

Summary

LLM output is now hard to tell apart from human writing. Two families of detectors compete to identify it. Watermarking embeds a key-dependent signal during generation, while post-hoc automatic detectors rely on statistical and stylistic cues. This Industry Track paper asks which family works better and under what conditions. The author generates short texts (motivational quotes, under 100 words) and long fiction (over 450 words) with six Qwen2.5 sizes from 1.5B to 72B. Each prompt yields an unwatermarked output plus outputs from six watermarking schemes run through MarkLLM. Text quality is checked with reference-based and reference-free metrics. The watermark detectors are compared against two zero-shot automatic detectors, MAGE and Binoculars, on clean texts and on texts altered by character-, word- and sentence-level obfuscation. No approach wins in every setting. Automatic detection is stronger on short texts, watermarks hold up better against misspellings, and homoglyph substitution defeats nearly every detector. The practical lesson is to pick a detector based on expected text length and likely attacks.

Key points

  • Setup: six Qwen2.5 models (1.5B, 3B, 7B, 14B, 32B, 72B). Watermarks: KGW, UPV, SWEET, EWD, TS, Unbiased. Automatic detectors: MAGE (Longformer-based) and Binoculars (Falcon perplexity-based).
  • Data: 1066 motivational-quote prompts and 1035 fiction-story instructions. Each model produces 7 versions per prompt (one unwatermarked, six watermarked).
  • Clean texts: MAGE exceeds 90% accuracy on both short and long texts. Binoculars is weaker on long texts, and both automatic detectors beat watermark detection on short texts.
  • Attacks:
    • Homoglyph substitution is the most damaging; even the best detector, MAGE, reaches only about 10–40%. TS only looks robust because it labels almost everything as watermarked.
    • Under the misspelling attack, KGW, SWEET, EWD and TS keep high accuracy.
    • Under Mistral-based paraphrasing, only MAGE stays above 90%.
  • Model size: no Qwen2.5 size was consistently easier or harder to detect across all detectors.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬MIT Lincoln LaboratoryLexington · United States4 works

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Adaku Uchendu. 2026. Synthetic Text Detection in the Age of Large Language Models: Watermark vs. Automatic Detection. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-industry.9

@inproceedings{uchendu2026synthetic,
  title = {{Synthetic Text Detection in the Age of Large Language Models: Watermark vs. Automatic Detection}},
  author = {Adaku Uchendu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.9},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.9}
}

References

7 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. George Alexandru Adam, Alexander Cui, Edwin Thomas, Emily Napier, Nazar Shmatko, Jacob Schnell, Jacob Junqi Tian, Alekhya Dronavalli, Edward Tian, and Dongwon Lee. 2026. Gptzero: Robust detection of llm-generated texts. Preprint, arXiv:2602.13042. — [arXiv:2602.13042](https://arxiv.org/abs/2602.13042) find
  2. Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, and 1 others. 2023. The falcon series of open language models. arXiv preprint arXiv:2311.16867.
  3. Kenneth Alperin, Rohan Leekha, Adaku Uchendu, Trang Nguyen, Srilakshmi Medarametla, Carlos Levya Capote, Seth Aycock, and Charlie Dagli. 2025. Masks and mimicry: Strategic obfuscation and impersonation attacks on authorship verification. In Proceedings of the 5th International Conference on Natural… find
  4. Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.
  5. Shuyang Cai and Wanyun Cui. 2023. Evade chatgpt detectors via a single space. Preprint, arXiv:2307.02599. — [arXiv:2307.02599](https://arxiv.org/abs/2307.02599) find
  6. Dialects and Languages: We only focus on standard English for our study, however, it will be beneficial to observe the performance of these detectors and obfuscators on other English dialects. Furthermore, we can explore different languages and their variants as well. find
  7. Miranda Christ, Sam Gunn, and Or Zamir. 2024. Undetectable watermarks for language models. In The Thirty Seventh Annual Conference on Learning Theory, pages 1125–1139. PMLR. find

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/uchendu-2026-synthetic-text-detection-age-large/