Elamur Research⌕
☰
home / papers / 2026 / A Linguistics-Aware LLM Watermarking via Syntactic Predictability

A Linguistics-Aware LLM Watermarking via Syntactic Predictability

Paper · 2026 · Shinwoo Park, Hyejin Park, Hyeseon An et al.

Summary

LLM watermarking must balance fluent output against a signal that can be reliably detected. Adaptive schemes such as SWEET and EWD manage this with token entropy, but their detectors then need the generating model's logits, so outside parties cannot verify them. Park et al. propose STELA, which uses a property of the language instead of model uncertainty: how predictable the next part-of-speech tag is, given the preceding k−1 tags. This conditional entropy of POS n-grams is estimated from a human-written corpus and normalized by the number of tag types seen after each context. It scales a KGW-style green-list bias during generation and also weights tokens in the detection z-statistic. Where grammar dictates the continuation, the bias shrinks to protect quality. Where many continuations are possible, it grows. Detection needs only a public hash and a lightweight POS tagger. Experiments covered English, Chinese and Korean with Llama-3.2, Qwen-3 and HyperCLOVA. STELA detected watermarks more robustly than prior methods and resisted synonym and paraphrase attacks. This makes adaptive watermark strength compatible with public auditing, such as the transparency demands of the EU AI Act.

Key points

  • Taxonomy of methods:
    • STELA is presented as the only method that combines weighted adaptive insertion, weighted adaptive detection and model-free verification.
    • It is compared against KGW (static, model-free), SWEET and EWD (entropy-based, model-dependent) and MorphMark (adaptive insertion only).
  • Typologically varied languages: English (analytic), Chinese (isolating) and Korean (agglutinative).
    • Language-specific tagsets helped more than coarse UD tags. For example, they separate the Korean case particles JKS and JKO, which UD would group under PART.
  • Where the watermark signal comes from (share of the z-score):
    • English: content words 43.1%, function words 43.7%.
    • Chinese: content words 67.1%.
    • Korean: content words 74.4%.
    • These shares are consistent with each language's typology.
  • Synonym attack: replacing up to 50% of English words with WordNet synonyms kept Best F1 above 0.85 for all three models.
  • Dipper paraphrasing (Llama-3.2, English):
    • Best F1 fell from 0.953 with no attack to 0.813 at L=40 and 0.825 at L=50.
    • Detection did not collapse even under heavy rewriting.
    • Code: github.com/Shinwoo-Park/stela_watermark.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Rensselaer Polytechnic InstituteTroy · United States21 works · 4,933 citations🎓Yonsei UniversitySeoul · South Korea17 works · 501 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 worksHardware and Architecture · 81 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Shinwoo Park, Hyejin Park, Hyeseon An and Yo-Sub Han. 2026. A Linguistics-Aware LLM Watermarking via Syntactic Predictability. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.2115

@inproceedings{park2026linguistics,
  title = {{A Linguistics-Aware LLM Watermarking via Syntactic Predictability}},
  author = {Shinwoo Park, Hyejin Park, Hyeseon An and Yo-Sub Han},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.2115},
  url = {https://doi.org/10.18653/v1/2026.acl-long.2115}
}

References

43 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hyeseon An, Shinwoo Park, Suyeon Woo, and Yo-Sub Han. 2026. DITTO: A Spoofing Attack Framework on Watermarked LLMs via Knowledge Distillation. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL). find
  2. Liang Chen, Yatao Bian, Yang Deng, Deng Cai, Shuaiyi Li, Peilin Zhao, and Kam-Fai Wong. 2024. WatME: Towards lossless watermarking through lexical redundancy. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL). find
  3. Bernard Comrie. 1989. Language Universals and Linguistic Typology: Syntax and Morphology. University of Chicago press. find
  4. Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, and 1 others. 2024. Scalable watermarking for identifying large language model outputs. Nature. find
  5. EU AI Act. 2024. EU AI Act Compliance Checker. find
  6. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL).
  7. Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database. MIT press. find
  8. Wikimedia Foundation. 2023. Wikimedia wikipedia dataset. find
  9. Eva Giboulot and Teddy Furon. 2024. Watermax: Breaking the llm watermark detectability-robustness-quality trade-off. In Advances in Neural Information Processing Systems (NeurIPS). find
  10. Carlos Gómez-Rodríguez and Paul Williams. 2023. A confederacy of models: a comprehensive evaluation of LLMs on creative writing. In Findings of the Association for Computational Linguistics (EMNLP). find
  11. Yuxuan Guo, Zhiliang Tian, Yiping Song, Tianlun Liu, Liang Ding, and Dongsheng Li. 2024. Context-aware watermark with semantic balanced green-red lists for large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  12. HAERAE-Team. 2024. Korean-webtext: A high-quality korean language corpus. find
  13. Han He and Jinho D. Choi. 2021. The stem cell hypothesis: Dilemma behind multi-task learning with transformer encoders. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  14. Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. 2020. spacy: Industrial-strength natural language processing in python. find
  15. Yinghao Hu, Leilei Gan, Wenyi Xiao, Kun Kuang, and Fei Wu. 2025. Fine-tuning large language models for improving factuality in legal question answering. In Proceedings of the 31st International Conference on Computational Linguistics (COLING). find
  16. Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Xi Zhang, Lijie Wen, Irwin King, Hui Xiong, and Philip Yu. 2024. A survey of text watermarking in the era of large language models. ACM Computing Surveys. find
  17. Che Jiang, Biqing Qi, Xiangyu Hong, Dayuan Fu, Yang Cheng, Fandong Meng, Mo Yu, Bowen Zhou, and Jie Zhou. 2024. On large language models’ hallucination with regard to known facts. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Hu… find
  18. Yijian Lu, Aiwei Liu, Dianzhi Yu, Jingjing Li, and Irwin King. 2024. An entropy-based text watermarking detection method. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL). find
  19. Jungin Kim, Shinwoo Park, and Yo-Sub Han. 2026. Marking Code Without Breaking It: Code Watermarking for Detecting LLM-Generated Code. In Findings of the Association for Computational Linguistics (EACL). find
  20. Meta AI. 2024. Llama 3.2: Revolutionizing edge ai and vision with open, customizable models. find
  21. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. 2023. DetectGPT: Zero-shot Machine-Generated Text Detection using Probability Curvature. In International Conference on Machine Learning (ICML).
  22. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. A watermark for large language models. In International Conference on Machine Learning (ICML).
  23. NAVER CLOVA. 2025. Planting the seeds for an ai ecosystem: Introducing hyperclova x seed, a commercial open-source ai. find
  24. Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2023. Paraphrasing Evades Detectors of AI-generated Text, but Retrieval is an Effective Defense. In Advances in Neural Information Processing Systems (NeurIPS).
  25. Pedro Javier Ortiz Suárez, Laurent Romary, and Benoît Sagot. 2020. A monolingual approach to contextualized word embeddings for mid-resource languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL).
  26. Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. 2024. Robust distortion-free watermarks for language models. Transactions on Machine Learning Research. find
  27. Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, Irwin King, and Philip S. Yu. 2024. MarkLLM: An open-source toolkit for LLM watermarking. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing:… find
  28. Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. 2024. BioMistral: A collection of open-source pretrained large language models for medical domains. In Findings of the Association for Computational Linguistics (ACL).
  29. Harsh Nishant Lalai, Aashish Anantha Ramakrishnan, Raj Sanjay Shah, and Dongwon Lee. 2025. From intentions to techniques: A comprehensive taxonomy and challenges in text watermarking for large language models. In Findings of the Association for Computational Linguistics (NAACL). find
  30. Min-chul Lee. 2024. Kiwi: Developing a korean morphological analyzer based on statistical language models and skip-bigram. Korean Journal of Digital Humanities. find
  31. Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. 2024. Who wrote this code? watermarking for code generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL).
  32. Charles N Li and Sandra A Thompson. 1989. Mandarin Chinese: A Functional Reference Grammar. Univ of California Press. find
  33. Shinwoo Park, Shubin Kim, Do-Kyung Kim, and Yo-Sub Han. 2025. KatFishNet: Detecting LLM-Generated Korean Text through Linguistic Feature Analysis. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL). find
  34. Shinwoo Park, Hyejin Park, Hyeseon Ahn, and Yo-Sub Han. 2026. WaterMod: Modular Token-Rank Partitioning for Probability-Balanced LLM Watermarking. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). find
  35. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. C4: Colossal clean crawled corpus. find
  36. Yijia Shao, Yucheng Jiang, Theodore Kanell, Peter Xu, Omar Khattab, and Monica Lam. 2024. Assisting in writing Wikipedia-like articles from scratch with large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human…
  37. SirlyDreamer. 2023. Thucnews. find
  38. Ho-Min Sohn. 2001. The Korean Language. Cambridge University Press. find
  39. Geralt Targaryen. 2025. Openwebtext2. find
  40. Ivan Vykopal, Matúš Pikuliak, Ivan Srba, Robert Moro, Dominik Macko, and Maria Bielikova. 2024. Disinformation capabilities of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL). find
  41. Zongqi Wang, Tianle Gu, Baoyuan Wu, and Yujiu Yang. 2025. MorphMark: Flexible adaptive watermarking for large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL).
  42. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  43. Xianjun Yang, Liangming Pan, Xuandong Zhao, Haifeng Chen, Linda Ruth Petzold, William Yang Wang, and Wei Cheng. 2024. A survey on detection of LLMs-generated content. In Findings of the Association for Computational Linguistics (EMNLP). find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/park-2026-linguistics-aware-llm-watermarking-syntactic/