Elamur Research⌕
☰
home / papers / 2026 / Anchored Sliding Window: Toward Robust and Imperceptible Linguistic Steganography

Anchored Sliding Window: Toward Robust and Imperceptible Linguistic Steganography

Paper · 2026 · Ruiyi Yan, Shiao Meng and Yugo Murawaki

Summary

LLM-based linguistic steganography usually assumes the stegotext reaches the receiver unchanged. Because extraction repeats autoregressive inference, altering one token corrupts decoding of everything after it. The earlier robust method, WinStega, conditions only on the most recent tokens. This makes it robust, but text quality drops sharply. The authors propose the Anchored Sliding Window (ASW), which builds each step's context from three parts:

  • The prompt. Both parties already share it, so an attacker cannot change it.
  • A bridge context. It does not depend on the tokens that were dropped from the window.
  • The latest w tokens.

The bridge first takes the form of a hand-written placeholder (a hard bridge). This tells the model that some text has been left out. It is then replaced by trainable soft embeddings. These are learned through a prompt-distillation formulation extended with self-distillation, so that ASW predictions stay close to full-context predictions. On instruction-following data with Qwen2.5 models, ASW beats WinStega in text quality, imperceptibility and robustness. The method makes covert text communication more practical over channels where messages may be edited.

Key points

  • The baseline is WinStega (Pang et al., 2025). Its entropy-threshold mechanism uses tokens outside its context window, which the authors argue weakens its robustness.
  • With Qwen2.5-7B-Instruct, ASW raises ROUGE-L by 104.08% over the baseline. The advantage grows with model size across Qwen2.5-3B, 7B and 14B-Instruct.
  • The main experiments use InstructionWild. Generalization is checked on databricks-dolly-15k and Super-NaturalInstructions.
  • Default settings are a soft bridge of 8 tokens and w = 10. Making the bridge longer than 8 adds compute with little gain. Increasing w improves quality but lowers robustness.
  • The soft bridge trained with forward KL gives a clearly higher embedding capacity than the other variants, and the authors leave the reason open. Code is available at github.com/ryehr/ASW_steganography.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China415 works · 94,028 citations🎓Kyoto UniversityKyoto · Japan42 works · 8,235 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

Cite

Cite as Ruiyi Yan, Shiao Meng and Yugo Murawaki. 2026. Anchored Sliding Window: Toward Robust and Imperceptible Linguistic Steganography. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.44

@inproceedings{yan2026anchored,
  title = {{Anchored Sliding Window: Toward Robust and Imperceptible Linguistic Steganography}},
  author = {Ruiyi Yan, Shiao Meng and Yugo Murawaki},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.44},
  url = {https://doi.org/10.18653/v1/2026.acl-long.44}
}

References

48 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  2. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  3. Jinyang Ding, Kejiang Chen, Yaofei Wang, Na Zhao, Weiming Zhang, and Nenghai Yu. 2023. Discop: Provably secure steganography in practice based on "distribution copies". In 2023 IEEE Symposium on Security and Privacy (SP), pages 2238–2255. find
  4. We express our gratitude to the anonymous reviewers for their valuable and insightful comments. This work was supported by JST SPRING, Grant Number JPMJSP2110. find
  5. Tina Fang, Martin Jaggi, and Katerina Argyraki. 2017. Generating steganographic text with LSTMs. In Proceedings of ACL 2017, Student Research Workshop, pages 100–106, Vancouver, Canada. Association for Computational Linguistics. find
  6. A.A. Fedotov, P. Harremoes, and F. Topsoe. 2003. Refinements of pinsker’s inequality. IEEE Transactions on Information Theory, 49(6):1491–1498. find
  7. Jonas Geiping, Alex Stein, Manli Shu, Khalid Saifullah, Yuxin Wen, and Tom Goldstein. 2024. Coercing LLMs to do and reveal (almost) anything. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models. find
  8. Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. MiniLLM: Knowledge distillation of large language models. In The Twelfth International Conference on Learning Representations. find
  9. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045–3059, Online and Punta Cana, Dominican Republic. Association for Computational Linguist…
  10. Lei Li, Yongfeng Zhang, and Li Chen. 2023. Prompt distillation for efficient llm-based recommendation. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, CIKM ’23, page 1348–1357, New York, NY, USA. Association for Computing Machinery. find
  11. Zheng Li, Xiang Li, Xinyi Fu, Xin Zhang, Weiqiang Wang, Shuo Chen, and Jian Yang. 2024. Promptkd: Unsupervised prompt distillation for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26617–26626. find
  12. Nicholas Hopper, Luis von Ahn, and John Langford. 2009. Provably secure steganography. IEEE Transactions on Computers, 58(5):662–676. find
  13. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  14. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  15. Chin-Yew Lin and Franz Josef Och. 2004. ORANGE: a method for evaluating automatic evaluation metrics for machine translation. In COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics, pages 501–507, Geneva, Switzerland. COLING.
  16. Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker. 1977. Perplexity—a measure of the difficulty of speech recognition tasks. The Journal of the Acoustical Society of America, 62(S1):S63–S63. find
  17. Mamta Juneja and Parvinder Singh Sandhu. 2009. Designing of robust image steganography technique based on lsb insertion and encryption. In 2009 International Conference on Advances in Recent Technologies in Communication and Computing, pages 302–305. find
  18. Gabriel Kaptchuk, Tushar M. Jois, Matthew Green, and Aviel D. Rubin. 2021. Meteor: Cryptographically secure steganography for realistic distributions. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, CCS ’21, page 1529–1548, New York, NY, USA. Association for… find
  19. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. find
  20. Mohammed Abdul Majeed, Rossilawati Sulaiman, Zarina Shukur, and Mohammad Kamrul Hasan. 2021. A review on text steganography techniques. Mathematics, 9(21). find
  21. Tom Minka and 1 others. 2005. Divergence measures and message passing. find
  22. Jinjie Ni, Fuzhao Xue, Kabir Jain, Mahir Hitesh Shah, Zangwei Zheng, and Yang You. 2023. Instruction in the wild: A user-based instruction dataset. https://github.com/XueFuzhao/InstructionWild. ↗
  23. Yoon Kim and Alexander M Rush. 2016. Sequence-level knowledge distillation. In Proceedings of the 2016 conference on empirical methods in natural language processing, pages 1317–1327.
  24. Jumon Nozaki and Yugo Murawaki. 2022. Addressing segmentation ambiguity in neural linguistic steganography. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Vo… find
  25. Sander Land and Max Bartolo. 2024. Fishing for magikarp: Automatically detecting under-trained tokens in large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11631–11646, Miami, Florida, USA. Association for Computational Linguistics…
  26. Kaiyi Pang, Minhao Bai, Jinshuai Yang, Wei-Qiang Zhang, Minghu Jiang, and Yongfeng Huang. 2025. Winstega: An adaptive robust enhancement framework for generative linguistic steganography. In ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1… find
  27. Yuang Qi, Kejiang Chen, Kai Zeng, Weiming Zhang, and Nenghai Yu. 2025. Provably secure disambiguating neural linguistic steganography. IEEE Transactions on Dependable and Secure Computing, 22(3):2430–2442. find
  28. Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, and 24 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15… find
  29. Ram Sundara Raman, Adrian Stoll, Jakub Dalek, Reethika Ramesh, Will Scott, and Roya Ensafi. 2020. Measuring the deployment of network censorship filters at global scale. In NDSS. find
  30. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2020. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. Preprint, arXiv:1910.01108.
  31. Jiaming Shen, Heng Ji, and Jiawei Han. 2020. Near-imperceptible neural linguistic steganography via self-adjusting arithmetic coding. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 303–313, Online. Association for Computational Linguistics. find
  32. Gustavus J. Simmons. 1984. The Prisoners’ Problem and the Subliminal Channel, pages 51–67. Springer US, Boston, MA. find
  33. Kaitao Song, Hao Sun, Xu Tan, Tao Qin, Jianfeng Lu, Hongzhi Liu, and Tie-Yan Liu. 2020. Lightpaff: A two-stage distillation framework for pre-training and fine-tuning. Preprint, arXiv:2004.12817. — [arXiv:2004.12817](https://arxiv.org/abs/2004.12817) find
  34. Nandhini Subramanian, Omar Elharrouss, Somaya Al-Maadeed, and Ahmed Bouridane. 2021. Image steganography: A review of the recent advances. IEEE Access, 9:23409–23423. find
  35. Yaofei Wang, Gang Pei, Kejiang Chen, Jinyang Ding, Chao Pan, Weilong Pang, Donghui Hu, and Weiming Zhang. 2025. SparSamp: Efficient provably secure steganography based on sparse sampling. In The 34th USENIX Security Symposium. find
  36. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal… find
  37. Jiaxuan Wu, Zhengxian Wu, Yiming Xue, Juan Wen, and Wanli Peng. 2024. Generative text steganography with large language model. In Proceedings of the 32nd ACM International Conference on Multimedia, MM ’24, page 10345–10353, New York, NY, USA. Association for Computing Machinery. find
  38. Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. Efficient streaming language models with attention sinks. In The Twelfth International Conference on Learning Representations. find
  39. Ruiyi Yan, Chenhui Chu, Zhongliang Yang, and Yugo Murawaki. 2025. A comprehensive survey on linguistic steganography: Methods, countermeasures, evaluation, and challenges. find
  40. Ruiyi Yan and Yugo Murawaki. 2025. Addressing tokenization inconsistency in steganography and watermarking based on large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 7076–7098, Suzhou, China. Association for Computational Linguist… find
  41. Ruiyi Yan, Tian Song, and Yating Yang. 2024. A near-imperceptible disambiguating approach via verification for generative linguistic steganography. In 2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 1638–1643. find
  42. Ruiyi Yan, Yating Yang, and Tian Song. 2023. A secure and disambiguating approach for generative linguistic steganography. IEEE Signal Processing Letters, 30:1047–1051. find
  43. Zhong-Liang Yang, Xiao-Qing Guo, Zi-Ming Chen, Yong-Feng Huang, and Yu-Jin Zhang. 2019. Rnn-stega: Linguistic steganography based on recurrent neural networks. IEEE Transactions on Information Forensics and Security, 14(5):1280–1295. find
  44. Zhong-Liang Yang, Si-Yu Zhang, Yu-Ting Hu, Zhi-Wen Hu, and Yong-Feng Huang. 2021a. Vae-stega: Linguistic steganography based on variational autoencoder. IEEE Transactions on Information Forensics and Security, 16:880–895. find
  45. Zhongliang Yang, Lingyun Xiang, Siyu Zhang, Xingming Sun, and Yongfeng Huang. 2021b. Linguistic generative steganography with enhanced cognitive-imperceptibility. IEEE Signal Processing Letters, 28:409–413. find
  46. Siyu Zhang, Zhongliang Yang, Jinshuai Yang, and Yongfeng Huang. 2021. Provably secure generative linguistic steganography. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3046–3055, Online. Association for Computational Linguistics. find
  47. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. find
  48. Zachary Ziegler, Yuntian Deng, and Alexander Rush. 2019. Neural linguistic steganography. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1210–1215, Hong Kong, Ch… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yan-2026-anchored-sliding-window-robust-imperceptible/