Elamur Research⌕
☰
home / papers / 2026 / No Innocence in Styling: Discovery of Privacy Protection Capabilities and Security Risks in Consumer Generative AI Writing Assistants

No Innocence in Styling: Discovery of Privacy Protection Capabilities and Security Risks in Consumer Generative AI Writing Assistants

Paper · 2026 · Mohd. Farhan Israk Soumik, Syed Mhamudul Hasan, Wanniarachchi Kankanamge Malithi Mithsara et al.

Summary

The paper asks what happens when built-in AI rewriting tools are used at scale before text reaches platform classifiers. The tools studied are Apple Intelligence and Microsoft Copilot. The authors frame this as a dual-use problem. Restyling text can hide emotional cues that platforms or third parties use for profiling. The same restyling can also help users slip misinformation past automated detectors. They define a threat model with two scenarios: privacy against emotion profiling, and moderation evasion. They then build parallel stylized corpora using four common modes, applied with both commercial assistants and three open-source models. Emotion and fake-news classifiers are fine-tuned, and GPT-4o is evaluated zero-shot. Results show that stylization lowers emotion-inference accuracy, which helps privacy, but raises misclassification of fake news as real. Retraining on stylized data reverses both effects. The main message is that writing assistants are not neutral productivity features. They act as system-level transformation layers. Moderation pipelines that depend on lexical and emotional cues therefore need style-invariant designs.

Key points

  • Stylization systems and modes:
    • Systems: Apple Intelligence (on-device), Microsoft Copilot (cloud), and open-source Qwen-30B-3AE, LLaMa3.1-8B and Mistral-7B.
    • Modes: Rewrite, Professional/Formal, Friendly and Concise.
  • Data, classifiers and metrics:
    • Emotion datasets: Dair-AI, DailyDialog and ISEAR. Misinformation dataset: ISOT Fake News.
    • Classifiers: BERT, RoBERTa, DistilBERT, DeBERTa-v3 and Flan-T5, plus GPT-4o zero-shot.
    • Metrics: entailment (E), sBLEU, BODEGA score, Privacy Preservation Rate (PPR) and LFR.
  • Professional mode is the most effective for evasion:
    • Apple Intelligence achieves an LFR of 62.0%, while Copilot reaches 68.0% and Qwen-30B-3AE reaches 55.0%.
    • BODEGA scores on the same systems are 0.413, 0.488 and 0.331.
    • Copilot shows the strongest overall balance of evasion and utility.
  • Other modes trade off differently:
    • Friendly gives the highest LFR on LLaMA3.1-8B (47.4%) and Mistral-7B (47.0%), but with low entailment (0.22 and 0.35).
    • Rewrite keeps meaning best but evades less.
    • Concise performs worst overall.
  • Augmentation with stylized data reverses the effects:
    • Adding about 24K stylized samples per dataset (10K for ISEAR), evaluated with an 80/20 split, sharply cuts fake-news evasion.
    • The same augmentation also weakens the privacy protection gained from stylization.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Florida Atlantic UniversityBoca Raton · United States2 works · 256 citations

Also on the byline: Southern Illinois University Carbondale.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSafety Research · 543 worksSociology and Political Science · 577 works

Cite

Cite as Mohd. Farhan Israk Soumik, Syed Mhamudul Hasan, Wanniarachchi Kankanamge Malithi Mithsara, Ahmed Imteaj and Abdur R. Shahid. 2026. No Innocence in Styling: Discovery of Privacy Protection Capabilities and Security Risks in Consumer Generative AI Writing Assistants. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-industry.128

@inproceedings{soumik2026no,
  title = {{No Innocence in Styling: Discovery of Privacy Protection Capabilities and Security Risks in Consumer Generative AI Writing Assistants}},
  author = {Mohd. Farhan Israk Soumik, Syed Mhamudul Hasan, Wanniarachchi Kankanamge Malithi Mithsara, Ahmed Imteaj and Abdur R. Shahid},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.128},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.128}
}

References

49 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ashweta A. Fondekar, Milind M. Shivolkar, and Jyoti D. Pawar. 2024. Unpacking faux-hate: Addressing faux-hate detection and severity prediction in code-mixed Hinglish text with HingRoBERTa and class weighting techniques. In Proceedings of the 21st International Conference on Natural Language Process… find
  2. Hadeer Ahmed, Issa Traore, and Sherif Saad. 2018. find
  3. Detecting opinion spams and fake news using text classification. Security and Privacy, 1(1):e9. find
  4. Martin Juan José Bucher and Marco Martini. 2024. Fine-tuned’small’llms (still) significantly outperform zero-shot generative ai models in text classification. arXiv preprint arXiv:2406.08660. — [arXiv:2406.08660](https://arxiv.org/abs/2406.08660) find
  5. Yize Cheng, Vinu Sankar Sadasivan, Mehrdad Saberi, Shoumik Saha, and Soheil Feizi. 2025. Adversarial paraphrasing: A universal attack for humanizing ai-generated text. arXiv preprint arXiv:2506.07001. find
  6. Ning Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang. 2019. Style transformer: Unpaired text style transfer without disentangled latent representation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5997– 6007, Florence, Italy. Association for Com…
  7. Mateja Durovic and Tommaso Corno. 2024. The privacy of emotions: From the gdpr to the ai act, an overview of emotional ai regulation and the protection of privacy and personal data. Privacy, Data Protection and Data-driven Technologies, pages 368– 404. find
  8. Daria Galimzianova, Aleksandr Boriskin, and Grigory Arshinov. 2025. From rag to reality: Coarse-grained hallucination detection via nli fine-tuning. In Proceedings of the Fifth Workshop on Scholarly Document Processing (SDP 2025), pages 353–359. find
  9. Tom Gunter, Zirui Wang, Chong Wang, Ruoming Pang, Andy Narayanan, Aonan Zhang, Bowen Zhang, Chen Chen, Chung-Cheng Chiu, David Qiu, et al. 2024. Apple intelligence foundation language models. arXiv preprint arXiv:2407.21075. — [arXiv:2407.21075](https://arxiv.org/abs/2407.21075) find
  10. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Toward controlled generation of text. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1587–1596. PMLR.
  11. Nils Klüwer, Irina Nalis, and Julia Neidhardt. 2025. Context over categories: Implementing the theory of constructed emotion with llm-guided user analysis. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA ’25, New York, NY, USA. Association… find
  12. Chaona Kong, Jianyi Liu, Yifan Tang, and Ru Zhang. 2025. Neuron activation modulation for text style transfer: Guiding large language models. In Findings of the Association for Computational Linguistics: ACL 2025, pages 7735–7747, Vienna, Austria. Association for Computational Linguistics. find
  13. Lindrit Kqiku, Marvin Kühn, and Delphine Reinhardt. 2022. From sentiment to sensitivity: The role of emotions on privacy exposure in twitter. In Proceedings of the 2022 Workshop on Open Challenges in Online Social Networks, pages 10–15. find
  14. Wen Lai, Viktor Hangya, and Alexander Fraser. 2024. Style-specific neurons for steering LLMs in text style transfer. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13427–13443, Miami, Florida, USA. Association for Computational Linguistics. find
  15. Juncen Li, Robin Jia, He He, and Percy Liang. 2018. Delete, retrieve, generate: a simple approach to sentiment and style transfer. arXiv preprint arXiv:1804.06437.
  16. Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. DailyDialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 986–995, Taipei, Taiwan. Asian Federati…
  17. Xun Liang, Hanyu Wang, Shichao Song, Mengting Hu, Xunzhi Wang, Zhiyu Li, Feiyu Xiong, and Bo Tang. 2024. Controlled text generation for large language model with dynamic attribute graphs. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5797–5814, Bangkok, Thailand. Asso… find
  18. Pusheng Liu, Lianwei Wu, Linyong Wang, Sensen Guo, and Yang Liu. 2024a. Step-by-step: Controlling arbitrary style in text with large language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15… find
  19. Shuai Liu and Jonathan May. 2025. Style transfer with multi-iteration preference optimization. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2663–2681, Albuquer… find
  20. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  21. Zhiwei Liu, Kailai Yang, Qianqian Xie, Tianlin Zhang, and Sophia Ananiadou. 2024b. Emollms: A series of emotional large language models and annotation tools for comprehensive affective analysis. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5487– 5496… find
  22. Alexander Loth, Martin Kappes, and Marc-Oliver Pahl. 2026. Industrialized deception: The collateral effects of llm-generated misinformation on digital ecosystems. arXiv preprint arXiv:2601.21963. — [arXiv:2601.21963](https://arxiv.org/abs/2601.21963) find
  23. Nestor Maslej, Loredana Fattorini, Raymond Perrault, Yolanda Gil, Vanessa Parli, Njenga Kariuki, Emily Capstick, Anka Reuel, Erik Brynjolfsson, John Etchemendy, et al. 2025. Artificial intelligence index report 2025. arXiv preprint arXiv:2504.07139. — [arXiv:2504.07139](https://arxiv.org/abs/2504.07139) find
  24. Microsoft. 2023. Introducing copilot for microsoft 365. https://www.microsoft.com/ en-us/microsoft-365/copilot. ↗
  25. Rostam J Neuwirth. 2023. Prohibited artificial intelligence practices in the proposed eu artificial intelligence act (aia). Computer Law & Security Review, 48:105798. find
  26. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  27. Jeffrey Pawlick, Edward Colbert, and Quanyan Zhu. 2019. A game-theoretic taxonomy and survey of defensive deception for cybersecurity and privacy. ACM Computing Surveys (CSUR), 52(4):1–28. find
  28. Martin Potthast, Johannes Kiesel, Kevin Reinartz, Janek Bevendorff, and Benno Stein. 2018. A stylometric inquiry into hyperpartisan and fake news. In Proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers), pages 231–240.
  29. Piotr Przybyła, Euan McGill, and Horacio Saggion. 2025. Attacking misinformation detection using adversarial examples generated by language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 27614–27630. find
  30. Piotr Przybyła, Alexander Shvets, and Horacio Saggion. 2024. Verifying the robustness of automatic credibility assessment. Natural Language Processing, pages 1–29. find
  31. Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. 2020. Pre-trained models for natural language processing: A survey. Science China technological sciences, 63(10):1872– 1897.
  32. Alec Radford and Karthik Narasimhan. 2018. Improving language understanding by generative pre-training. find
  33. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. find
  34. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67.
  35. Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of the 2017 conference on empirical methods in natural language processing, pages 2931–2937.
  36. Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. 2018. CARER: Contextualized affect representations for emotion recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3687–3697, Brussels, Belgium. Association for Co…
  37. Klaus R Scherer and Harald G Wallbott. 1994. Evidence for universality and cultural variation of differential emotion response patterning. Journal of personality and social psychology, 66(2):310. find
  38. Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. Advances in neural information processing systems, 30.
  39. Robin Staab, Mark Vero, Mislav Balunović, and Martin Vechev. 2023. Beyond memorization: Violating privacy via inference with large language models. arXiv preprint arXiv:2310.07298. — [arXiv:2310.07298](https://arxiv.org/abs/2310.07298) find
  40. Martina Toshevska, Slobodan Kalajdziski, and Sonja Gievska. 2025. Style knowledge graph: Augmenting text style transfer with knowledge graphs. In Proceedings of the Workshop on Generative AI and Knowledge Graphs (GenAIK), pages 123–135, Abu Dhabi, UAE. International Committee on Computational Lingui… find
  41. Qianli Wang, Nils Feldhus, Luis Felipe Villa-Arenas, Christin Seifert, Sebastian Möller, Vera Schmitt, et al. 2025a. Truth or twist? optimal model selection for reliable label flipping evaluation in llm-based counterfactuals. In Proceedings of the 18th International Natural Language Generation Confe… find
  42. Xinyu Wang, Wenbo Zhang, Sai Koneru, Hangzhi Guo, Bonam Mingole, S Shyam Sundar, Sarah Rajtmajer, and Amulya Yadav. 2025b. Have llms reopened the pandora’s box of ai-generated fake news? In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational… find
  43. Jiaying Wu, Jiafeng Guo, and Bryan Hooi. 2024. Fake news in sheep’s clothing: Robust fake news detection against llm-empowered style attacks. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pages 3367–3378. find
  44. Yusen Wu and Xiaotie Deng. 2025. Implementing long text style transfer with llms through dual-layered sentence and paragraph structure extraction and mapping. arXiv preprint arXiv:2505.07888. find
  45. Xinfeng Yuan, Siyu Yuan, Yuhan Cui, Tianhe Lin, Xintao Wang, Rui Xu, Jiangjie Chen, and Deqing Yang. 2024. Evaluating character understanding of large language models via character profiling from fictional works. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processi… find
  46. Self-BLEU (sBLEU): Measures lexical overlap between the original text x and stylized text x styl to quantify the degree of stylistic transformation (Papineni et al., 2002). A lower score indicates higher diversity and a more significant rewrite: sBLEU (x, x styl ) = exp N X w n ln p n ! find
  47. Chiyu Zhang, Honglong Cai, Yuezhang Li, Yuexin Wu, Le Hou, and Muhammad Abdul-Mageed. 2024. Distilling text style transfer with self-explanation from llms. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie… find
  48. BODEGA Score (S BODEGA ): Aggregates attack success with semantic and lexical fidelity into one utility score (Przybyła et al., 2024): Ting Zhang, Ivana Clairine Irsan, Ferdian Thung, and David Lo. 2025. Revisiting sentiment analysis for software engineering in the era of large language models. ACM… find
  49. Wenxuan Zhang, Yue Deng, Bing Liu, Sinno Jialin Pan, and Lidong Bing. 2023. Sentiment analysis in the era of large language models: A reality check. arXiv preprint arXiv:2305.15005.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/soumik-2026-no-innocence-styling-discovery-privacy/