Elamur Research
☰
home / papers / 2021 / Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer

Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2021.emnlp-main.374.pdf

Summary

The paper examines whether text style can be weaponized against NLP classifiers. Style is the set of lexical and syntactic habits that sit apart from meaning, so it is irrelevant to most tasks. The authors argue that both adversarial and backdoor attacks rely on such irrelevant features, which makes style transfer a natural tool for both. They present what they describe as the first attacks of this kind:

  • Adversarial attack: a test input is rewritten repeatedly into several styles until the victim model mislabels it.
  • Backdoor attack: part of the training data is converted into a chosen trigger style, for example a Bible-like register. An auxiliary loss is added during training so the model remembers the trigger more firmly.

They evaluate both attacks on three popular models across three tasks: identifying sentiment, flagging hateful language and sorting news articles by topic. The attacks frequently exceed 90% success and beat baseline attacks in many respects. The backdoor stays effective even when a defense is applied. The authors conclude that current NLP models handle stylistic variation poorly, a security weakness that the community has largely overlooked.

Key points

  • Datasets: SST-2, HS and AG's News, all openly available. The authors used base versions of pre-trained models rather than large ones to save energy.
  • The adversarial attack exceeds 90% success on SST-2 against every victim model. It also outperforms baselines on success rate, adversarial example quality and attack validity.
  • The backdoor attack reaches over 90% success in almost all settings, even with a backdoor defense in place. Without a defense it trails the baselines slightly, but it is far stronger when a defense is present, which reflects how hard the trigger is to detect.
  • Human evaluation was carried out through a professional annotation company. Suggested future defenses include style-based data augmentation and style-transferring test inputs before they reach the model.
  • Code and data are available at github.com/thunlp/StyleAttack.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Cited by

61 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 60 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.emnlp-main.374

@inproceedings{qi2021mind,
  title = {{Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer}},
  author = {Qi, Fanchao and Chen, Yangyi and Zhang, Xurui and Li, Mukai and Liu, Zhiyuan and Sun, Maosong},
  booktitle = {Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.374},
  url = {https://doi.org/10.18653/v1/2021.emnlp-main.374}
}

References

65 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. In Proceedings of the EMNLP. find our card
  2. Chuanshuai Chen and Jiazhu Dai. 2020. Mitigating backdoor attacks in lstm-based text classification systems by backdoor keyword identification. arXiv preprint arXiv:2007.12070. ↗
  3. Xiaoyi Chen, Ahmed Salem, Michael Backes, Shiqing Ma, and Yang Zhang. 2020. Badnl: Backdoor attacks against nlp models. arXiv preprint arXiv:2006.01043. ↗
  4. Jiazhu Dai, Chuanshuai Chen, and Yufeng Li. 2019a. A backdoor attack against lstm-based text classification systems. IEEE Access, 7:138872–138878. find
  5. Ning Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang. 2019b. Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation. In Proceedings of ACL. find our card
  6. Ona de Gibert, Naiara Perez, Aitor García-Pablos, and Montse Cuadros. 2018. Hate speech dataset from a white supremacy forum. In Proceedings of the 2nd Workshop on Abusive Language Online. find our card
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT. find
  8. Chrysanne DiMarco and Graeme Hirst. 1993. A computational theory of goal-directed style in syntax. Computational Linguistics, 19(3):451–500. find
  9. Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018. Hotflip: White-box adversarial examples for text classification. In Proceedings of ACL. find our card
  10. Steffen Eger, Gözde Gül Şahin, Andreas Rücklé, Ji-Ung Lee, Claudia Schulz, Mohsen Mesgar, Krishnkant Swarnkar, Edwin Simpson, and Iryna Gurevych. 2019. Text processing like humans do: Visually attacking and shielding NLP systems. In Proceedings of NAACL-HLT. find
  11. Elizabeth Ford, John A Carroll, Helen E Smith, Donia Scott, and Jackie A Cassell. 2016. Extracting information from the text of electronic medical records to improve case detection: a systematic review. Journal of the American Medical Informatics Association, 23(5):1007–1015. find
  12. Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. 2018. Style Transfer in Text: Exploration and Evaluation. In Proceedings of AAAI. find
  13. Yonatan Belinkov and Yonatan Bisk. 2018. Synthetic and natural noise both break neural machine translation. In Proceedings of ICLR. find
  14. Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. 2018. Black-box generation of adversarial text sequences to evade deep learning classifiers. In Proceedings of IEEE Security and Privacy Workshops. find our card
  15. Alexy Bhowmick and Shyamanta M Hazarika. 2018. E-mail spam filtering: a review of techniques and trends. In Advances in Electronics, Communication and Computing, pages 583–590. Springer. find
  16. Hongyu Gong, Suma Bhat, Lingfei Wu, JinJun Xiong, and Wen-Mei Hwu. 2019. Reinforcement learning based text style transfer without parallel training corpus. In Proceedings of NAACL-HLT. find
  17. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Proceedings of NIPS. find
  18. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In Proceedings of ICLR. find our card
  19. Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. 2017. BadNets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733. ↗
  20. Junxian He, Xinyi Wang, Graham Neubig, and Taylor Berg-Kirkpatrick. 2020. A probabilistic formulation of unsupervised text style transfer. In Proceedings of ICLR. find
  21. Hossein Hosseini, Sreeram Kannan, Baosen Zhang, and Radha Poovendran. 2017. Deceiving google’s perspective api built for detecting toxic comments. arXiv preprint arXiv:1702.08138. ↗
  22. Eduard Hovy. 1987. Generating natural language under pragmatic constraints. Journal of Pragmatics, 11(6):689–719. ↗
  23. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. 2017. Toward controlled generation of text. In Proceedings of ICML. find our card
  24. Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial example generation with syntactically controlled paraphrase networks. In Proceedings of NAACL-HLT. find our card
  25. Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Proceedings of EMNLP. find our card
  26. Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of AAAI. find our card
  27. Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. 2019. Disentangled representation learning for non-parallel text style transfer. In Proceedings of ACL. find our card
  28. Kalpesh Krishna, John Wieting, and Mohit Iyyer. 2020. Reformulating Unsupervised Style Transfer as Paraphrase Generation. In Proceedings of EMNLP. find our card
  29. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. In Proceedings of ICLR. find
  30. Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. 2019. Textbugger: Generating adversarial text against real-world applications. In Proceedings of Network and Distributed Systems Security Symposium. find our card
  31. Juncen Li, Robin Jia, He He, and Percy Liang. 2018. Delete, Retrieve, Generate: a Simple Approach to Sentiment and Style Transfer. In Proceedings of NAACL-HLT. find our card
  32. Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020. Bert-attack: Adversarial attack against bert using bert. In Proceedings of EMNLP. find our card
  33. Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. 2018. Trojaning Attack on Neural Networks. In Proceedings of Network and Distributed Systems Security Symposium. find
  34. Fuli Luo, Peng Li, Jie Zhou, Pengcheng Yang, Baobao Chang, Zhifang Sui, and Xu Sun. 2019. A dual reinforcement learning framework for unsupervised text style transfer. In Proceedings of IJCAI. find
  35. Daniel Naber et al. 2003. A rule-based style and grammar checker. Citeseer. find
  36. Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W. Black. 2018. Style Transfer Through Back-Translation. In Proceedings of ACL. find our card
  37. Fanchao Qi, Yangyi Chen, Mukai Li, Zhiyuan Liu, and Maosong Sun. 2020. Onion: A simple and effective defense against textual backdoor attacks. arXiv preprint arXiv:2011.10369. ↗
  38. Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, and Maosong Sun. 2021a. Hidden killer: Invisible textual backdoor attacks with syntactic trigger. In Proceedings of ACL. find
  39. Fanchao Qi, Yuan Yao, Sophia Xu, Zhiyuan Liu, and Maosong Sun. 2021b. Turn the combination lock: Learnable textual backdoor attacks via word substitution. In Proceedings of ACL. find
  40. Keita Kurita, Paul Michel, and Graham Neubig. 2020. Weight poisoning attacks on pre-trained models. In Proceedings of ACL. find
  41. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8). find
  42. Guillaume Lample, Sandeep Subramanian, Eric Smith, Ludovic Denoyer, Marc’Aurelio Ranzato, and Y-Lan Boureau. 2018. Multiple-attribute text rewriting. In Proceedings of ICLR. find
  43. Sudha Rao and Joel Tetreault. 2018. Dear sir or madam, may i introduce the gyafc dataset: Corpus, benchmarks and metrics for formality style transfer. In Proceedings of NAACL-HLT. find our card
  44. Sravana Reddy and Kevin Knight. 2016. Obfuscating gender in social media writing. In Proceedings of the First Workshop on NLP and Computational Social Science. find
  45. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of EMNLP-IJCNLP. find
  46. Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of ACL. find our card
  47. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. In Proceedings of the 5th Workshop on Energy Efficient Machine Learning and Cognitive Computing. find
  48. Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style Transfer from Non-Parallel Text by Cross-Alignment. In Proceedings of NeurIPS. find our card
  49. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of EMNLP. find
  50. Murat Cihan Sorkun and Taner Toraman. 2017. Fraud detection on financial statements using data mining techniques. International Journal of Intelligent Systems and Applications in Engineering, 5(3):132– 134. find
  51. Akhilesh Sudhakar, Bhargav Upadhyay, and Arjun Maheswaran. 2019. “Transforming” Delete, Retrieve, Generate Approach for Controlled Text Style Transfer. In Proceedings of EMNLP-IJCNLP. find our card
  52. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In Proceedings of ICLR. find our card
  53. Boxin Wang, Hengzhi Pei, Boyuan Pan, Qian Chen, Shuohang Wang, and Bo Li. 2020a. T3: Tree-autoencoder constrained adversarial text generation for targeted attack. In Proceedings of EMNLP. find
  54. Han Xu, Yao Ma, Hao-Chen Liu, Debayan Deb, Hui Liu, Ji-Liang Tang, and K. Anil Jain. 2020. Adversarial attacks and defenses in images, graphs and text: A review. International Journal of Automation and Computing, 17(2):151–178. find
  55. Jingjing Xu, Xu Sun, Qi Zeng, Xiaodong Zhang, Xuancheng Ren, Houfeng Wang, and Wenjie Li. 2018. Unpaired sentiment-to-sentiment translation: A cycled reinforcement learning approach. In Proceedings of ACL. find our card
  56. Zichao Yang, Zhiting Hu, Chris Dyer, Eric P. Xing, and Taylor Berg-Kirkpatrick. 2018. Unsupervised Text Style Transfer using Language Models as Discriminators. In Proceedings of NeurIPS. find
  57. Yuan Zang, Bairu Hou, Fanchao Qi, Zhiyuan Liu, Xiaojun Meng, and Maosong Sun. 2020a. Learning to attack: Towards textual adversarial attacking in real-world situations. arXiv preprint arXiv:2009.09192. ↗
  58. Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. 2020b. Word-level textual adversarial attacking as combinatorial optimization. In Proceedings of ACL. find our card
  59. Wei Emma Zhang, Quan Z Sheng, Ahoud Alhazmi, and Chenliang Li. 2020. Adversarial attacks on deep-learning models in natural language processing: A survey. ACM Transactions on Intelligent Systems and Technology (TIST), 11(3):1–41. find
  60. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Proceedings of NeurIPS. find our card
  61. Ye Zhang, Nan Ding, and Radu Soricut. 2018. Shaped: Shared-private encoder-decoder for text style adaptation. In Proceedings of NAACL-HLT. find
  62. Zhengli Zhao, Dheeru Dua, and Sameer Singh. 2018. Generating natural adversarial examples. In Proceedings of ICLR. find
  63. Ganbin Zhou, Ping Luo, Rongyu Cao, Fen Lin, Bo Chen, and Qing He. 2017. Mechanism-aware neural machine for dialogue response generation. In Proceedings of AAAI. find
  64. Tianlu Wang, Xuezhi Wang, Yao Qin, Ben Packer, Kang Li, Jilin Chen, Alex Beutel, and Ed Chi. 2020b. Cat-gen: Improving robustness in nlp models via controlled adversarial text generation. In Proceedings of EMNLP. find
  65. Thomas Wolf, Julien Chaumond, Lysandre Debut, Victor Sanh, Clement Delangue, Anthony Moi, Pier-ric Cistac, Morgan Funtowicz, Joe Davison, Sam Shleifer, et al. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of EMNLP. find our card