Elamur Research⌕
☰
home / papers / 2021 / Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger

Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger

Summary

The paper addresses backdoor attacks on NLP models. A model trained on poisoned data behaves normally on ordinary inputs but returns an attacker-chosen label whenever a hidden trigger is present. Earlier textual attacks inserted fixed rare words or sentences as triggers. These insertions damage fluency and are easily filtered out by defenses. The authors instead use a syntactic structure as the trigger. A syntactically controlled paraphrase model (SCPN) rewrites a subset of training samples into a chosen syntactic template and relabels them with the target class. At test time, the attacker activates the backdoor by paraphrasing inputs the same way. Experiments show that this attack reaches success rates comparable to insertion-based baselines, often near 100%. The poisoned sentences remain natural and fluent, so they are much harder to detect. The attack also stays effective against word-level and sentence-level defenses that neutralise the baselines. The work shows that textual backdoors can be far stealthier than previously assumed, which motivates research on stronger defenses. Code and data are publicly released.

Key points

  • The trigger is the syntactic template S(SBAR)(,)(NP)(VP)(.), and poisoned samples are produced with SCPN paraphrasing. Baselines insert rare tokens such as "bb" (Kurita et al.) or fixed sentences (Dai et al.).
  • Without defenses, all attacks exceed 90% success. Under the ONION outlier-word defense, baseline success drops by over 40% on average, while the syntactic attack loses under 1.2%.
  • The authors tested two sentence-level defenses on SST-2:
    • Back-translation through Chinese with Google Translation only weakly affects the syntactic attack but mitigates the baselines.
    • SCPN paraphrasing into the common template S(NP)(VP)(.) does block it, but both defenses noticeably reduce clean accuracy (CACC).
  • Victim models include BERT (base version) under the pre-train/fine-tune setting. Code and data are available at github.com/thunlp/HiddenKiller.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

86 papers of our library cite it (of the 140884 whose bibliographies we read); 5 of them have a page here, 81 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.acl-long.37

@inproceedings{qi2021hidden,
  title = {{Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger}},
  author = {Qi, Fanchao and Li, Mukai and Chen, Yangyi and Zhang, Zhengyan and Liu, Zhiyuan and Wang, Yasheng and Sun, Maosong},
  booktitle = {Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.37},
  url = {https://doi.org/10.18653/v1/2021.acl-long.37}
}

References

45 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR. find our card
  2. Alexy Bhowmick and Shyamanta M Hazarika. 2018. E-mail spam filtering: a review of techniques and trends. In Advances in Electronics, Communication and Computing, pages 583–590. Springer. find
  3. Battista Biggio, Blaine Nelson, and Pavel Laskov. 2012. find
  4. Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Proceedings of NeurIPS. find
  5. Chuanshuai Chen and Jiazhu Dai. 2020. Mitigating backdoor attacks in lstm-based text classification systems by backdoor keyword identification. arXiv preprint arXiv:2007.12070. — [arXiv:2007.12070](https://arxiv.org/abs/2007.12070) find
  6. Xiaoyi Chen, Ahmed Salem, Michael Backes, Shiqing Ma, and Yang Zhang. 2020. Badnl: Backdoor attacks against nlp models. arXiv preprint arXiv:2006.01043. — [arXiv:2006.01043](https://arxiv.org/abs/2006.01043) find
  7. Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. 2017. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526. — [arXiv:1712.05526](https://arxiv.org/abs/1712.05526) find
  8. Jiazhu Dai, Chuanshuai Chen, and Yufeng Li. 2019. A backdoor attack against lstm-based text classification systems. IEEE Access, 7:138872–138878. find
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT. find
  10. Min Du, Ruoxi Jia, and Dawn Song. 2020. Robust anomaly detection and backdoor attack detection via differential privacy. In Proceedings of ICLR. find
  11. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In Proceedings of ICLR. find our card
  12. Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. 2017. BadNets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733. find
  13. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. find
  14. Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial example generation with syntactically controlled paraphrase networks. In Proceedings of NAACL-HLT. find our card
  15. Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of ICLR. find
  16. Soheil Kolouri, Aniruddha Saha, Hamed Pirsiavash, and Heiko Hoffmann. 2020. Universal litmus patterns: Revealing backdoor attacks in cnns. In Proceedings of CVPR. find
  17. Keita Kurita, Paul Michel, and Graham Neubig. 2020. Weight poisoning attacks on pre-trained models. In Proceedings of ACL. find
  18. But some new kinds of backdoor attacks or other security threats will always appear even with the trusted third party. It is a dynamic and never-ending game. find
  19. Yiming Li, Baoyuan Wu, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. 2020. Backdoor learning: A survey. arXiv preprint arXiv:2007.08745. — [arXiv:2007.08745](https://arxiv.org/abs/2007.08745) find
  20. Cong Liao, Haoti Zhong, Anna Squicciarini, Sencun Zhu, and David Miller. 2018. Backdoor embedding in convolutional neural network models via invisible perturbation. arXiv preprint arXiv:1808.10307. — [arXiv:1808.10307](https://arxiv.org/abs/1808.10307) find
  21. Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018a. Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions, and Defenses. find
  22. Murat Cihan Sorkun and Taner Toraman. 2017. Fraud detection on financial statements using data mining techniques. International Journal of Intelligent Systems and Applications in Engineering, 5(3):132– 134. find
  23. Jacob Steinhardt, Pang Wei W Koh, and Percy S Liang. 2017. Certified defenses for data poisoning attacks. In Proceedings of NeurIPS. find
  24. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In Proceedings of ICLR. find our card
  25. Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. 2018b. Trojaning Attack on Neural Networks. In Proceedings of NDSS. find
  26. Ann Taylor, Mitchell Marcus, and Beatrice Santorini. 2003. The penn treebank: an overview. In Treebanks, pages 5–22. Springer. find
  27. Yunfei Liu, Xingjun Ma, James Bailey, and Feng Lu. 2020. Reflection backdoor: A natural backdoor attack on deep neural networks. In Proceedings of ECCV. find
  28. Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. 2019. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy. IEEE. find
  29. Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David Mc-Closky. 2014. The stanford corenlp natural language processing toolkit. In Proceedings of ACL. find our card
  30. Fanchao Qi, Yangyi Chen, Mukai Li, Zhiyuan Liu, and Maosong Sun. 2020. Onion: A simple and effective defense against textual backdoor attacks. arXiv preprint arXiv:2011.10369. find our card
  31. Fanchao Qi, Yuan Yao, Sophia Xu, Zhiyuan Liu, and Maosong Sun. 2021. Turn the combination lock: Learnable textual backdoor attacks via word substitution. In Proceedings of ACL-IJCNLP. find
  32. Ximing Qiao, Yukun Yang, and Hai Li. 2019. Defending neural backdoors via generative distribution modeling. In Proceedings of NeurIPS. find
  33. Thomas Wolf, Julien Chaumond, Lysandre Debut, Victor Sanh, Clement Delangue, Anthony Moi, Pier-ric Cistac, Morgan Funtowicz, Joe Davison, Sam Shleifer, et al. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of EMNLP. find our card
  34. Han Xu, Yao Ma, Hao-Chen Liu, Debayan Deb, Hui Liu, Ji-Liang Tang, and K. Anil Jain. 2020. Adversarial attacks and defenses in images, graphs and text: A review. International Journal of Automation and Computing, 17(2):151–178. find
  35. Chaofei Yang, Qing Wu, Hai Li, and Yiran Chen. 2017. Generative poisoning attack method against neural networks. arXiv preprint arXiv:1703.01340. — [arXiv:1703.01340](https://arxiv.org/abs/1703.01340) find
  36. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8). find
  37. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. Predicting the type and target of offensive posts in social media. In Proceedings of NAACL-HLT. find our card
  38. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Semantically equivalent adversarial rules for debugging nlp models. In Proceedings ACL. find
  39. Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. 2020. Word-level textual adversarial attacking as combinatorial optimization. In Proceedings of ACL. find our card
  40. Aniruddha Saha, Akshayvarun Subramanya, and Hamed Pirsiavash. 2020. Hidden trigger backdoor attacks. In Proceedings of AAAI. find
  41. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Proceedings of NeurIPS. find our card
  42. Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of ACL. find
  43. Xinyang Zhang, Zheng Zhang, and Ting Wang. 2020. Trojaning language models for fun and profit. arXiv preprint arXiv:2008.00312. — [arXiv:2008.00312](https://arxiv.org/abs/2008.00312) find
  44. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of EMNLP. find
  45. Shihao Zhao, Xingjun Ma, Xiang Zheng, James Bailey, Jingjing Chen, and Yu-Gang Jiang. 2020. Clean-label backdoor attacks on video recognition models. In Proceedings of CVPR. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/qi-2021-hidden-killer-invisible-textual-backdoor/