Elamur Research
☰
home / papers / 2020 / BERT-ATTACK: Adversarial Attack Against BERT Using BERT

BERT-ATTACK: Adversarial Attack Against BERT Using BERT

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2020.emnlp-main.500.pdf

Summary

Generating adversarial examples for text is harder than for images. Language is discrete, so gradient tricks do not transfer directly. Earlier attacks also relied on hand-built rules, such as synonym lists, POS checks and embedding similarity, which often harm fluency or meaning. Li et al. propose BERT-Attack, a black-box method that uses a pre-trained masked language model (BERT) to fool fine-tuned BERT classifiers and other neural models.

The attack works in two stages:

  1. Ranking words. Each word is scored by how much the target model's logit for the correct label falls when that word is replaced with [MASK].
  2. Replacing words. The highest-ranked words are swapped one at a time with context-aware candidates from BERT's top-K predictions, until the target's prediction changes. Words split into several sub-words are also handled.

Because candidates come from one forward pass, no separate scoring model is needed, and the method queries the target model fewer times. The authors report higher attack success and lower perturbation rates than prior state-of-the-art attacks. Human evaluation found the outputs fluent and meaning-preserving. This matters for robustness testing of strong pre-trained models and for producing data for adversarial training.

Key points

  • Word importance. A word's importance is the drop in the gold-label logit when that word is masked. Only a small share of the top-ranked words is perturbed. Ranking the most important words first worked clearly better than random order or least-important-first.
  • Runtime on IMDB. BERT-Attack took about 16.0 s per sample with BPE and 14.2 s without it. TextFooler (Jin et al., 2019) took 42.4 s and the genetic algorithm (GA) of Alzantot et al. (2018) took 2582.0 s.
  • Sub-word attacks. Allowing replacements of words that BPE splits into several pieces lowered the target model's accuracy under attack and also reduced the share of perturbed words.
  • Transferability. Adversarial samples crafted against one target model (LSTM, ESIM, BERT-base, BERT-large) transferred well between models on MNLI. They transferred less well on text classification tasks such as IMDB and Yelp.
  • Adversarial training and limits. Fine-tuning on generated adversarial samples made MNLI models harder to attack while keeping clean accuracy close to normal. For NLI, the authors attack premises rather than hypotheses, since context-based replacement works poorly on very short sentences.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

266 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 256 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.emnlp-main.500

@inproceedings{li2020bert,
  title = {{BERT-ATTACK: Adversarial Attack Against BERT Using BERT}},
  author = {Li, Linyang and Ma, Ruotian and Guo, Qipeng and Xue, Xiangyang and Qiu, Xipeng},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.500},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.500}
}

References

26 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani B. Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. CoRR, abs/1804.07998. find our card
  2. Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326. find
  3. Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, et al. 2018. Universal sentence encoder. arXiv preprint arXiv:1803.11175. find our card
  4. Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopadhyay. 2018. Adversarial attacks and defences: A survey. arXiv preprint arXiv:1810.00069. — [arXiv:1810.00069](https://arxiv.org/abs/1810.00069) find
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805. find
  6. Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. 2018. Black-box generation of adversarial text sequences to evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops (SPW), pages 50–56. find our card
  7. Max Glockner, Vered Shwartz, and Yoav Goldberg. 2018. Breaking nli systems with sentences that require simple lexical inferences. arXiv preprint arXiv:1805.02266. find our card
  8. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. find our card
  9. Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. arXiv preprint arXiv:1707.07328. find our card
  10. Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2019. Is BERT really robust? natural language attack on text classification and entailment. CoRR, abs/1907.11932. find our card
  11. Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2016. Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533. — [arXiv:1607.02533](https://arxiv.org/abs/1607.02533) find
  12. Qi Lei, Lingfei Wu, Pin-Yu Chen, Alexandros G Dimakis, Inderjit S Dhillon, and Michael Witbrock. 2019. Discrete adversarial attacks and submodular optimization with applications to text classification. Systems and Machine Learning (SysML). find
  13. Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. 2018. Textbugger: Generating adversarial text against real-world applications. arXiv preprint arXiv:1812.05271. find our card
  14. Bin Liang, Hongcheng Li, Miaoqiang Su, Pan Bian, Xirong Li, and Wenchang Shi. 2017. Deep text classification can be fooled. arXiv preprint arXiv:1704.08006. — [arXiv:1704.08006](https://arxiv.org/abs/1704.08006) find
  15. Takeru Miyato, Shin ichi Maeda, Masanori Koyama, and Shin Ishii. 2017. Virtual adversarial training: A regularization method for supervised and semi-supervised learning. volume 41, pages 1979–1993. find
  16. Danish Pruthi, Bhuwan Dhingra, and Zachary C Lipton. 2019. Combating adversarial misspellings with robust word recognition. arXiv preprint arXiv:1905.11268. find our card
  17. Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1085–1097. find our card
  18. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing NLP. Empirical Methods in Natural Language Processing. find
  19. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1112–1122. find
  20. Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, and Michael I Jordan. 2018. Greedy attack and gumbel attack: Generating adversarial examples for discrete data. arXiv preprint arXiv:1805.12316. — [arXiv:1805.12316](https://arxiv.org/abs/1805.12316) find
  21. Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. 2020. Word-level textual adversarial attacking as combinatorial optimization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6066–6080. find our card
  22. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in neural information processing systems, pages 649–657. find our card
  23. Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, and Ming Zhou. 2019. BERT-based lexical substitution. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3368–3373, Florence, Italy. Association for Computational Linguistics. find
  24. Nikola Mrkšić, Diarmuid O Séaghdha, Blaise Thomson, Milica Gašić, Lina Rojas-Barahona, Pei-Hao Su, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016. Counter-fitting word vectors to linguistic constraints. arXiv preprint arXiv:1603.00892. find
  25. Anh Nguyen, Jason Yosinski, and Jeff Clune. 2015. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 427–436. find
  26. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the conference on empirical methods in natural language processing, pages 1532–1543. find