Elamur Research
☰
home / papers / 2018 / HotFlip: White-Box Adversarial Examples for Text Classification

HotFlip: White-Box Adversarial Examples for Text Classification

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/P18-2006.pdf

Summary

Character-level neural text classifiers can be fooled by tiny, nearly invisible edits, yet earlier NLP attacks mostly used black-box heuristics or random noise. This paper introduces HotFlip, a white-box attack. It treats each character edit as a direction in one-hot input space. Substitutions are handled directly, and insertions and deletions are expressed as chains of substitutions. One forward pass and one backward pass give first-order estimates of how much each possible edit would raise the loss. A beam search then combines several edits. On AG's news with a CharCNN-LSTM classifier, the attack succeeded on more than 90% of attacked instances even when it had to cause a misclassification at 0.9 confidence. At 0.5 confidence it changed only about 4% of characters, and crowd workers seldom saw any change in meaning. Because the examples are cheap to generate, the authors also use them for adversarial training. This lowers clean test error and sharply reduces attack success. A constrained word-level variant, tested on a sentiment CNN trained on SST, works but finds few valid examples. The paper offers an early, efficient gradient-based tool for probing and hardening NLP models.

Key points

  • Data and model: The main experiments use AG's news, with 120,000 training and 7,600 test texts in four classes. The target is a CharCNN-LSTM with two stacked LSTM layers of 500 units each. The attacker uses beam width 10 and may change at most 10% of characters.
  • Cost: Beam search needs O(br) forward and backward passes, where b is the beam width and r is the edit budget. A naive approach that measures the loss of every possible change needs O(brL|V|) queries. Flips make up about 80% of the chosen operations, and deletions are used twice as often as insertions.
  • Adversarial training (Table 2): Training with HotFlip examples lowers clean error from 8.27% to 7.65%. It also lowers attack success from 98.16% to 69.32%. This beats the embedding-noise method of Miyato et al. (2017) and training with black-box examples.
  • Human check: 600 Mechanical Turk items were annotated, half of them adversarial. Median human accuracy fell only from 87.49% to 85.71%.
  • Word level: Against Kim's CNN on SST, the attack is restricted to swaps with cosine similarity above 0.8 and the same part of speech, and it may not replace stop-words. Under these rules it produced only 41 adversarial examples, about 2% of the correctly classified test instances.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

326 papers of our library cite it (of the 140673 whose bibliographies we read); 20 of them have a page here, 306 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p18-2006

@inproceedings{ebrahimi2018hotflip,
  title = {{HotFlip: White-Box Adversarial Examples for Text Classification}},
  author = {Ebrahimi, Javid and Rao, Anyi and Lowd, Daniel and Dou, Dejing},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-2006},
  url = {https://doi.org/10.18653/v1/p18-2006}
}

References

18 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of EMNLP, pages 1631–1642. find
  2. Yonatan Belinkov and Yonatan Bisk. 2018. Synthetic and natural noise both break neural machine translation. In Proceedings of ICLR. find
  3. Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In Security and Privacy (SP), 2017 IEEE Symposium on, pages 39–57. IEEE. find
  4. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In Proceedings of ICLR. find our card
  5. Alexis Conneau, Holger Schwenk, Loı̈c Barrault, and Yann Lecun. 2017. Very deep convolutional networks for text classification. In Proceedings of EACL, volume 1, pages 1107–1116. find
  6. Zhengli Zhao, Dheeru Dua, and Sameer Singh. 2018. Generating natural adversarial examples. In Proceedings of ICLR. find
  7. Nilesh Dalvi, Pedro Domingos, Sumit Sanghai, Deepak Verma, et al. 2004. Adversarial classification. In Proceedings of KDD, pages 99–108. find
  8. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In Proceedings of ICLR. find our card
  9. Hossein Hosseini, Sreeram Kannan, Baosen Zhang, and Radha Poovendran. 2017. Deceiving google’s perspective api built for detecting toxic comments. arXiv preprint arXiv:1702.08138. — [arXiv:1702.08138](https://arxiv.org/abs/1702.08138) find
  10. Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial example generation with syntactically controlled paraphrase networks. arXiv preprint arXiv:1804.06059. find our card
  11. Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Proceedings of EMNLP. find our card
  12. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of EMNLP. find our card
  13. Yoon Kim, Yacine Jernite, David Sontag, and Alexander M Rush. 2016. Character-aware neural language models. Proceedings of AAAI. find our card
  14. Daniel Lowd and Christopher Meek. 2005. Adversarial learning. In Proceedings of KDD, pages 641–647. find
  15. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In Proceedings of ICLR. find our card
  16. Takeru Miyato, Andrew M Dai, and Ian Goodfellow. 2017. Adversarial training methods for semi-supervised text classification. In Proceedings of ICLR. find our card
  17. Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. 2016. The limitations of deep learning in adversarial settings. In Security and Privacy (EuroS&P), 2016 IEEE European Symposium on, pages 372– 387. IEEE. find
  18. Graham Ernest Rawlinson. 1976. The Significance of Letter Position in Word Recognition. Ph.D. thesis, University of Nottingham. find