Elamur Research
☰
home / papers / 2018 / Generating Natural Language Adversarial Examples

Generating Natural Language Adversarial Examples

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1804.07998

Summary

Adversarial attacks are well studied for images, but text is harder. Words are discrete, so input gradients are unavailable, and every edit is visible to a reader. Alzantot et al. propose a black-box attack that needs only the victim model's output probabilities. It replaces individual words with synonyms, guided by a genetic algorithm. Candidate substitutes come from counter-fitted GloVe neighbours. The Google 1 billion words language model then filters them for fit with the surrounding context. From the survivors, the attack keeps the word that most increases the target class probability. A population of perturbed texts evolves through fitness-weighted selection, crossover and further mutation. The authors tested the attack on an LSTM sentiment classifier trained on IMDB and an entailment model trained on SNLI, where only the hypothesis could be edited. It flipped 97% and 70% of correctly classified inputs, respectively, and outperformed a greedy baseline. Human raters mostly still assigned the original sentiment. Retraining with adversarial examples gave no protection, which shows that NLP models need stronger defences.

Key points

  • Search settings: population S = 60, at most G = 20 generations, N = 8 embedding neighbours within δ = 0.5, and the top K = 4 by language-model score. Edits were capped at 20% of words for IMDB and 25% for SNLI. Articles and prepositions were never replaced.
  • Success rates and runtime: 97% on IMDB and 70% on SNLI. The authors attribute the lower SNLI rate to short hypotheses, which average about 9 words, versus 229 words for IMDB reviews (truncated to 100). The greedy Perturb baseline did worse on IMDB and achieved no successes on SNLI under the change cap. On a single TitanX, a successful attack took 43.5 s on average for IMDB and 5 s for SNLI.
  • User study (20 volunteers): for 100 adversarial reviews, 92.3% of sentiment labels matched the original. Raters gave original/adversarial pairs a mean similarity score of 2.23 ± 0.25, where 1 indicated very similar and 4 indicated very different.
  • Adversarial training: 1000 attack examples from the IMDB training set were added and the model was retrained from scratch. Test-set robustness did not improve, even though the model classified the injected examples almost perfectly.
  • Code: the attack is released at github.com/nesl/nlp_adversarial_examples.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

315 papers of our library cite it (of the 140673 whose bibliographies we read); 22 of them have a page here, 293 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1804.07998 · arXiv 1804.07998

@misc{alzantot2018generating,
  title = {{Generating Natural Language Adversarial Examples}},
  author = {Alzantot, Moustafa and Sharma, Yash and Elgohary, Ahmed and Ho, Bo-Jhang and Srivastava, Mani and Chang, Kai-Wei},
  year = {2018},
  doi = {10.48550/arxiv.1804.07998},
  eprint = {1804.07998},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1804.07998}
}

References

26 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Alzantot, Y. Sharma, S. Chakraborty, and M. Srivastava. 2018. Genattack: Practical black-box attacks with gradient-free optimization. arXiv preprint arXiv:1805.11090. — [arXiv:1805.11090](https://arxiv.org/abs/1805.11090) find
  2. Edward J Anderson and Michael C Ferris. 1994. Genetic algorithms for combinatorial optimization: the assemble line balancing problem. ORSA Journal on Computing, 6(2):161–173. find
  3. Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326. find
  4. N. Carlini and D. Wagner. 2017. Towards evaluating the robustness of neural networks. arXiv preprint arXiv:1608.04644. — [arXiv:1608.04644](https://arxiv.org/abs/1608.04644) find
  5. Nicholas Carlini and David Wagner. 2018. Audio adversarial examples: Targeted attacks on speech-to-text. arXiv preprint arXiv:1801.01944. — [arXiv:1801.01944](https://arxiv.org/abs/1801.01944) find
  6. Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. 2013. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005. find our card
  7. P. Chen, H Zhang, Y. Sharma, J. Yi, and C. Hseih. 2017a. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. arXiv preprint arXiv:1708.03999. — [arXiv:1708.03999](https://arxiv.org/abs/1708.03999) find
  8. P. Y. Chen, Y. Sharma, H. Zhang, J. Yi, and C. Hsieh. 2017b. Ead: Elastic-net attacks to deep neural networks via adversarial examples. arXiv preprint arXiv:1709.0414. — [arXiv:1709.0414](https://arxiv.org/abs/1709.0414) find
  9. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017c. Enhanced lstm for natural language inference. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1657–1668. find
  10. J. Ebrahimi, A. Rao, D. Lowd, and D. Dou. 2018. Hotflip: White-box adversarial examples for text classification. ACL’18; arXiv preprint arXiv:1712.06751. find our card
  11. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. find our card
  12. Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial example generation with syntactically controlled paraphrase networks. In Proceedings of NAACL. find our card
  13. Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. arXiv preprint arXiv:1707.07328. find our card
  14. V. Kuleshov, S. Thakoor, T. Lau, and S. Ermon. 2018. Adversarial examples for natural language classification problems. OpenReview submission OpenReview:r1QZ3zbAZ. find
  15. A. Kurakin, I. Goodfellow, and S. Bengio. 2016. Adversarial machine learning at scale. ICLR’17; arXiv preprint arXiv:1611.01236. — [arXiv:1611.01236](https://arxiv.org/abs/1611.01236) find
  16. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142–150, Portland, Oregon… find our card
  17. A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083. find our card
  18. Nikola Mrkšić, Diarmuid O Séaghdha, Blaise Thomson, Milica Gašić, Lina Rojas-Barahona, Pei-Hao Su, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016. Counter-fitting word vectors to linguistic constraints. arXiv preprint arXiv:1603.00892. find
  19. Heinz Mühlenbein. 1989. Parallel genetic algorithms, population genetics and combinatorial optimization. In Workshop on Parallel Processing: Logic, Organization, and Technology, pages 398–406. Springer. find
  20. N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. Celik, and A. Swami. 2016a. Practical black-box attacks against machine learning. arXiv preprint arXiv:1602.02697. — [arXiv:1602.02697](https://arxiv.org/abs/1602.02697) find
  21. N. Papernot, P. McDaniel, A. Swami, and R. Harang. 2016b. Crafting adversarial input sequences for recurrent neural networks. arXiv preprint arXiv:1604.08275. — [arXiv:1604.08275](https://arxiv.org/abs/1604.08275) find
  22. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. find
  23. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Semantically equivalent adversarial rules for debugging nlp models. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 856–865. find
  24. Y. Sharma and P. Y. Chen. 2017. Attacking the madry defense model with l1-based adversarial examples. arXiv preprint arXiv:1710.10733. — [arXiv:1710.10733](https://arxiv.org/abs/1710.10733) find
  25. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199. find our card
  26. Zhengli Zhao, Dheeru Dua, and Sameer Singh. 2018. Generating natural adversarial examples. arXiv preprint arXiv:1710.11342. — [arXiv:1710.11342](https://arxiv.org/abs/1710.11342) find