Elamur Research
☰
home / papers / 2019 / Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency

Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/P19-1103.pdf

Summary

Attacking text classifiers is harder than attacking image models: words are discrete, and small edits easily break spelling, grammar or meaning. The authors propose PWWS (Probability Weighted Word Saliency), a greedy black-box attack. It replaces words with WordNet synonyms, and named entities with same-type entities that are frequent in other classes. For each position, PWWS picks the candidate that most lowers the probability of the true class. It then orders positions by weighting that drop with word saliency, which measures how much the prediction shifts when the word becomes an unknown token. The method was tested on IMDB, AG's News and Yahoo! Answers against word-based CNN, char-based CNN and LSTM classifiers. PWWS degraded accuracy more than earlier attacks while changing very few words. Human raters could hardly distinguish perturbed texts from originals. The examples also transferred to other models, and training on them made a CNN more robust. The study shows that neural NLP classifiers are vulnerable even when attacks must preserve lexical, grammatical and semantic correctness.

Key points

  • PWWS lowered classifier accuracy by as much as 84.03% and outperformed prior text attack methods.

  • On IMDB, PWWS changed only 3.81% of words, far fewer than competing methods.

  • Six Amazon Mechanical Turk workers judged three groups of 100 clean texts and their adversarial versions:

    • IMDB with word-based CNN
    • IMDB with Bi-directional LSTM
    • AG's News with char-based CNN

    Their classification accuracy stayed close to that on clean data, and their 1–5 "machine-modified" scores rose only slightly.

  • Transferability was tested by attacking word-based CNN-1 and evaluating on three other models: CNN-2 (an extra dense layer), CNN-3 (GloVe embeddings) and a Bi-directional LSTM. When all methods replaced the same 10% of words, PWWS transferred clearly better than the alternatives.

  • Adversarial training with 4000 PWWS examples from IMDB improved a word-based CNN's accuracy on the clean test set and its robustness to several attack methods. Gains grew as more adversarial examples were added.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

249 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 240 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1103

@inproceedings{ren2019generating,
  title = {{Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency}},
  author = {Ren, Shuhuai and Deng, Yihe and He, Kun and Che, Wanxiang},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1103},
  url = {https://doi.org/10.18653/v1/p19-1103}
}

References

19 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani B. Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pag… find our card
  2. Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. 2018. Boosting adversarial attacks with momentum. In The Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018. Hotflip: White-box adversarial examples for text classification. In Proceedings of the 56th An… find our card
  3. Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. 2018. Black-box generation of adversarial text sequences to evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops, SP Workshops 2018, San Francisco, CA, USA, May 24, 2018, pages 50–56. find our card
  4. Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In International Conference on Learning Representations. find our card
  5. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1746–1751. find our card
  6. Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2016. Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533. — [arXiv:1607.02533](https://arxiv.org/abs/1607.02533) find
  7. Jiwei Li, Xinlei Chen, Eduard H. Hovy, and Dan Jurafsky. 2016a. Visualizing and understanding neural models in NLP. In NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-1… find our card
  8. Jiwei Li, Will Monroe, and Dan Jurafsky. 2016b. Understanding neural networks through representation erasure. CoRR, abs/1612.08220. — [arXiv:1612.08220](https://arxiv.org/abs/1612.08220) find
  9. Bin Liang, Hongcheng Li, Miaoqiang Su, Pan Bian, Xirong Li, and Wenchang Shi. 2018. Deep text classification can be fooled. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden., pages 4208–4215. find
  10. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. CoRR, abs/1312.6199. find our card
  11. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, 19-24 June, 201… find our card
  12. Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. 2018. Ensemble adversarial training: Attacks and defenses. In International Conference on Learning Representations. find
  13. Anh Mai Nguyen, Jason Yosinski, and Jeff Clune. 2015. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015, pages 427–436. find
  14. Nicolas Papernot, Patrick D. McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. 2016a. The limitations of deep learning in adversarial settings. In IEEE European Symposium on Security and Privacy, EuroS&P 2016, Saarbrücken, Germany, March 21-24, 2016, pages 372–387. find
  15. Nicolas Papernot, Patrick D. McDaniel, Ananthram Swami, and Richard E. Harang. 2016b. Crafting adversarial input sequences for recurrent neural networks. In 2016 IEEE Military Communications Conference, MILCOM 2016, Baltimore, MD, USA, November 1-3, 2016, pages 49–54. find
  16. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group… find
  17. Eric Wong and J. Zico Kolter. 2018. Provable defenses against adversarial examples via the convex outer adversarial polytope. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, pages 5283–5292. find
  18. Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, and Michael I. Jordan. 2018. Greedy attack and gumbel attack: Generating adversarial examples for discrete data. CoRR, abs/1805.12316. — [arXiv:1805.12316](https://arxiv.org/abs/1805.12316) find
  19. Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 649–657. find our card