Elamur Research⌕
☰
home / papers / 2020 / Reevaluating Adversarial Examples in Natural Language

Reevaluating Adversarial Examples in Natural Language

Paper · 2020 · John Morris, Eli Lifland, Jack Lanchantin et al.

Summary

Research on adversarial attacks against NLP models has no agreed notion of what makes an attack successful. This makes published attacks hard to compare and limits their usefulness for analysing models. The authors propose a common framework. A valid textual adversarial example must change the model's output while satisfying four kinds of linguistic constraint: semantics, grammaticality, overlap and non-suspicion. They classify earlier attacks by these constraints and suggest human and automatic ways to check each one.

Applying these checks to two leading synonym-substitution attacks, GENETICATTACK and TEXTFOOLER, shows that many of their perturbations change meaning, look suspicious or introduce grammar errors. Human studies were used to set stricter thresholds for word-embedding and sentence-encoder similarity, and a grammar checker was added. The result is TFADJUSTED, which yields better examples but succeeds far less often.

Using TextAttack, the authors kept the constraints fixed and compared only the search strategies. This revealed that TEXTFOOLER's reported advantage came from looser constraints, not from a better search. The work argues that new search methods should be evaluated separately from changes to constraints.

Key points

  • Error rates: About 38% of perturbations from the evaluated attacks introduce grammatical errors. A grammar checker flags 39% more errors in perturbed texts than in the originals.
  • TFADJUSTED trade-off: It keeps TEXTFOOLER's greedy search but tightens the similarity constraints and adds grammar validation. The attack success rate drops from above 80% to below 20%, a fall of more than 70 percentage points. Its examples reach semantic-preservation Likert scores above 4.
  • Most restrictive constraint: The word-embedding distance constraint reduced attack success the most, followed by the sentence-encoder constraint.
  • Search comparison (BERT fine-tuned on MR, 1000 samples): GENETICATTACK's genetic algorithm beats TEXTFOOLER's greedy search under both constraint sets.
    • TFADJUSTED constraints: 12.0% vs 10.6% success.
    • TEXTFOOLER constraints: 95.0% vs 91.1% success.
    • The genetic search needs more than 40x as many model queries.
  • Adversarial training: Training on TEXTFOOLER's examples lowered model accuracy, while training on TFADJUSTED's examples did not. Code and data are released.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of VirginiaCharlottesville · United States18 works · 8,786 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

Cite

Cite as John Morris, Eli Lifland, Jack Lanchantin, Yangfeng Ji and Yanjun Qi. 2020. Reevaluating Adversarial Examples in Natural Language. Findings of the Association for Computational Linguistics: EMNLP 2020.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.findings-emnlp.341

@inproceedings{morris2020reevaluating,
  title = {{Reevaluating Adversarial Examples in Natural Language}},
  author = {John Morris, Eli Lifland, Jack Lanchantin, Yangfeng Ji and Yanjun Qi},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.341},
  url = {https://doi.org/10.18653/v1/2020.findings-emnlp.341}
}

References

39 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. arXiv preprint arXiv:1804.07998. our card
  2. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. CoRR, abs/1508.05326. find
  3. Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. 2017. SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 1–14, Vancouver, Canada.… our card
  4. Daniel Cer, Yinfei Yang, Sheng yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2018. Universal sentence encoder. ArXiv, abs/1803.11175. our card
  5. Minhao Cheng, Jinfeng Yi, Huan Zhang, Pin-Yu Chen, and Cho-Jui Hsieh. 2018. Seq2sick: Evaluating the robustness of sequence-to-sequence models with adversarial examples. arXiv preprint arXiv:1803.01128. — [arXiv:1803.01128](https://arxiv.org/abs/1803.01128) find
  6. Michael Denkowski and Alon Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 376–380, Baltimore, Maryland, USA. Association for Computational Linguistics. our card
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. our card
  8. Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2017. Hotflip: White-box adversarial examples for text classification. arXiv preprint arXiv:1712.06751. our card
  9. Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. 2018. Black-box generation of adversarial text sequences to evade deep learning classifiers. In IEEE Security and Privacy Workshops (SPW). our card
  10. Justin Gilmer, Ryan P. Adams, Ian J. Goodfellow, David Andersen, and George E. Dahl. 2018. Motivating the rules of the game for adversarial example research. CoRR, abs/1807.06732. — [arXiv:1807.06732](https://arxiv.org/abs/1807.06732) find
  11. Ian Goodfellow, Nicolas Papernot, Sandy Huang, Rocky Duan, Pieter Abbeel, and Jack Clark. Attacking machine learning with adversarial examples [online]. 2017. find
  12. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. our card
  13. Po-Sen Huang, Robert Stanforth, Johannes Welbl, Chris Dyer, Dani Yogatama, Sven Gowal, Krishnamurthy Dvijotham, and Pushmeet Kohli. 2019. Achieving verified robustness to symbol substitutions via interval bound propagation. ArXiv, abs/1909.01492. our card
  14. Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial example generation with syntactically controlled paraphrase networks. CoRR, abs/1804.06059. our card
  15. Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2019. Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment. arXiv e-prints, page arXiv:1907.11932. our card
  16. Volodymyr Kuleshov, Shantanu Thakoor, Tingfung Lau, and Stefano Ermon. 2018. Adversarial examples for natural language classification problems. find
  17. Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. 2018. Textbugger: Generating adversarial text against real-world applications. arXiv preprint arXiv:1812.05271. our card
  18. Bin Liang, Hongcheng Li, Miaoqiang Su, Pan Bian, Xirong Li, and Wenchang Shi. 2017. Deep text classification can be fooled. arXiv preprint arXiv:1704.08006. — [arXiv:1704.08006](https://arxiv.org/abs/1704.08006) find
  19. R. Likert. 1932. A Technique for the Measurement of Attitudes. Number nos. 136-165 in A Technique for the Measurement of Attitudes. publisher not identified. find
  20. Zachary Chase Lipton and Jacob Steinhardt. 2018. Troubling trends in machine learning scholarship. ArXiv, abs/1807.03341. find
  21. Paul Michel, Xian Li, Graham Neubig, and Juan Miguel Pino. 2019. On evaluation of adversarial perturbations for sequence-to-sequence models. CoRR, abs/1903.06620. our card
  22. John X. Morris, Eli Lifland, Jin Yong Yoo, and Yanjun Qi. 2020. Textattack: A framework for adversarial attacks in natural language processing. find
  23. Nikola Mrksic, Diarmuid Ó Séaghdha, Blaise Thomson, Milica Gasic, Lina Maria Rojas-Barahona, Pei hao Su, David Vandyke, Tsung-Hsien Wen, and Steve J. Young. 2016. Counter-fitting word vectors to linguistic constraints. In HLT-NAACL. find
  24. Daniel Naber. 2003. A rule-based style and grammar checker. find
  25. Bo Pang and Lillian Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 115– 124, Ann Arbor, Michigan. Association for Computatio… our card
  26. Nicolas Papernot, Patrick McDaniel, Ananthram Swami, and Richard Harang. 2016. Crafting adversarial input sequences for recurrent neural networks. In Military Communications Conference, MILCOM 2016-2016 IEEE, pages 49–54. IEEE. find
  27. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. CoRR, abs/1905.12616. our card
  28. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational… our card
  29. Maja Popović. 2015. chrF: character n-gram f-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. our card
  30. Wei Emma Zhang, Quan Z. Sheng, and Ahoud Abdul-rahmn F. Alhazmi. 2019. Generating textual adversarial examples for deep learning models: A survey. CoRR, abs/1901.06796. find
  31. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in neural information processing systems, pages 649–657. our card
  32. Zhengli Zhao, Dheeru Dua, and Sameer Singh. 2017. Generating natural adversarial examples. arXiv preprint arXiv:1710.11342. find
  33. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. our card
  34. Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1085–1097, Florence, Italy. Association for Computati… our card
  35. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Semantically equivalent adversarial rules for debugging nlp models. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 856–865. find
  36. Suranjana Samanta and Sameep Mehta. 2017. Towards crafting text adversarial samples. arXiv preprint arXiv:1707.02812. find
  37. Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2018. Neural network acceptability judgments. CoRR, abs/1805.12471. find
  38. John Wieting and Kevin Gimpel. 2018. Paranmt-50m: Pushing the limits of paraphrastic sentence embeddings with millions of machine translations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 451–462. our card
  39. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/morris-2020-reevaluating-adversarial-examples-natural-language/