Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/1907.11932
Summary
NLP classifiers, including pre-trained BERT, can be fooled by small edits to their input. Building such adversarial text is harder than for images because language is discrete, and the edits must keep the meaning, the grammar and the label a human would assign. The authors introduce TextFooler, a black-box attack that needs only the target model's output scores. It first ranks words by how much deleting each one lowers confidence in the true label, and it skips stop words. It then swaps the most influential words for synonyms from counter-fitted word embeddings (Mrkšić et al. 2016). A candidate is kept only if it has the same part of speech and the new sentence stays close to the original under Universal Sentence Encoder. Swaps continue until the prediction flips. The attack was tested on five classification and two entailment datasets against WordCNN, WordLSTM, BERT, InferSent and ESIM. It cut almost every model's accuracy below 10% while changing under a fifth of the words, and human judges found the outputs meaningful and grammatical. This gives a cheap, reproducible baseline for stress-testing models and for adversarial training.
Key points
- Target models: WordCNN, WordLSTM and BERT for classification (e.g., MR, IMDB); InferSent, ESIM and BERT for entailment (SNLI, MNLI). Each test used 1,000 randomly sampled examples.
- Synonym search keeps the top N=50 neighbours with cosine similarity of at least δ=0.7. POS tags come from spaCy, and stop words are filtered with NLTK and spaCy lists. Cost grows linearly with text length.
- Against BERT-Base, after-attack accuracy fell to 11.5% on MR, 13.6% on IMDB, 4.0% on SNLI and 9.6% on MNLI(m). Only 6.1% of words were changed on IMDB and 18.5% on SNLI.
- Retraining BERT with the generated adversarial examples made it harder to attack. After-attack accuracy rose from 11.5% to 18.7% on MR and from 4.0% to 8.3% on SNLI, and more words had to be changed.
- The main error types were wrong word senses, grammatical slips and label-changing content shifts (especially in entailment). Code, target models and test samples are released at github.com/jind11/TextFooler.
Cited by
- On the Opportunities and Risks of Foundation Models
- A Closer Look into the Robustness of Neural Dependency Parsers Using Better Adversarial Examples
- BAE: BERT-based Adversarial Examples for Text Classification
- A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios
- BERT-ATTACK: Adversarial Attack Against BERT Using BERT
- GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
- TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
- Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer
- Tool Learning with Foundation Models
- "Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models
- Prompting GPT-3 To Be Reliable
- DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
- The Rise and Potential of Large Language Model Based Agents: A Survey
- Word-level Textual Adversarial Attacking as Combinatorial Optimization
- Evaluating and Enhancing the Robustness of Neural Network-based Dependency Parsing Models with Adversarial Examples
- PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts
Cite
@misc{jin2019is,
title = {{Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment}},
author = {Jin, Di and Jin, Zhijing and Zhou, Joey Tianyi and Szolovits, Peter},
year = {2019},
doi = {10.48550/arxiv.1907.11932},
eprint = {1907.11932},
archivePrefix = {arXiv},
url = {https://doi.org/10.48550/arxiv.1907.11932}
}References
- [2018] Alzantot, M.; Sharma, Y.; Elgohary, A.; Ho, B.-J.; Srivastava, M.; and Chang, K.-W. 2018. Generating natural language adversarial examples. arXiv preprint arXiv:1804.07998. [2015] Bowman, S. R.; Angeli, G.; Potts, C.; and Manning, C. D. 2015. A large annotated corpus for learning natural lang… find our card
- [2015] Hill, F.; Reichart, R.; and Korhonen, A. 2015. Simlex-999: Evaluating semantic models with (genuine) similarity estimation. Computational Linguistics 41(4):665– 695. [1997] Hochreiter, S., and Schmidhuber, J. 1997. Long short-term memory. Neural computation 9(8):1735–1780. [2014] Kim, Y. 2014… find our card
- ACM. [2014] Pennington, J.; Socher, R.; and Manning, C. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 1532–1543. [2018] Ribeiro, M. T.; Singh, S.; and Guestrin, C. 2018. Semantically equivalent… find
- [2013] Szegedy, C.; Zaremba, W.; Sutskever, I.; Bruna, J.; Erhan, D.; Goodfellow, I.; and Fergus, R. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199. [2017] Williams, A.; Nangia, N.; and Bowman, S. R. 2017. A broad-coverage challenge corpus for sentence understanding t… find our card