Elamur Research
☰
home / papers / 2019 / EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks

EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1901.11196

Summary

Text classifiers trained on small corpora often overfit. Unlike computer vision, NLP had no cheap, general-purpose augmentation recipe, and existing options such as back-translation or language-model-based word substitution take real engineering effort. Wei and Zou propose EDA, a set of four lightweight edits: synonym replacement using WordNet, insertion of synonyms at random positions, random word swaps, and random word deletion. A parameter α ties how much each edit changes to the sentence's length, and n_aug sets how many new sentences each original produces. The authors tested EDA with an LSTM-RNN and a Kim-style CNN on five benchmarks (SST-2, CR, SUBJ, TREC, PC), using training subsets from 500 examples up to the full set. Every task improved, and the gains were largest when data was scarce. Models trained with EDA on half the data matched baselines trained on all of it. A t-SNE plot suggests augmented sentences mostly keep their original labels. EDA needs no trained language model and no external corpus, so it is a simple baseline for low-resource text classification. Its benefit shrinks with large datasets or pre-trained models.

Key points

  • Averaged over both models and all five tasks, accuracy rose from 76.9% to 79.9% with 500 training examples (+3.0) and from 87.8% to 88.6% with the full data (+0.8).
  • With 50% of the training data, EDA reached 88.6% average accuracy. The best score without augmentation was 88.3%, using 100% of the data.
  • In the ablation, α ranged from 0.05 to 0.5, and each of the four operations helped on its own. α≈0.1 worked best overall. Large α values hurt synonym replacement and deletion most, while random insertion was the most stable.
  • Recommended settings: for 500 examples, α=0.05 and n_aug=16; for 2,000, α=0.05 and n_aug=8; for 5,000 or more, α=0.1 and n_aug=4.
  • Limitations: the gain was under 1% on the full datasets, and a follow-up study found almost no benefit with ULMFit (the authors expect the same for ELMo and BERT). Code is at github.com/jasonwei20/eda_nlp.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

563 papers of our library cite it (of the 140673 whose bibliographies we read); 12 of them have a page here, 551 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1901.11196 · arXiv 1901.11196

@misc{wei2019eda,
  title = {{EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks}},
  author = {Wei, Jason and Zou, Kai},
  year = {2019},
  doi = {10.48550/arxiv.1901.11196},
  eprint = {1901.11196},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1901.11196}
}

References

33 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Xiaodong Cui, Vaibhava Goel, and Brian Kingsbury. 2015. Data augmentation for deep neural network acoustic modeling. IEEE/ACM Trans. Audio, Speech and Lang. Proc., 23(9):1469–1477. find
  2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805. find
  3. Marzieh Fadaee, Arianna Bisazza, and Christof Monz. 2017. Data augmentation for low-resource neural machine translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 567– 573. Association for Computational Linguistics. find our card
  4. Murthy Ganapathibhotla and Bing Liu. 2008. Mining opinions in comparative sentences. In Proceedings of the 22Nd International Conference on Computational Linguistics - Volume 1, COLING ’08, pages 241–248, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  5. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’04, pages 168–177, New York, NY, USA. ACM. find
  6. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Toward controlled generation of text. In ICML. find our card
  7. Kushal Kafle, Mohammed Yousefhussien, and Christopher Kanan. 2017. Data augmentation for visual question answering. In Proceedings of the 10th International Conference on Natural Language Generation, pages 198–202. Association for Computational Linguistics. find
  8. Yoon Kim. 2014. Convolutional neural networks for sentence classification. CoRR, abs/1408.5882. find our card
  9. Tom Ko, Vijayaditya Peddinti, Daniel Povey, and Sanjeev Khudanpur. 2015. Audio augmentation for speech recognition. In INTERSPEECH. find
  10. Sosuke Kobayashi. 2018. Contextual augmentation: Data augmentation by words with paradigmatic relations. In NAACL-HLT. find our card
  11. Oleksandr Kolomiyets, Steven Bethard, and Marie-Francine Moens. 2011. Model-portability experiments for textual temporal analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers - Volume 2, HLT ’11, pages 271–276,… find
  12. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2017. Imagenet classification with deep convolutional neural networks. Commun. ACM, 60(6):84– 90. find
  13. Xin Li and Dan Roth. 2002. Learning question classifiers. In Proceedings of the 19th International Conference on Computational Linguistics - Volume 1, COLING ’02, pages 1–7, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  14. Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016. Recurrent neural network for text classification with multi-task learning. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI’16, pages 2873–2879. AAAI Press. find
  15. Qian Liu, Zhiqiang Gao, Bing Liu, and Yuanlin Zhang. 2015. Automated rule selection for aspect extraction in opinion mining. In Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI’15, pages 1291–1297. AAAI Press. find
  16. George A. Miller. 1995. Wordnet: A lexical database for english. Commun. ACM, 38(11):39–41. find our card
  17. Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42Nd Annual Meeting on Association for Computational Linguistics, ACL ’04, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  18. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532– 1543. find
  19. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. CoRR, abs/1802.05365. find
  20. David Rolnick, Andreas Veit, Serge J. Belongie, and Nir Shavit. 2017. Deep learning is robust to massive label noise. CoRR, abs/1705.10694. — [arXiv:1705.10694](https://arxiv.org/abs/1705.10694) find
  21. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 86–96. Association for Computational Linguistics. find
  22. Sam Shleifer. 2019. Low resource text classification with ulmfit and backtranslation. CoRR, abs/1903.09244. — [arXiv:1903.09244](https://arxiv.org/abs/1903.09244) find
  23. Miikka Silfverberg, Adam Wiemerslage, Ling Liu, and Lingshuang Jack Mao. 2017. Data augmentation for morphological reinflection. In Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection, pages 90–99. Association for Computational Linguistics. find
  24. Patrice Simard, Yann LeCun, John S. Denker, and Bernard Victorri. 1998. Transformation invariance in pattern recognition-tangent distance and tangent propagation. In Neural Networks: Tricks of the Trade, This Book is an Outgrowth of a 1996 NIPS Workshop, pages 239–27, London, UK, UK. Springer-Verlag… find
  25. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher Manning, Andrew Ng, and Christopher Potts. 2013. Parsing With Compositional Vector Grammars. In EMNLP. find
  26. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2014. Going deeper with convolutions. CoRR, abs/1409.4842. — [arXiv:1409.4842](https://arxiv.org/abs/1409.4842) find
  27. Duyu Tang, Bing Qin, and Ting Liu. 2015. Document modeling with gated recurrent neural network for sentiment classification. pages 1422–1432. find
  28. Simon Tong and Daphne Koller. 2002. Support vector machine active learning with applications to text classification. J. Mach. Learn. Res., 2:45–66. find
  29. Laurens Van Der Maaten. 2014. Accelerating t-sne using tree-based algorithms. J. Mach. Learn. Res., 15(1):3221–3245. find
  30. William Yang Wang and Diyi Yang. 2015. That’s so annoying!!!: A lexical and frame-semantic embedding based data augmentation approach to automatic categorization of annoying behaviors using \#petpeeve tweets. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, p… find
  31. Ziang Xie, Sida I. Wang, Jiwei Li, Daniel Levy, Aiming Nie, Dan Jurafsky, and Andrew Y. Ng. 2017. Data noising as smoothing in neural network language models. find
  32. Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V. Le. 2018. Qanet: Combining local convolution with global self-attention for reading comprehension. CoRR, abs/1804.09541. find our card
  33. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1, NIPS’15, pages 649–657, Cambridge, MA, USA. MIT Press. find our card