Elamur Research
☰
home / papers / 2014 / Explaining and Harnessing Adversarial Examples

Explaining and Harnessing Adversarial Examples

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1412.6572

Summary

Classifiers, including state-of-the-art neural networks, can be pushed into confident errors by tiny input changes chosen to do the most damage. Earlier work blamed this on the strong nonlinearity or overfitting of deep models. Goodfellow, Shlens and Szegedy argue that the main cause is the opposite: linear behaviour in high-dimensional input spaces. Each feature can change by less than the sensor's precision, but across thousands of dimensions those changes add up to a large shift in a unit's activation. Components such as ReLUs, LSTMs and maxout units are built to act almost linearly so they are easier to optimize, and that is why they inherit the weakness. The same reasoning explains why adversarial examples transfer between architectures and training sets. The authors then derive the fast gradient sign method (FGSM), which needs only one backpropagation pass. They test it on MNIST, CIFAR-10 and ImageNet and use it to make adversarial training cheap enough to be practical. Adversarial training of a maxout network lowered its MNIST test error. The paper became a foundation of adversarial-robustness research.

Key points

  • FGSM: the perturbation is η = ε·sign(∇ₓJ(θ, x, y)). For logistic regression this is the exact worst case. With ε = 0.007, GoogLeNet's label for a panda image changes to "gibbon" at 99.3% confidence.
  • Attack strength: on MNIST with ε = 0.25, a shallow softmax classifier reaches 99.9% error (79.3% average confidence) and a maxout network 89.4% error (97.6% confidence). On CIFAR-10 with ε = 0.1, a convolutional maxout network reaches 87.15% error.
  • Adversarial training vs. weight decay: for logistic regression, adversarial training acts like an L1 penalty that is subtracted from the activation rather than added to the cost. It therefore fades once the model's margins are large, while L1 weight decay stays more pessimistic. Adversarial training adds regularization on top of dropout. Dropout, pretraining and model averaging do little for robustness, whereas RBF networks resist these perturbations.
  • Rubbish inputs: on 10,000 Gaussian-noise samples from N(0, I₇₈₄), a maxout+softmax MNIST model makes 98.35% errors, softmax regression 59.8% and an RBF network 0%.
  • Class-targeted fooling images: on CIFAR-10 the gradient-based method succeeds on 75.3% of steps on average. Airplanes are the hardest class at 24.7%.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

572 papers of our library cite it (of the 140673 whose bibliographies we read); 38 of them have a page here, 534 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1412.6572 · arXiv 1412.6572

@misc{goodfellow2014explaining,
  title = {{Explaining and Harnessing Adversarial Examples}},
  author = {Goodfellow, Ian J. and Shlens, Jonathon and Szegedy, Christian},
  year = {2014},
  doi = {10.48550/arxiv.1412.6572},
  eprint = {1412.6572},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1412.6572}
}

References

19 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Bergstra, James, Goodfellow, Ian J., Bergeron, Arnaud, Bouchard, Nicolas, and Bengio, Yoshua. Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop, 2012. find
  2. Bergstra, James, Breuleux, Olivier, Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Desjardins, Guillaume, Turian, Joseph, Warde-Farley, David, and Bengio, Yoshua. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy), June 20… find
  3. Chalupka, K., Perona, P., and Eberhardt, F. Visual Causal Feature Learning. ArXiv e-prints, December 2014. find
  4. Dean, Jeffrey, Corrado, Greg S., Monga, Rajat, Chen, Kai, Devin, Matthieu, Le, Quoc V., Mao, Mark Z., Ranzato, MarcAurelio, Senior, Andrew, Tucker, Paul, Yang, Ke, and Ng, Andrew Y. Large scale distributed deep networks. In NIPS, 2012. find
  5. Deng, Jia, Dong, Wei, Socher, Richard, jia Li, Li, Li, Kai, and Fei-fei, Li. Imagenet: A large-scale hierarchical image database. In In CVPR, 2009. find
  6. Glorot, Xavier, Bordes, Antoine, and Bengio, Yoshua. Deep sparse rectifier neural networks. In JMLR W&CP: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2011), April 2011. find
  7. Goodfellow, Ian J., Mirza, Mehdi, Courville, Aaron, and Bengio, Yoshua. Multi-prediction deep Boltzmann machines. In Neural Information Processing Systems, December 2013a. find
  8. Goodfellow, Ian J., Warde-Farley, David, Lamblin, Pascal, Dumoulin, Vincent, Mirza, Mehdi, Pascanu, Razvan, Bergstra, James, Bastien, Frédéric, and Bengio, Yoshua. Pylearn2: a machine learning research library. arXiv preprint arXiv:1308.4214, 2013b. — [arXiv:1308.4214](https://arxiv.org/abs/1308.4214) find
  9. Goodfellow, Ian J., Warde-Farley, David, Mirza, Mehdi, Courville, Aaron, and Bengio, Yoshua. Maxout networks. In Dasgupta, Sanjoy and McAllester, David (eds.), International Conference on Machine Learning, pp. 1319–1327, 2013c. find
  10. Gu, Shixiang and Rigazio, Luca. Towards deep neural network architectures robust to adversarial examples. In NIPS Workshop on Deep Learning and Representation Learning, 2014. find
  11. Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997. find
  12. Hornik, Kurt, Stinchcombe, Maxwell, and White, Halbert. Multilayer feedforward networks are universal approximators. Neural Networks, 2:359–366, 1989. find
  13. Jarrett, Kevin, Kavukcuoglu, Koray, Ranzato, Marc’Aurelio, and LeCun, Yann. What is the best multi-stage architecture for object recognition? In Proc. International Conference on Computer Vision (ICCV’09), pp. 2146–2153. IEEE, 2009. find
  14. Krizhevsky, Alex and Hinton, Geoffrey. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009. find
  15. Nguyen, A., Yosinski, J., and Clune, J. Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images. ArXiv e-prints, December 2014. find
  16. Rust, Nicole, Schwartz, Odelia, Movshon, J. Anthony, and Simoncelli, Eero. Spatiotemporal elements of macaque V1 receptive fields. Neuron, 46(6):945–956, 2005. find
  17. Srivastava, Nitish, Hinton, Geoffrey, Krizhevsky, Alex, Sutskever, Ilya, and Salakhutdinov, Ruslan. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15 (1):1929–1958, 2014. find
  18. Szegedy, Christian, Liu, Wei, Jia, Yangqing, Sermanet, Pierre, Reed, Scott, Anguelov, Dragomir, Erhan, Dumitru, Vanhoucke, Vincent, and Rabinovich, Andrew. Going deeper with convolutions. Technical report, arXiv preprint arXiv:1409.4842, 2014a. — [arXiv:1409.4842](https://arxiv.org/abs/1409.4842) find
  19. Szegedy, Christian, Zaremba, Wojciech, Sutskever, Ilya, Bruna, Joan, Erhan, Dumitru, Goodfellow, Ian J., and Fergus, Rob. Intriguing properties of neural networks. ICLR, abs/1312.6199, 2014b. URL http: //arxiv.org/abs/1312.6199. find our card