Elamur Research
☰
home / papers / 2017 / Towards Deep Learning Models Resistant to Adversarial Attacks

Towards Deep Learning Models Resistant to Adversarial Attacks

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1706.06083

Summary

Deep neural networks can be fooled by adversarial examples: inputs altered so slightly that humans see no difference, yet the classifier errs. Earlier defenses were usually tuned against particular known attacks and gave no clear guarantee. The authors recast robustness as a robust-optimization saddle point problem. An inner maximization looks for the worst-case perturbation inside an ℓ∞-ball around each input, and an outer minimization fits the model parameters to that worst-case loss. In this framing, attacks such as FGSM are approximate solvers of the inner problem, and adversarial training is a way of solving the outer one. Experiments show that the inner problem, although non-concave, has many local maxima of very similar value. This makes projected gradient descent (PGD) a dependable "first-order adversary" for both attacking and training. The authors also find that robust models need substantially more capacity than standard ones. PGD-trained networks on MNIST and CIFAR10 resisted a broad suite of white-box and black-box attacks. This made the paper a widely used baseline for robustness that comes with a concrete threat model.

Key points

  • The core method is min-max (saddle point) adversarial training, using PGD with random restarts as the inner attack. The threat model is ℓ∞-bounded perturbations with ε = 0.3 on MNIST and ε = 8 on CIFAR10.
  • On MNIST, the best model keeps over 89% accuracy against the strongest white-box attacks tested and over 95% under black-box/transfer attacks.
  • On CIFAR10, natural accuracy is 87.3%, but robust accuracy drops to about 45.8% under 20-step PGD and 46.8% under CW. Transfer attacks from an independently trained copy reach about 64%.
  • Increasing network capacity improves robustness. ℓ∞-trained models are more fragile against ℓ2 attacks, though still more robust to them than standard models.
  • Code, pre-trained models and a public attack challenge are available at github.com/MadryLab/mnist_challenge and github.com/MadryLab/cifar10_challenge.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

337 papers of our library cite it (of the 140673 whose bibliographies we read); 14 of them have a page here, 323 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1706.06083 · arXiv 1706.06083

@misc{madry2017deep,
  title = {{Towards Deep Learning Models Resistant to Adversarial Attacks}},
  author = {Madry, Aleksander and Makelov, Aleksandar and Schmidt, Ludwig and Tsipras, Dimitris and Vladu, Adrian},
  year = {2017},
  doi = {10.48550/arxiv.1706.06083},
  eprint = {1706.06083},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1706.06083}
}

References

31 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aharon Ben-Tal, Laurent El Ghaoui, and Arkadi Nemirovski. Robust optimization. Princeton University Press, 2009. find
  2. Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases (ECML-KDD), 2013. find
  3. Battista Biggio and Fabio Roli. Wild patterns: Ten years after the rise of adversarial machine learning. 2018. find
  4. Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In International Conference on Learning Representations (ICLR), 2017. find
  5. Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Workshop on Artificial Intelligence and Security (AISec), 2017. find
  6. Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In Symposium on Security and Privacy (SP), 2017. find
  7. Ronan Collobert and Jason Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning, pages 160–167, 2008. find
  8. Nilesh Dalvi, Pedro Domingos, Sumit Sanghai, and Deepak Verma. Adversarial classification. In international conference on Knowledge discovery and data mining, 2004. find
  9. Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Analysis of classifiers’ robustness to adversarial perturbations. Machine Learning, 107(3):481–508, 2018. find
  10. Amir Globerson and Sam Roweis. Nightmare at test time: robust learning by feature deletion. In Proceedings of the 23rd international conference on Machine learning, 2006. find
  11. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015. find our card
  12. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In international conference on computer vision (ICCV), 2015. find
  13. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016. find
  14. Warren He, James Wei, Xinyun Chen, Nicholas Carlini, and Dawn Song. Adversarial example defense: Ensembles of weak defenses are not strong. In USENIX Workshop on Offensive Technologies (WOOT), 2017. find
  15. Ruitong Huang, Bing Xu, Dale Schuurmans, and Csaba Szepesvari. Learning with a strong adversary. arXiv preprint arXiv:1511.03034, 2015. — [arXiv:1511.03034](https://arxiv.org/abs/1511.03034) find
  16. Alex Krizhevsky. Learning multiple layers of features from tiny images. In Technical report, 2009. find
  17. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2012. find
  18. Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In International Conference on Learning Representations (ICLR), 2017. find
  19. Yann LeCun. The mnist database of handwritten digits. In Technical report, 1998. find
  20. Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin. Second-order adversarial attack and certifiable robustness. arXiv preprint arXiv:1809.03113, 2018. — [arXiv:1809.03113](https://arxiv.org/abs/1809.03113) find
  21. Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Computer Vision and Pattern Recognition (CVPR), 2016. find
  22. Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Conference on computer vision and pattern recognition (CVPR), 2015. find
  23. Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. In ArXiv preprint arXiv:1605.07277, 2016. — [arXiv:1605.07277](https://arxiv.org/abs/1605.07277) find
  24. Nicolas Papernot, Patrick D. McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neural networks. In Symposium on Security and Privacy (SP), 2016. find
  25. Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel. Towards the first adversarially robust neural network model on MNIST. In International Conference on Learning Representations (ICLR), 2019. find
  26. Uri Shaham, Yutaro Yamada, and Sahand Negahban. Understanding adversarial training: Increasing local stability of supervised models through robust optimization. Neurocomputing, 307:195–204, 2018. find
  27. Jure Sokolić, Raja Giryes, Guillermo Sapiro, and Miguel RD Rodrigues. Robust large margin deep neural networks. In Transactions on Signal Processing, 2017. find
  28. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR), 2014. find our card
  29. Florian Tramer, Nicolas Papernot, Ian Goodfellow, and Patrick McDaniel Dan Boneh. The space of transferable adversarial examples. In ArXiv preprint arXiv:1704.03453, 2017. — [arXiv:1704.03453](https://arxiv.org/abs/1704.03453) find
  30. Abraham Wald. Statistical decision functions which minimize the maximum risk. In Annals of Mathematics, 1945. find
  31. Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In Network and Distributed Systems Security Symposium (NDSS), 2018. find