Elamur Research
☰
home / papers / 2016 / Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization

Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1610.02391

Summary

Convolutional networks perform well across vision tasks, but their internal reasoning is opaque, so it is hard to see why a prediction was made. The authors propose Grad-CAM, a generalization of Class Activation Mapping (CAM). It takes the gradient of a chosen output, such as a class score, a generated caption or a VQA answer, with respect to the feature maps of the last convolutional layer. These gradients are pooled into importance weights, which combine the maps into a coarse heatmap of the image regions that support that output. Unlike CAM, the method works on unmodified, already-trained models, including VGG-style networks with fully-connected layers, captioning, VQA and reinforcement-learning models. Fusing the heatmap with Guided Backpropagation produces Guided Grad-CAM, which is both detailed and class-specific. The paper reports several results:

  • better weakly-supervised localization on ILSVRC-15;
  • resistance to adversarial perturbations and higher faithfulness to the model;
  • usefulness for exposing dataset bias;
  • human studies in which untrained users could identify the stronger of two models that made the same predictions.

The work offers a cheap, broadly applicable way to inspect existing CNNs and to decide how far to trust them.

Key points

  • In a modified top-5 pointing game on VGG-16, Grad-CAM scored 70.58%, compared with 60.30% for Excitation Backprop (c-MWP).
  • Grad-CAM localizes the target well at the final convolutional layer of a 200-layer ResNet (fb.resnet.torch). Localization degrades in earlier residual blocks that follow downsampling, and ResNet-18 and ResNet-50 show the same pattern.
  • Textual explanations come from combining Grad-CAM neuron importance with neuron names from prior work, demonstrated on VGG-16 trained on Places365.
  • Applied to Neuraltalk2 and to VQA models with AlexNet, VGG-16 and VGG-19 backbones, Grad-CAM shows that models without attention still focus on relevant regions.
  • Code is available at github.com/ramprs/grad-cam, with a demo on CloudCV.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

166 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 162 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1007/s11263-019-01228-7 · arXiv 1610.02391

@misc{selvaraju2016grad,
  title = {{Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization}},
  author = {Selvaraju, Ramprasaath R. and Cogswell, Michael and Das, Abhishek and Vedantam, Ramakrishna and Parikh, Devi and Batra, Dhruv},
  year = {2016},
  doi = {10.1007/s11263-019-01228-7},
  eprint = {1610.02391},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1007/s11263-019-01228-7}
}

References

61 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Agrawal, D. Batra, and D. Parikh. Analyzing the Behavior of Visual Question Answering Models. In EMNLP, 2016. 2 find
  2. H. Agrawal, C. S. Mathialagan, Y. Goyal, N. Chavali, P. Banik, A. Mohapatra, A. Osman, and D. Batra. CloudCV: Large Scale Distributed Computer Vision as a Cloud Service. In Mobile Cloud Visual Media Computing, pages 265–290. Springer, 2015. 1 find
  3. S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. Lawrence Zitnick, and D. Parikh. VQA: Visual Question Answering. In ICCV, 2015. 1, 2, 10, 12 find
  4. D. Bau, B. Zhou, A. Khosla, A. Oliva, and A. Torralba. Network dissection: Quantifying interpretability of deep visual representations. In Computer Vision and Pattern Recognition, 2017. 1, 3, 10 find
  5. L. Bazzani, A. Bergamo, D. Anguelov, and L. Torresani. Self-taught object localization with deep networks. In WACV, 2016. 4 find
  6. Y. Bengio, A. Courville, and P. Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798–1828, 2013. 4 find our card
  7. X. Chen, H. Fang, T.-Y. Lin, R. Vedantam, S. Gupta, P. Dollár, and C. L. Zitnick. Microsoft COCO captions: Data Collection and Evaluation Server. arXiv preprint arXiv:1504.00325, 2015. 1, 10 find our card
  8. R. G. Cinbis, J. Verbeek, and C. Schmid. Weakly supervised object localization with multi-fold multiple instance learning. IEEE transactions on pattern analysis and machine intelligence, 2016. 3 find
  9. A. Das, H. Agrawal, C. L. Zitnick, D. Parikh, and D. Batra. Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions? In EMNLP, 2016. 12 find
  10. A. Das, S. Datta, G. Gkioxari, S. Lee, D. Parikh, and D. Batra. Embodied Question Answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 1 find
  11. A. Das, S. Kottur, K. Gupta, A. Singh, D. Yadav, J. M. Moura, D. Parikh, and D. Batra. Visual Dialog. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 1 find
  12. A. Das, S. Kottur, J. M. Moura, S. Lee, and D. Batra. Learning cooperative visual dialog agents with deep reinforcement learning. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017. 1 find
  13. H. de Vries, F. Strub, S. Chandar, O. Pietquin, H. Larochelle, and A. C. Courville. Guesswhat?! visual object discovery through multi-modal dialogue. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 1 find
  14. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR, 2009. 2, 6, 14 find
  15. A. Dosovitskiy and T. Brox. Inverting Convolutional Networks with Convolutional Networks. In CVPR, 2015. 3 find
  16. D. Erhan, Y. Bengio, A. Courville, and P. Vincent. Visualizing Higher-layer Features of a Deep Network. University of Montreal, 1341, 2009. 3 find
  17. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results. http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html, 2009. 19, 20 ↗
  18. H. Fang, S. Gupta, F. Iandola, R. K. Srivastava, L. Deng, P. Dollár, J. Gao, X. He, M. Mitchell, J. C. Platt, et al. From Captions to Visual Concepts and Back. In CVPR, 2015. 1 find
  19. C. Gan, N. Wang, Y. Yang, D.-Y. Yeung, and A. G. Hauptmann. Devnet: A deep event network for multimedia event detection and evidence recounting. In CVPR, 2015. 3 find
  20. H. Gao, J. Mao, J. Zhou, Z. Huang, L. Wang, and W. Xu. Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering. In NIPS, 2015. 1, 10, 12 find
  21. R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. In CVPR, 2014. 1 find our card
  22. I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. stat, 2015. 9 find our card
  23. D. Gordon, A. Kembhavi, M. Rastegari, J. Redmon, D. Fox, and A. Farhadi. Iqa: Visual question answering in interactive environments. arXiv preprint arXiv:1712.03316, 2017. 1 — [arXiv:1712.03316](https://arxiv.org/abs/1712.03316) find
  24. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016. 1, 2, 13, 21 find
  25. D. Hoiem, Y. Chodpathumwan, and Q. Dai. Diagnosing Error in Object Detectors. In ECCV, 2012. 2 find
  26. P. Jackson. Introduction to Expert Systems. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 3rd edition, 1998. 2 find
  27. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional Architecture for Fast Feature Embedding. In ACM MM, 2014. 6 find our card
  28. E. Johns, O. Mac Aodha, and G. J. Brostow. Becoming the Expert - Interactive Multi-Class Machine Teaching. In CVPR, 2015. 2 find
  29. J. Johnson, A. Karpathy, and L. Fei-Fei. DenseCap: Fully Convolutional Localization Networks for Dense Captioning. In CVPR, 2016. 1, 10, 11 find
  30. A. Karpathy. What I learned from competing against a ConvNet on ImageNet. http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/, 2014. 2 ↗
  31. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In CVPR, 2015. 10, 11 find our card
  32. A. Kolesnikov and C. H. Lampert. Seed, expand and constrain: Three principles for weakly-supervised image segmentation. In ECCV, 2016. 7 find
  33. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012. 1, 5, 6, 16 find
  34. M. Lin, Q. Chen, and S. Yan. Network in network. In ICLR, 2014. 3 find
  35. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. In ECCV. 2014. 14, 15 find our card
  36. Z. C. Lipton. The Mythos of Model Interpretability. ArXiv e-prints, June 2016. 2, 3 find
  37. J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015. 1 find
  38. J. Lu, X. Lin, D. Batra, and D. Parikh. Deeper LSTM and normalized CNN Visual Question Answering model. https: //github.com/VT-vision-lab/VQA\_LSTM\_CNN, 2015. 12, 13, 15 find
  39. J. Lu, J. Yang, D. Batra, and D. Parikh. Hierarchical question-image co-attention for visual question answering. In NIPS, 2016. 13 find
  40. A. Mahendran and A. Vedaldi. Salient deconvolutional networks. In European Conference on Computer Vision, 2016. 3 find
  41. A. Mahendran and A. Vedaldi. Visualizing deep convolutional neural networks using natural pre-images. International Journal of Computer Vision, pages 1–23, 2016. 3, 4 find
  42. M. Malinowski, M. Rohrbach, and M. Fritz. Ask your neurons: A neural-based approach to answering questions about images. In ICCV, 2015. 1, 10, 12 find
  43. M. Oquab, L. Bottou, I. Laptev, and J. Sivic. Learning and transferring mid-level image representations using convolutional neural networks. In CVPR, 2014. 3, 4 find
  44. M. Oquab, L. Bottou, I. Laptev, and J. Sivic. Is object localization for free? – weakly-supervised learning with convolutional neural networks. In CVPR, 2015. 3 find
  45. P. O. Pinheiro and R. Collobert. From image-level to pixel-level labeling with convolutional networks. In CVPR, 2015. 3 find
  46. M. Ren, R. Kiros, and R. Zemel. Exploring models and data for image question answering. In NIPS, 2015. 1, 10, 12 find
  47. M. T. Ribeiro, S. Singh, and C. Guestrin. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In SIGKDD, 2016. 3, 8 find our card
  48. R. R. Selvaraju, P. Chattopadhyay, M. Elhoseiny, T. Sharma, D. Batra, D. Parikh, and S. Lee. Choose your neuron: Incorporating domain knowledge through neuron-importance. In Proceedings of the European Conference on Computer Vision (ECCV), pages 526–541, 2018. 3 find
  49. R. R. Selvaraju, S. Lee, Y. Shen, H. Jin, S. Ghosh, L. Heck, D. Batra, and D. Parikh. Taking a hint: Leveraging explanations to make vision and language models more grounded. In Proceedings of the International Conference on Computer Vision (ICCV), 2019. 3 find
  50. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484–489, 2016. 2 find
  51. K. Simonyan, A. Vedaldi, and A. Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. CoRR, abs/1312.6034, 2013. 3, 6 find our card
  52. K. Simonyan and A. Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. In ICLR, 2015. 5, 6, 14, 15, 16 find our card
  53. J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. A. Riedmiller. Striving for Simplicity: The All Convolutional Net. CoRR, abs/1412.6806, 2014. 2, 3, 6, 14 — [arXiv:1412.6806](https://arxiv.org/abs/1412.6806) find
  54. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016. 6, 12 find
  55. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A neural image caption generator. In CVPR, 2015. 1, 10, 12 find our card
  56. C. Vondrick, A. Khosla, T. Malisiewicz, and A. Torralba. HOGgles: Visualizing Object Detection Features. ICCV, 2013. 9 find
  57. M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In ECCV, 2014. 2, 3, 4, 6, 8, 10, 14 find our card
  58. J. Zhang, Z. Lin, J. Brandt, X. Shen, and S. Sclaroff. Top-down Neural Attention by Excitation Backprop. In ECCV, 2016. 4, 6, 7, 12, 16, 20 find
  59. B. Zhou, A. Khosla, L. A., A. Oliva, and A. Torralba. Learning Deep Features for Discriminative Localization. In CVPR, 2016. 2, 3, 5, 6, 20 find
  60. B. Zhou, A. Khosla, À. Lapedriza, A. Oliva, and A. Torralba. Object detectors emerge in deep scene cnns. CoRR, abs/1412.6856, 2014. 10 — [arXiv:1412.6856](https://arxiv.org/abs/1412.6856) find
  61. B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017. 10, 11, 20 find