Elamur Research⌕
☰
home / papers / 2013 / DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

Paper · 2013 · Jeff Donahue, Yangqing Jia, Oriol Vinyals et al.

Summary

The paper asks whether the internal activations of a large convolutional network, trained with full supervision on ImageNet object classes, can serve as a ready-made image descriptor for different vision problems that lack enough data to train deep models. The authors retrain the Krizhevsky et al. (2012) architecture (five convolutional and three fully-connected layers) on ILSVRC-2012. They then freeze it and use activations from various layers as fixed features, with no fine-tuning. Simple classifiers are trained on top. They compare these features with hand-crafted descriptors such as GIST and LLC using t-SNE plots, and test them on object recognition (Caltech-101), domain adaptation (Office), fine-grained bird classification (Caltech-UCSD Birds) and scene recognition (SUN-397). The deep features, especially from the upper hidden layers, group images by semantic category more clearly than traditional descriptors. They also beat prior state-of-the-art pipelines, including multi-kernel combinations of many engineered features. The work matters because it established pretrained CNN activations as a generic off-the-shelf representation. It also released the code and weights publicly as DeCAF.

Key points

  • The authors' single instance of the model reaches 42.9% top-1 error on the ILSVRC-2012 validation set, compared with 40.7% for the original single model. Inputs are 224×224 crops of images warped to 256×256.
  • The open-source Python/C implementation, decaf, runs on CPU and processes roughly 40 images per second on an 8-core machine. It is released together with the trained network parameters.
  • On Caltech-UCSD Birds, DeCAF6 alone reaches 58.75% accuracy. Combining it with the part-based DPD pipeline gives 64.96%, versus 50.98% for DPD and 56.78% for POOF.
  • On SUN-397 (50 train / 50 test images per class), the best setting, using DeCAF7 features, scores 40.94%. The previous best of 38.0% (Xiao et al., 2010) relied on multi-kernel learning over many features.
  • t-SNE visualizations on ILSVRC-2012 validation data and SUN-397 show that deeper layers (e.g., DeCAF6) separate semantic categories much better than DeCAF1, GIST or LLC.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California, BerkeleyBerkeley · United States209 works · 244,782 citations🔬International Computer Science InstituteBerkeley · United States18 works · 16,608 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Vision and Pattern Recognition · 3,486 works

Cite

Cite as Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng and Trevor Darrell. 2013. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition. arXiv:1310.1531.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1310.1531 · arXiv 1310.1531

@misc{donahue2013decaf,
  title = {{DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition}},
  author = {Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng and Trevor Darrell},
  year = {2013},
  doi = {10.48550/arxiv.1310.1531},
  eprint = {1310.1531},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1310.1531}
}

References

45 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ando, R. and Zhang, T. A framework for learning predictive structures from multiple tasks and unlabeled data. JMLR, 6, 2005. find
  2. Argyriou, Andreas, Evgeniou, Theodoros, and Pontil, Massimiliano. Multi-task feature learning. In NIPS, 2006. find
  3. Bay, H., Tuytelaars, T., and Gool, L. Van. SURF: Speeded up robust features. In ECCV, 2006. find
  4. Berg, A., Deng, J., and Fei-Fei, L. ImageNet large scale visual recognition challenge 2012. 2012. URL http://www.image-net.org/challenges/LSVRC/2012/. ↗
  5. Berg, T. and Belhumeur, P. POOF: Part-based one-vs-one features for fine-grained categorization, face verification, and attribute estimation. In CVPR, 2013. find
  6. Bo, L., Ren, X., and Fox, D. Kernel descriptors for visual recognition. In NIPS, 2010. find
  7. Bourdev, Lubomir, Maji, Subhransu, and Malik, Jitendra. Describing people: A poselet-based approach to attribute classification. In ICCV, 2011. find
  8. Caruana, R. Multitask learning. Machine Learning, 28, 1997. find
  9. Chopra, S., Balakrishnan, S., and Gopalan, R. Dlid: Deep learning for domain adaptation by interpolating between domains. In ICML Workshop on Challenges in Representation Learning, 2013. find
  10. Dalal, N. and Triggs, B. Histograms of oriented gradients for human detection. In CVPR, 2005. find
  11. Daume III, H. Frustratingly easy domain adaptation. In ACL, 2007.
  12. Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Fei-Fei, L. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR, 2009. find
  13. Fei-Fei, L., Fergus, R., and Perona, P. Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. In CVPR, 2004. find
  14. Felzenszwalb, P., Girshick, R., McAllester, D., and Ramanan, D. Object detection with discriminatively trained part-based models. PAMI, 32, 2010. find
  15. Fidler, S. and Leonardis, A. Towards scalable representations of object categories: Learning a hierarchy of parts. In CVPR, 2007. find
  16. Gong, B., Shi, Y., Sha, F., and Grauman, K. Geodesic flow kernel for unsupervised domain adaptation. In CVPR, 2012. find
  17. Hinton, G. and Salakhutdinov, R. Reducing the dimensionality of data with neural networks. Science, 2006. find
  18. Hinton, G., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012. find
  19. Hoffman, J., Rodner, E., Donahue, J., Saenko, K., and Darrell, T. Efficient learning of domain-invariant image representations. In ICLR, 2013. find
  20. Hsu, D., Kakade, S., Langford, J., and Zhang, T. Multi-label prediction via compressed sensing. arXiv preprint arXiv:0902.1284, 2009. find
  21. Jarrett, K., Kavukcuoglu, K., Ranzato, M., and LeCun, Y. What is the best multi-stage architecture for object recognition? In ICCV, 2009. find
  22. Kennedy, L. and Hauptmann, A. LSCOM lexicon definitions and annotations (version 1.0). 2006. find
  23. Krizhevsky, A., Sutskever, I., and Hinton, G. E. ImageNet classification with deep convolutional neural networks. In NIPS, 2012.
  24. Kulis, B., Saenko, K., and Darrell, T. What you saw is not what you get: Domain adaptation using asymmetric kernel transforms. In CVPR, 2011. find
  25. Le, Q., Zou, W., Yeung, S., and Ng, A. Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis. In CVPR, 2011. find
  26. Le, Q., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G., Dean, J., and Ng, A. Building high-level features using large scale unsupervised learning. In ICML, 2012. find
  27. LeCun, Y., Boser, B., Denker, J., Henderson, D., Howard, R., Hubbard, W., and Jackel, L. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1989. find
  28. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. In IEEE, 1998. find
  29. Li, L., Su, H., Fei-Fei, L., and Xing, E. Object bank: A high-level image representation for scene classification & semantic feature sparsification. In NIPS, 2010. find
  30. Mesnil, G., Dauphin, Y., Glorot, X., Rifai, S., Bengio, Y., Goodfellow, I., Lavoie, E., Muller, X., Desjardins, G., Warde-Farley, D., Vincent, P., Courville, A., and Berkgstra, J. Unsupervised and transfer learning challenge: a deep learning approach. JMLR, 27, 2012. find
  31. Oliva, A. and Torralba, A. Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV, 2001. find
  32. Quattoni, A., Collins, M., and Darrell, T. Transfer learning for image classication with sparse prototype representations. In CVPR, 2008. find
  33. Raina, R. at, Battle, A., Lee, H., Packer, B., and Ng, A. Self-taught learning: Transfer learning from unlabeled data. In ICML, 2007. find
  34. Ren, X. and Ramanan, D. Histograms of sparse codes for object detection. In CVPR, 2013. find
  35. Saenko, K., Kulis, B., Fritz, M., and Darrell, T. Adapting visual category models to new domains. In ECCV, 2010. find
  36. Singh, S., Gupta, A., and Efros, A. Unsupervised discovery of mid-level discriminative patches. In ECCV, 2012. find
  37. Thrun, S. Is learning the n-th thing any easier than learning the first? In NIPS, 1996. find
  38. Torralba, A. and Efros, A. Unbiased look at dataset bias. In CVPR, 2011. find
  39. Torresani, L., Szummer, M., and Fitzgibbon, A. Efficient object category recognition using classemes. In ECCV. 2010. van der Maaten, L. and Hinton, G. Visualizing data using t-sne. JMLR, 9, 2008. find
  40. Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., and Gong, Y. Locality-constrained linear coding for image classification. In CVPR, 2010. find
  41. Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., and Perona, P. Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-001, California Institute of Technology, 2010. find
  42. Xiao, J., Hays, J., Ehinger, K., Oliva, A., and Torralba, A. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR, 2010. find
  43. Yang, J., L., Y., Tian, Y., Duan, L., and Gao, W. Group-sensitive multiple kernel learning for object categorization. In ICCV, 2009. find
  44. Zhang, N., Farrell, R., Iandola, F., and Darrell, T. Deformable part descriptors for fine-grained recognition and attribute prediction. In ICCV, 2013. find
  45. Zhu, L., Chen, Y., and Yuille, A. Unsupervised learning of a probabilistic grammar for object detection and parsing. In NIPS, 2007. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/donahue-2013-decaf-deep-convolutional-activation-feature/