Elamur Research
☰
home / papers / 2019 / Similarity of Neural Network Representations Revisited

Similarity of Neural Network Representations Revisited

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1905.00414

Summary

Researchers often compare internal representations across layers and across separately trained models. Methods based on canonical correlation analysis (CCA) are a common tool for this, but the paper shows they have a basic flaw. Any similarity index that is unchanged by every invertible linear transformation becomes uninformative once a layer has at least as many neurons as there are data points. The authors argue that a useful index should instead ignore only orthogonal transformations and uniform rescaling. Scale information, especially in the dominant principal components, turns out to be consistent across trained networks. Their method first builds example-by-example similarity matrices for each representation and then compares those matrices. Grounded in the Hilbert-Schmidt Independence Criterion (HSIC), this approach turns out to be centered kernel alignment (CKA), which is also closely linked to CCA. In experiments on CIFAR-10 and CIFAR-100, CKA reliably matches corresponding layers in networks with different random seeds, widths and architectures, where earlier indexes fail. This makes CKA a practical tool for diagnosing and comparing deep models.

Key points

  • Theorem 1: Any index invariant to invertible linear maps gives the same value for all full-rank representations whenever width ≥ number of examples. Wide convolutional layers often meet this condition.
  • Method: CKA is a normalized HSIC computed on centered kernel (Gram) matrices. The authors use linear and RBF kernels.
  • Width and depth: Wider networks learn more similar representations, and early layers reach saturation at fewer channels than later ones (widths tested up to 4096). In an 8x-deep network, CKA shows that later layers collapse toward the last layer, which logistic-regression probes confirm.
  • Datasets: Early layers of CIFAR-10 and CIFAR-100 models are alike (averaged over 10 models per type, 45 pairs). Their similarity to untrained networks is far lower.
  • Shared subspace: Between two CIFAR-10 networks, the shared subspace is dominated by the top eigenvectors. In the penultimate (global average pooling) layer it has about 10 dimensions, matching the 10 classes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

178 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 176 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1905.00414 · arXiv 1905.00414

@misc{kornblith2019similarity,
  title = {{Similarity of Neural Network Representations Revisited}},
  author = {Kornblith, Simon and Norouzi, Mohammad and Lee, Honglak and Hinton, Geoffrey},
  year = {2019},
  doi = {10.48550/arxiv.1905.00414},
  eprint = {1905.00414},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1905.00414}
}

References

56 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Advani, M. S. and Saxe, A. M. High-dimensional dynamics of generalization error in neural networks. arXiv preprint arXiv:1710.03667, 2017. — [arXiv:1710.03667](https://arxiv.org/abs/1710.03667) find
  2. Amari, S.-i., Ozeki, T., Karakida, R., Yoshida, Y., and Okada, M. Dynamics of learning in MLP: Natural gradient and singularity revisited. Neural Computation, 30(1): 1–33, 2018. find
  3. Björck, Å. and Golub, G. H. Numerical methods for computing angles between linear subspaces. Mathematics of Computation, 27(123):579–594, 1973. find
  4. Bojar, O., Federmann, C., Fishel, M., Graham, Y., Haddow, B., Huck, M., Koehn, P., and Monz, C. Findings of the 2018 Conference on Machine Translation (WMT18). In EMNLP 2018 Third Conference on Machine Translation (WMT18), 2018. find our card
  5. Chen, A. M., Lu, H.-m., and Hecht-Nielsen, R. On the geometry of feedforward neural network error surfaces. Neural Computation, 5(6):910–927, 1993. find
  6. Connolly, A. C., Guntupalli, J. S., Gors, J., Hanke, M., Halchenko, Y. O., Wu, Y.-C., Abdi, H., and Haxby, J. V. The representation of biological classes in the human brain. Journal of Neuroscience, 32(8):2608–2618, 2012. find
  7. Cortes, C., Mohri, M., and Rostamizadeh, A. Algorithms for learning kernels based on centered alignment. Journal of Machine Learning Research, 13(Mar):795–828, 2012. find
  8. Cristianini, N., Shawe-Taylor, J., Elisseeff, A., and Kandola, J. S. On kernel-target alignment. In Advances in Neural Information Processing Systems, 2002. find
  9. Dinh, L., Sohl-Dickstein, J., and Bengio, S. Density estimation using real NVP. In International Conference on Learning Representations, 2017. find
  10. Dumoulin, V., Shlens, J., and Kudlur, M. A learned representation for artistic style. In International Conference on Learning Representations, 2017. find
  11. Edelman, S. Representation is representation of similarities. Behavioral and Brain Sciences, 21(4):449–467, 1998. find
  12. Elsayed, G. F., Lara, A. H., Kaufman, M. T., Churchland, M. M., and Cunningham, J. P. Reorganization between preparatory and movement population responses in motor cortex. Nature Communications, 7:13239, 2016. find
  13. Freiwald, W. A. and Tsao, D. Y. Functional compartmentalization and viewpoint generalization within the macaque face-processing system. Science, 330(6005):845–851, 2010. find
  14. Garriga-Alonso, A., Rasmussen, C. E., and Aitchison, L. Deep convolutional networks as shallow Gaussian processes. In International Conference on Learning Representations, 2019. find
  15. Gatys, L. A., Ecker, A. S., and Bethge, M. Image style transfer using convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition, 2016. find
  16. Golub, G. H. and Zha, H. The canonical correlations of matrix pairs and their numerical computation. In Linear Algebra for Signal Processing, pp. 27–49. Springer, 1995. find
  17. Gretton, A., Bousquet, O., Smola, A., and Schölkopf, B. Measuring statistical dependence with Hilbert-Schmidt norms. In International Conference on Algorithmic Learning Theory, 2005. find
  18. Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., and Pietrini, P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293(5539):2425–2430, 2001. find
  19. He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, 2016. find
  20. Ioffe, S. and Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, 2015. find
  21. Jacobsen, J.-H., Smeulders, A. W., and Oyallon, E. i-RevNet: Deep invertible networks. In International Conference on Learning Representations, 2018. find
  22. Jacot, A., Gabriel, F., and Hongler, C. Neural tangent kernel: Convergence and generalization in neural networks. In Advances in Neural Information Processing Systems, 2018. find
  23. Johnson, J., Alahi, A., and Fei-Fei, L. Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision, 2016. find
  24. Khaligh-Razavi, S.-M. and Kriegeskorte, N. Deep supervised, but not unsupervised, models may explain it cortical representation. PLoS Computational Biology, 10(11): e1003915, 2014. find
  25. Kriegeskorte, N., Mur, M., and Bandettini, P. A. Representational similarity analysis-connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2:4, 2008a. find
  26. Kriegeskorte, N., Mur, M., Ruff, D. A., Kiani, R., Bodurka, J., Esteky, H., Tanaka, K., and Bandettini, P. A. Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron, 60(6):1126–1141, 2008b. find
  27. Kuss, M. and Graepel, T. The geometry of kernel canonical correlation analysis. Technical report, Max Planck Institute for Biological Cybernetics, 2003. find
  28. Laakso, A. and Cottrell, G. Content and cluster analysis: assessing representational similarity in neural systems. Philosophical Psychology, 13(1):47–76, 2000. find
  29. LeCun, Y., Kanter, I., and Solla, S. A. Second order properties of error surfaces: Learning time and generalization. In Advances in Neural Information Processing Systems, 1991. find
  30. Lee, J., Sohl-dickstein, J., Pennington, J., Novak, R., Schoenholz, S., and Bahri, Y. Deep neural networks as gaussian processes. In International Conference on Learning Representations, 2018. find
  31. Li, Y., Yosinski, J., Clune, J., Lipson, H., and Hopcroft, J. Convergent learning: Do different neural networks learn the same representations? In NIPS 2015 Workshop on Feature Extraction: Modern Questions and Challenges, 2015. find
  32. Lorenzo-Seva, U. and Ten Berge, J. M. Tucker’s congruence coefficient as a meaningful index of factor similarity. Methodology, 2(2):57–64, 2006. find
  33. Morcos, A., Raghu, M., and Bengio, S. Insights on representational similarity in neural networks with canonical correlation. In Advances in Neural Information Processing Systems, 2018. find
  34. Mroueh, Y., Marcheret, E., and Goel, V. Asymmetrically weighted CCA and hierarchical kernel sentence embedding for multimodal retrieval. arXiv preprint arXiv:1511.06267, 2015. — [arXiv:1511.06267](https://arxiv.org/abs/1511.06267) find
  35. Novak, R., Xiao, L., Bahri, Y., Lee, J., Yang, G., Abolafia, D. A., Pennington, J., and Sohl-dickstein, J. Bayesian deep convolutional networks with many channels are Gaussian processes. In International Conference on Learning Representations, 2019. find
  36. Orhan, E. and Pitkow, X. Skip connections eliminate singularities. In International Conference on Learning Representations, 2018. find
  37. Press, W. H. Canonical correlation clarified by singular value decomposition, 2011. URL http://numerical.recipes/whp/notes/CanonCorrBySVD.pdf. ↗
  38. Raghu, M., Gilmer, J., Yosinski, J., and Sohl-Dickstein, J. SVCCA: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. In Advances in Neural Information Processing Systems, 2017. find
  39. Ramsay, J., ten Berge, J., and Styan, G. Matrix correlation. Psychometrika, 49(3):403–423, 1984. find
  40. Robert, P. and Escoufier, Y. A unifying tool for linear multivariate statistical methods: the RV-coefficient. Applied Statistics, 25(3):257–265, 1976. find
  41. Romero, A., Ballas, N., Kahou, S. E., Chassang, A., Gatta, C., and Bengio, Y. FitNets: Hints for thin deep nets. In International Conference on Learning Representations, 2015. find
  42. SAS Institute. Introduction to Regression Procedures. 2015. URL https://support.sas. com/documentation/onlinedoc/stat/141/ introreg.pdf. ↗
  43. Saxe, A. M., McClelland, J. L., and Ganguli, S. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. In International Conference on Learning Representations, 2014. find
  44. Sejdinovic, D., Sriperumbudur, B., Gretton, A., and Fukumizu, K. Equivalence of distance-based and RKHS-based statistics in hypothesis testing. The Annals of Statistics, pp. 2263–2291, 2013. find
  45. Smith, S. L., Turban, D. H., Hamblin, S., and Hammerla, N. Y. Offline bilingual word vectors, orthogonal transformations and the inverted softmax. In International Conference on Learning Representations, 2017. find our card
  46. Song, L., Smola, A., Gretton, A., Borgwardt, K. M., and Bedo, J. Supervised feature selection via dependence estimation. In International Conference on Machine learning, 2007. find
  47. Springenberg, J. T., Dosovitskiy, A., Brox, T., and Riedmiller, M. Striving for simplicity: The all convolutional net. In International Conference on Learning Representations Workshop, 2015. find
  48. StataCorp. Stata Multivariate Statistics Reference Manual. 2015. URL https://www.stata.com/manuals14/mv.pdf. ↗
  49. Sussillo, D., Churchland, M. M., Kaufman, M. T., and Shenoy, K. V. A neural network that finds a naturalistic solution for the production of muscle activity. Nature Neuroscience, 18(7):1025, 2015. find
  50. Tucker, L. R. A method for synthesis of factor analysis studies. Technical report, Educational Testing Service, Princeton, NJ, 1951. find
  51. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. Attention is all you need. In Advances in Neural Information Processing Systems, pp. 5998–6008, 2017. find
  52. Vaswani, A., Bengio, S., Brevdo, E., Chollet, F., Gomez, A. N., Gouws, S., Jones, L., Kaiser, Ł., Kalchbrenner, N., Parmar, N., et al. Tensor2tensor for neural machine translation. arXiv preprint arXiv:1803.07416, 2018. find our card
  53. Vinod, H. D. Canonical ridge and econometrics of joint production. Journal of Econometrics, 4(2):147–166, 1976. find
  54. Wang, L., Hu, L., Gu, J., Wu, Y., Hu, Z., He, K., and Hopcroft, J. E. Towards understanding learning representations: To what extent do different neural networks learn the same representation. In Advances in Neural Information Processing Systems, 2018. find
  55. Yamins, D. L., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., and DiCarlo, J. J. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 111(23):8619–8624, 2014. find
  56. Zagoruyko, S. and Komodakis, N. Wide residual networks. In British Machine Vision Conference, 2016. find