Elamur Research
☰
home / papers / 2015 / FaceNet: A Unified Embedding for Face Recognition and Clustering

FaceNet: A Unified Embedding for Face Recognition and Clustering

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1503.03832

Summary

Most deep face recognition systems train a network to classify known identities. They then reuse an intermediate bottleneck layer as the face descriptor, often adding PCA, SVMs or a Joint Bayesian model. This is indirect and yields large representations. FaceNet, from Google, trains a deep convolutional network end to end so that its L2-normalised output is a compact 128-D embedding on a hypersphere, where squared Euclidean distance measures face similarity. Training uses a triplet loss related to LMNN: an anchor face must be closer to a positive of the same person than to a negative of another person, by a fixed margin. Most triplets are trivially satisfied, so the authors select hard and semi-hard examples online inside large mini-batches. They test two backbones, Zeiler&Fergus-style and Inception-style networks. Inputs are just tight face crops, with no 2D or 3D alignment. The system sets new records on LFW and YouTube Faces DB. With such embeddings, verification, k-NN recognition and clustering become simple. The paper also introduces harmonic embeddings, which keep different model versions directly comparable.

Key points

  • Accuracy of 99.63% on Labeled Faces in the Wild (LFW) and 95.12% on YouTube Faces DB, about 30% lower error than the previous best, DeepId2+.
  • Each face is represented by a 128-D embedding that fits in only 128 bytes. Verification is a distance threshold, recognition is k-NN, and clustering uses k-means or agglomerative methods.
  • Online triplet mining uses mini-batches of a few thousand faces, with roughly 40 images per identity, following a curriculum-like schedule of increasingly difficult triplets.
  • Inception-based models need up to 20× fewer parameters than Zeiler&Fergus-style networks for comparable performance.
  • A harmonic triplet loss lets an improved model (NN2) produce embeddings compatible with an older one (NN1). Mixed NN2–NN1 comparisons on the internal 3G dataset still beat NN1 alone.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

257 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 250 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1109/cvpr.2015.7298682 · arXiv 1503.03832

@misc{schroff2015facenet,
  title = {{FaceNet: A Unified Embedding for Face Recognition and Clustering}},
  author = {Schroff, Florian and Kalenichenko, Dmitry and Philbin, James},
  year = {2015},
  doi = {10.1109/cvpr.2015.7298682},
  eprint = {1503.03832},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1109/cvpr.2015.7298682}
}

References

23 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Y. Bengio, J. Louradour, R. Collobert, and J. Weston. Curriculum learning. In Proc. of ICML, New York, NY, USA, 2009. 2 find our card
  2. D. Chen, X. Cao, L. Wang, F. Wen, and J. Sun. Bayesian face revisited: A joint formulation. In Proc. ECCV, 2012. 2 find
  3. D. Chen, S. Ren, Y. Wei, X. Cao, and J. Sun. Joint cascade face detection and alignment. In Proc. ECCV, 2014. 7 find
  4. J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, M. Mao, M. Ranzato, A. Senior, P. Tucker, K. Yang, Q. V. Le, and A. Y. Ng. Large scale distributed deep networks. In P. Bartlett, F. Pereira, C. Burges, L. Bottou, and K. Weinberger, editors, NIPS, pages 1232–1240. 2012. 10 find
  5. J. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res., 12:2121–2159, July 2011. 4 find
  6. I. J. Goodfellow, D. Warde-farley, M. Mirza, A. Courville, and Y. Bengio. Maxout networks. In In ICML, 2013. 4 find
  7. G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst, October 2007. 5 find
  8. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4):541–551, Dec. 1989. 2, 4 find
  9. M. Lin, Q. Chen, and S. Yan. Network in network. CoRR, abs/1312.4400, 2013. 2, 4, 6 — [arXiv:1312.4400](https://arxiv.org/abs/1312.4400) find
  10. C. Lu and X. Tang. Surpassing human-level face verification performance on LFW with gaussianface. CoRR, abs/1404.3840, 2014. 1 — [arXiv:1404.3840](https://arxiv.org/abs/1404.3840) find
  11. D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning representations by back-propagating errors. Nature, 1986. 2, 4 find
  12. M. Schultz and T. Joachims. Learning a distance metric from relative comparisons. In S. Thrun, L. Saul, and B. Schölkopf, editors, NIPS, pages 41–48. MIT Press, 2004. 2 find
  13. T. Sim, S. Baker, and M. Bsat. The CMU pose, illumination, and expression (PIE) database. In In Proc. FG, 2002. 2 find
  14. Y. Sun, X. Wang, and X. Tang. Deep learning face representation by joint identification-verification. CoRR, abs/1406.4773, 2014. 1, 2, 3 — [arXiv:1406.4773](https://arxiv.org/abs/1406.4773) find
  15. Y. Sun, X. Wang, and X. Tang. Deeply learned face representations are sparse, selective, and robust. CoRR, abs/1412.1265, 2014. 1, 2, 5, 8 — [arXiv:1412.1265](https://arxiv.org/abs/1412.1265) find
  16. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. CoRR, abs/1409.4842, 2014. 2, 3, 4, 5, 6, 10 — [arXiv:1409.4842](https://arxiv.org/abs/1409.4842) find
  17. Y. Taigman, M. Yang, M. Ranzato, and L. Wolf. Deepface: Closing the gap to human-level performance in face verification. In IEEE Conf. on CVPR, 2014. 1, 2, 5, 7, 8, 9 find
  18. J. Wang, Y. Song, T. Leung, C. Rosenberg, J. Wang, J. Philbin, B. Chen, and Y. Wu. Learning fine-grained image similarity with deep ranking. CoRR, abs/1404.4661, 2014. 2 — [arXiv:1404.4661](https://arxiv.org/abs/1404.4661) find
  19. K. Q. Weinberger, J. Blitzer, and L. K. Saul. Distance metric learning for large margin nearest neighbor classification. In NIPS. MIT Press, 2006. 2, 3 find
  20. D. R. Wilson and T. R. Martinez. The general inefficiency of batch training for gradient descent learning. Neural Networks, 16(10):1429–1451, 2003. 4 find
  21. L. Wolf, T. Hassner, and I. Maoz. Face recognition in unconstrained videos with matched background similarity. In IEEE Conf. on CVPR, 2011. 5 find
  22. M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. CoRR, abs/1311.2901, 2013. 2, 3, 4, 6 find our card
  23. Z. Zhu, P. Luo, X. Wang, and X. Tang. Recover canonical-view faces in the wild with deep neural networks. CoRR, abs/1404.3543, 2014. 2 — [arXiv:1404.3543](https://arxiv.org/abs/1404.3543) find