Elamur Research
☰
home / papers / 2017 / Focal Loss for Dense Object Detection

Focal Loss for Dense Object Detection

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1708.02002

Summary

One-stage detectors such as SSD and YOLO scan a dense grid of candidate boxes. That makes them simpler and faster than proposal-based two-stage systems like Faster R-CNN, but they had been less accurate. The authors argue that the main reason is a severe foreground–background imbalance during training. A dense detector scores around 100k locations per image, and most of them are easy background cases that dominate the loss and the gradients. Earlier work handled this with sampling heuristics or hard example mining. Instead, the paper changes the cross entropy loss by multiplying it by (1 − p_t)^γ. This factor shrinks the penalty for predictions that are already confident and correct, so training concentrates on the hard ones. To test this Focal Loss, the authors built RetinaNet, a plain single-stage network that uses a Feature Pyramid Network backbone and anchor boxes, and evaluated it on COCO. RetinaNet runs about as fast as other one-stage models and beat all previously published single models in accuracy, including two-stage ones. The result points to the training objective, rather than network architecture, as what had been holding dense detectors back.

Key points

  • Loss definition: FL(p_t) = −α_t (1 − p_t)^γ log(p_t). With γ = 0 it is the same as cross entropy. γ = 2 worked best, together with α = 0.25. At γ = 2, an example with p_t = 0.9 gets 100× less loss than with CE, and one with p_t ≈ 0.968 gets about 1000× less.
  • Main result: RetinaNet-101-800 (ResNet-101-FPN) scores 39.1 AP on COCO test-dev at about 5 fps. That is 5.9 AP above the strongest one-stage competitor, DSSD (33.2), and 2.3 AP above the best Faster R-CNN variant (Inception-ResNet-v2-TDM). Switching to a ResNeXt-32x8d-101-FPN backbone raises it to 40.8 AP.
  • Ablations (RetinaNet-50-600): plain CE reaches 31.1 AP and focal loss 34.0 AP. A single square anchor already gives 30.3 AP, and 9 anchors per location (3 scales × 3 aspect ratios) give 34.0. Adding more anchors beyond that brought no further gain.
  • Form of the loss is flexible: an alternative version, FL*, with steepness and shift parameters (γ, β), reaches 33.8–33.9 AP, about the same as FL. Any loss that down-weights well-classified examples appears to work.
  • Code: released in Detectron (github.com/facebookresearch/Detectron).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

645 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 636 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1708.02002 · arXiv 1708.02002

@misc{lin2017focal,
  title = {{Focal Loss for Dense Object Detection}},
  author = {Lin, Tsung-Yi and Goyal, Priya and Girshick, Ross and He, Kaiming and Dollár, Piotr},
  year = {2017},
  doi = {10.48550/arxiv.1708.02002},
  eprint = {1708.02002},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1708.02002}
}

References

39 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. S. Bell, C. L. Zitnick, K. Bala, and R. Girshick. Inside-outside net: Detecting objects in context with skip pooling and recurrent neural networks. In CVPR, 2016. 6 find
  2. S. R. Bulo, G. Neuhold, and P. Kontschieder. Loss max-pooling for semantic image segmentation. In CVPR, 2017. 3 find
  3. J. Dai, Y. Li, K. He, and J. Sun. R-FCN: Object detection via region-based fully convolutional networks. In NIPS, 2016. 1 find
  4. N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In CVPR, 2005. 2 find
  5. P. Dollár, Z. Tu, P. Perona, and S. Belongie. Integral channel features. In BMVC, 2009. 2, 3 find
  6. D. Erhan, C. Szegedy, A. Toshev, and D. Anguelov. Scalable object detection using deep neural networks. In CVPR, 2014. 2 find
  7. M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes (VOC) Challenge. IJCV, 2010. 2 find
  8. P. F. Felzenszwalb, R. B. Girshick, and D. McAllester. Cascade object detection with deformable part models. In CVPR, 2010. 2, 3 find
  9. C.-Y. Fu, W. Liu, A. Ranga, A. Tyagi, and A. C. Berg. DSSD: Deconvolutional single shot detector. arXiv:1701.06659, 2016. 1, 2, 8 — [arXiv:1701.06659](https://arxiv.org/abs/1701.06659) find
  10. R. Girshick. Fast R-CNN. In ICCV, 2015. 1, 2, 4, 6, 8 find
  11. R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, 2014. 1, 2, 5 find our card
  12. R. Girshick, I. Radosavovic, G. Gkioxari, P. Dollár, and K. He. Detectron. https://github.com/facebookresearch/detectron, 2018. 8 ↗
  13. T. Hastie, R. Tibshirani, and J. Friedman. The elements of statistical learning. Springer series in statistics Springer, Berlin, 2008. 3, 7 find
  14. K. He, G. Gkioxari, P. Dollár, and R. Girshick. Mask R-CNN. In ICCV, 2017. 1, 2, 4 find
  15. K. He, X. Zhang, S. Ren, and J. Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. In ECCV. 2014. 2 find
  16. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016. 2, 4, 5, 6, 8 find
  17. J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Song, S. Guadarrama, and K. Murphy. Speed/accuracy trade-offs for modern convolutional object detectors. In CVPR, 2017. 2, 8 find
  18. A. Krizhevsky, I. Sutskever, and G. Hinton. ImageNet classification with deep convolutional neural networks. In NIPS, 2012. 2 find
  19. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1989. 2 find
  20. T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie. Feature pyramid networks for object detection. In CVPR, 2017. 1, 2, 4, 5, 6, 8 find
  21. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft COCO: Common objects in context. In ECCV, 2014. 1, 6 find our card
  22. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, and S. Reed. SSD: Single shot multibox detector. In ECCV, 2016. 1, 2, 3, 6, 7, 8 find
  23. J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015. 4 find
  24. P. O. Pinheiro, R. Collobert, and P. Dollar. Learning to segment object candidates. In NIPS, 2015. 2, 4 find
  25. P. O. Pinheiro, T.-Y. Lin, R. Collobert, and P. Dollár. Learning to refine object segments. In ECCV, 2016. 2 find
  26. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi. You only look once: Unified, real-time object detection. In CVPR, 2016. 1, 2 find
  27. J. Redmon and A. Farhadi. YOLO9000: Better, faster, stronger. In CVPR, 2017. 1, 2, 8 find
  28. S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In NIPS, 2015. 1, 2, 4, 5, 8 find our card
  29. H. Rowley, S. Baluja, and T. Kanade. Human face detection in visual scenes. Technical Report CMU-CS-95-158R, Carnegie Mellon University, 1995. 2 find
  30. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun. Overfeat: Integrated recognition, localization and detection using convolutional networks. In ICLR, 2014. 2 find
  31. A. Shrivastava, A. Gupta, and R. Girshick. Training region-based object detectors with online hard example mining. In CVPR, 2016. 2, 3, 6, 7 find
  32. A. Shrivastava, R. Sukthankar, J. Malik, and A. Gupta. Beyond skip connections: Top-down modulation for object detection. arXiv:1612.06851, 2016. 2, 8 — [arXiv:1612.06851](https://arxiv.org/abs/1612.06851) find
  33. K.-K. Sung and T. Poggio. Learning and Example Selection for Object and Pattern Detection. In MIT A.I. Memo No. 1521, 1994. 2, 3 find
  34. C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. In AAAI Conference on Artificial Intelligence, 2017. 8 find
  35. J. R. Uijlings, K. E. van de Sande, T. Gevers, and A. W. Smeulders. Selective search for object recognition. IJCV, 2013. 2, 4 find
  36. R. Vaillant, C. Monrocq, and Y. LeCun. Original approach for the localisation of objects in images. IEE Proc. on Vision, Image, and Signal Processing, 1994. 2 find
  37. P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In CVPR, 2001. 2, 3 find
  38. S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He. Aggregated residual transformations for deep neural networks. In CVPR, 2017. 8 find
  39. C. L. Zitnick and P. Dollár. Edge boxes: Locating object proposals from edges. In ECCV, 2014. 2 find