Elamur Research⌕
☰
home / papers / 2018 / AMC: AutoML for Model Compression and Acceleration on Mobile Devices

AMC: AutoML for Model Compression and Acceleration on Mobile Devices

Paper · 2018 · Yihui He; Ji Lin; Zhijian Liu; Hanrui Wang; Li-Jia Li; and Song Han

Summary

Picking how much to prune each layer of a CNN is usually done by hand, using heuristics and expert intuition. The search space is large, and manual choices tend to be both slow and suboptimal. The authors propose AMC, which casts this choice as a reinforcement learning task. A DDPG agent moves through a pretrained network one layer at a time. For each layer it reads a compact description and outputs a continuous pruning ratio, and channel pruning applies that ratio. After all layers are compressed, the pruned model's validation accuracy is measured without retraining, which keeps the search cheap. Only the best candidate is fine-tuned at the end.

There are two modes. In the first, the agent's choices are restricted so the model always fits a FLOPs or latency budget. In the second, a reward that combines error and FLOPs pushes toward the smallest model that keeps accuracy.

Across VGG, ResNet and MobileNet on ImageNet, AMC outperforms hand-crafted pruning policies. It roughly doubles MobileNet's measured speed on an Android phone with negligible accuracy loss, which matters for deploying models on constrained hardware.

Key points

  • VGG-16: at 4× FLOPs reduction on ImageNet, AMC is 2.7% more accurate than a hand-designed compression policy.
  • ResNet-50: the lossless compression ratio rises from an expert-tuned 3.4× to 5×.
  • MobileNet, 50% FLOPs target:
    • 70.5% Top-1 at 285 MFLOPs, versus 68.4% for 0.75 MobileNet at 325 MFLOPs and 70.6% for the full model.
    • 1.81× speedup on a Google Pixel 1, timed with TensorFlow Lite.
    • 1.43× speedup on a Titan XP GPU.
  • MobileNet, 50% latency target: 70.2% Top-1 and 1.95× faster on the phone (8.1 → 16.0 fps). GPU speedup is 1.53×, and memory use drops from 20.1MB to 13.2MB.
  • Detection transfer: a 4× AMC-compressed VGG-16 used as the Faster R-CNN backbone on PASCAL VOC 2007 scores 68.8 mAP (baseline 68.7) and 37.2 mAP[0.5, 0.95] (baseline 36.7). Both beat hand-crafted 4× pruning.

Interesting ideas

  • I1. Cheap proxy reward: each candidate is scored by its accuracy before fine-tuning, instead of retraining it. Using an inexpensive stand-in for the true objective can make many search loops affordable, such as hyperparameter search or data selection.
  • I2. Continuous actions: the agent outputs a real-valued ratio rather than choosing from a few fixed options. This preserves the ordering between choices (more vs. less aggressive) and allows fine control, which helps whenever outcomes are sensitive to small changes.
  • I3. Budgets enforced by limiting actions: rather than penalizing budget violations in the reward, the allowed actions are bounded so every result is feasible. This is a simple way to guarantee hard constraints in any sequential decision process.
  • I4. Layer descriptors with progress terms: each layer's state includes its shape, its FLOPs, and how much computation has already been removed or still remains. This lets one small, non-RNN policy reason about a global budget while making local decisions.
  • I5. Optimizing measured latency: swapping FLOPs for latency measured on the target device made the achieved speedup land close to the requested one. This is a reminder to optimize the real deployment metric rather than a theoretical proxy.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States492 works · 440,463 citations🎓Carnegie Mellon UniversityPittsburgh · United States598 works · 191,226 citations🎓Massachusetts Institute of TechnologyCambridge · United States243 works · 76,506 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 1Research subjects and languages the work is about.Efficient models & compression · 297 works

Cite

Cite as Yihui He; Ji Lin; Zhijian Liu; Hanrui Wang; Li-Jia Li; and Song Han. 2018. AMC: AutoML for Model Compression and Acceleration on Mobile Devices. arXiv:1802.03494.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1007/978-3-030-01234-2_48 · arXiv 1802.03494

@misc{li2018amc,
  title = {{AMC: AutoML for Model Compression and Acceleration on Mobile Devices}},
  author = {Yihui He; Ji Lin; Zhijian Liu; Hanrui Wang; Li-Jia Li; and Song Han},
  year = {2018},
  doi = {10.1007/978-3-030-01234-2\_48},
  eprint = {1802.03494},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1007/978-3-030-01234-2_48}
}

References

57 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anwar, S., Sung, W.: Compact deep convolutional neural networks with coarse pruning. arXiv preprint arXiv:1610.09639 (2016) find
  2. Ashok, A., Rhinehart, N., Beainy, F., Kitani, K.M.: N2n learning: Network to network compression via policy gradient reinforcement learning. arXiv preprint arXiv:1709.06030 (2017) — [arXiv:1709.06030](https://arxiv.org/abs/1709.06030) find
  3. Bagherinezhad, H., Rastegari, M., Farhadi, A.: Lcnn: Lookup-based convolutional neural network. arXiv preprint arXiv:1611.06473 (2016) find
  4. Baker, B., Gupta, O., Naik, N., Raskar, R.: Designing neural network architectures using reinforcement learning. arXiv preprint arXiv:1611.02167 (2016) — [arXiv:1611.02167](https://arxiv.org/abs/1611.02167) find
  5. Brock, A., Lim, T., Ritchie, J.M., Weston, N.: Smash: one-shot model architecture search through hypernetworks. arXiv preprint arXiv:1708.05344 (2017) — [arXiv:1708.05344](https://arxiv.org/abs/1708.05344) find
  6. Cai, H., Chen, T., Zhang, W., Yu, Y., Wang, J.: Reinforcement learning for architecture search by network transformation. arXiv preprint arXiv:1707.04873 (2017)
  7. Canziani, A., Paszke, A., Culurciello, E.: An analysis of deep neural network models for practical applications. arXiv preprint arXiv:1605.07678 (2016) — [arXiv:1605.07678](https://arxiv.org/abs/1605.07678) find
  8. Chen, T., Goodfellow, I., Shlens, J.: Net2net: Accelerating learning via knowledge transfer. arXiv preprint arXiv:1511.05641 (2015) — [arXiv:1511.05641](https://arxiv.org/abs/1511.05641) find
  9. Chollet, F.: Xception: Deep learning with depthwise separable convolutions. arXiv preprint arXiv:1610.02357 (2016) find
  10. Courbariaux, M., Bengio, Y.: Binarynet: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830 (2016)
  11. Denton, E.L., Zaremba, W., Bruna, J., LeCun, Y., Fergus, R.: Exploiting linear structure within convolutional networks for efficient evaluation. In: Advances in Neural Information Processing Systems. pp. 1269–1277 (2014) find
  12. Dong, X., Huang, J., Yang, Y., Yan, S.: More is less: A more complicated network with less inference complexity. arXiv preprint arXiv:1703.08651 (2017) find
  13. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PAS-CAL Visual Object Classes Challenge 2007 (VOC2007) Results. http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html ↗
  14. Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 1440–1448 (2015) find
  15. Gong, Y., Liu, L., Yang, M., Bourdev, L.: Compressing deep convolutional networks using vector quantization. arXiv preprint arXiv:1412.6115 (2014) — [arXiv:1412.6115](https://arxiv.org/abs/1412.6115) find
  16. Han, S.: Efficient methods and hardware for deep learning, https://stacks.stanford.edu/file/druid:qf934gh3708/EFFICIENT%20METHODS% 20AND%20HARDWARE%20FOR%20DEEP%20LEARNING-augmented.pdf ↗
  17. Han, S., Kang, J., Mao, H., Hu, Y., Li, X., Li, Y., Xie, D., Luo, H., Yao, S., Wang, Y., et al.: Ese: Efficient speech recognition engine with sparse lstm on fpga. In: Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. pp. 75–84. ACM (2017) find
  18. Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M.A., Dally, W.J.: Eie: efficient inference engine on compressed deep neural network. In: Proceedings of the 43rd International Symposium on Computer Architecture. pp. 243–254. IEEE Press (2016)
  19. Han, S., Mao, H., Dally, W.J.: Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 (2015) Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li and Song Han find
  20. Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. In: Advances in Neural Information Processing Systems. pp. 1135–1143 (2015) find
  21. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 770–778 (2016)
  22. He, Y., Zhang, X., Sun, J.: Channel pruning for accelerating very deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1389–1397 (2017) find
  23. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)
  24. Hu, H., Peng, R., Tai, Y.W., Tang, C.K.: Network trimming: A data-driven neuron pruning approach towards efficient deep architectures. arXiv preprint arXiv:1607.03250 (2016) — [arXiv:1607.03250](https://arxiv.org/abs/1607.03250) find
  25. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015) find
  26. Jaderberg, M., Vedaldi, A., Zisserman, A.: Speeding up convolutional neural networks with low rank expansions. arXiv preprint arXiv:1405.3866 (2014) find
  27. Kim, Y.D., Park, E., Yoo, S., Choi, T., Yang, L., Shin, D.: Compression of deep convolutional neural networks for fast and low power mobile applications. arXiv preprint arXiv:1511.06530 (2015) find
  28. Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009) find
  29. Lavin, A.: Fast algorithms for convolutional neural networks. arXiv preprint arXiv:1509.09308 (2015)
  30. Lebedev, V., Ganin, Y., Rakhuba, M., Oseledets, I., Lempitsky, V.: Speeding-up convolutional neural networks using fine-tuned cp-decomposition. arXiv preprint arXiv:1412.6553 (2014) — [arXiv:1412.6553](https://arxiv.org/abs/1412.6553) find
  31. Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) — [arXiv:1608.08710](https://arxiv.org/abs/1608.08710) find
  32. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015) find
  33. Lin, J., Rao, Y., Lu, J.: Runtime neural pruning. In: Advances in Neural Information Processing Systems. pp. 2178–2188 (2017) find
  34. Luo, J.H., Wu, J., Lin, W.: Thinet: A filter level pruning method for deep neural network compression. arXiv preprint arXiv:1707.06342 (2017) — [arXiv:1707.06342](https://arxiv.org/abs/1707.06342) find
  35. Masana, M., van de Weijer, J., Herranz, L., Bagdanov, A.D., Alvarez, J.M.: Domain-adaptive deep network compression. In: The IEEE International Conference on Computer Vision (ICCV) (Oct 2017) find
  36. Mathieu, M., Henaff, M., LeCun, Y.: Fast training of convolutional networks through ffts. arXiv preprint arXiv:1312.5851 (2013) — [arXiv:1312.5851](https://arxiv.org/abs/1312.5851) find
  37. Miikkulainen, R., Liang, J., Meyerson, E., Rawal, A., Fink, D., Francon, O., Raju, B., Navruzyan, A., Duffy, N., Hodjat, B.: Evolving deep neural networks. arXiv preprint arXiv:1703.00548 (2017) — [arXiv:1703.00548](https://arxiv.org/abs/1703.00548) find
  38. Molchanov, P., Tyree, S., Karras, T., Aila, T., Kautz, J.: Pruning convolutional neural networks for resource efficient transfer learning. CoRR, abs/1611.06440 (2016) AMC: AutoML for Model Compression and Acceleration on Mobile Devices — [arXiv:1611.06440](https://arxiv.org/abs/1611.06440) find
  39. Parashar, A., Rhu, M., Mukkara, A., Puglielli, A., Venkatesan, R., Khailany, B., Emer, J., Keckler, S., Dally, W.J.: Scnn: An accelerator for compressed-sparse convolutional neural networks. In: 44th International Symposium on Computer Architecture (2017) find
  40. Polyak, A., Wolf, L.: Channel-level acceleration of deep face representations. IEEE Access 3, 2163–2175 (2015) find
  41. Rastegari, M., Ordonez, V., Redmon, J., Farhadi, A.: Xnor-net: Imagenet classification using binary convolutional neural networks. In: European Conference on Computer Vision. pp. 525–542. Springer (2016)
  42. Real, E., Moore, S., Selle, A., Saxena, S., Suematsu, Y.L., Le, Q., Kurakin, A.: Large-scale evolution of image classifiers. arXiv preprint arXiv:1703.01041 (2017) find
  43. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems. pp. 91–99 (2015)
  44. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Inverted residuals and linear bottlenecks: Mobile networks for classification, detection and segmentation. arXiv preprint arXiv:1801.04381 (2018) — [arXiv:1801.04381](https://arxiv.org/abs/1801.04381) find
  45. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  46. Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evolutionary computation 10(2), 99–127 (2002) find
  47. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1–9 (2015) find
  48. Vasilache, N., Johnson, J., Mathieu, M., Chintala, S., Piantino, S., LeCun, Y.: Fast convolutional nets with fbfft: A gpu performance evaluation. arXiv preprint arXiv:1412.7580 (2014) find
  49. Wang, H., Zhang, Q., Wang, Y., Hu, R.: Structured probabilistic pruning for deep convolutional neural network acceleration. arXiv preprint arXiv:1709.06994 (2017) — [arXiv:1709.06994](https://arxiv.org/abs/1709.06994) find
  50. Watkins, C.J.C.H.: Learning from delayed rewards. Ph.D. thesis, King’s College, Cambridge (1989) find
  51. Xue, J., Li, J., Gong, Y.: Restructuring of deep neural network acoustic models with singular value decomposition. In: INTERSPEECH. pp. 2365–2369 (2013) find
  52. Yang, T.J., Howard, A., Chen, B., Zhang, X., Go, A., Sze, V., Adam, H.: Netadapt: Platform-aware neural network adaptation for mobile applications. arXiv preprint arXiv:1804.03230 (2018) find
  53. Zhang, X., Zou, J., He, K., Sun, J.: Accelerating very deep convolutional networks for classification and detection. IEEE transactions on pattern analysis and machine intelligence 38(10), 1943–1955 (2016) find
  54. Zhong, Z., Yan, J., Liu, C.L.: Practical network blocks design with q-learning. arXiv preprint arXiv:1708.05552 (2017) find
  55. Zhu, C., Han, S., Mao, H., Dally, W.J.: Trained ternary quantization. arXiv preprint arXiv:1612.01064 (2016) — [arXiv:1612.01064](https://arxiv.org/abs/1612.01064) find
  56. Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016)
  57. Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition. arXiv preprint arXiv:1707.07012 (2017)

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/he-2018-amc-automl-model-compression-acceleration/