Elamur Research⌕
☰
home / papers / 2023 / Can GPT-4 Perform Neural Architecture Search?

Can GPT-4 Perform Neural Architecture Search?

Paper · 2023 · Mingkai Zheng, Xiu Su, Shan You et al.

Summary

The authors ask whether GPT-4, used without any fine-tuning, can help with Neural Architecture Search (NAS), a task that usually requires substantial expert knowledge. Their method, GENIUS, treats GPT-4 as a black-box optimiser. The search problem is described in plain text, and GPT-4 proposes an architecture configuration. That model is then trained or looked up and evaluated, and its accuracy is fed back through a fixed prompt asking for a better design with a justification. The loop repeats for a set number of rounds. The approach is tested on NAS-Bench-Macro and Channel-Bench-Macro, which contain precomputed CIFAR-10 results, against a random-sampling baseline. The best discovered architecture is also compared with leading NAS methods on a widely used benchmark. GENIUS outperforms random search and is competitive with established techniques, although the authors stress that state-of-the-art results were not their goal. The work matters because it suggests general-purpose LLMs can act as optimisers with minimal domain input. This raises research opportunities and also AI-safety concerns, such as eroding human know-how and enabling model self-improvement.

Key points

  • Benchmarks:
    • NAS-Bench-Macro has 8 searchable layers, each with 3 candidate blocks, giving 3^8 = 6561 architectures.
    • Channel-Bench-Macro has 7 layers, each with 4 width options, giving 4^7 = 16,384 architectures. It uses MobileNet and ResNet base models.
    • Both benchmarks provide CIFAR-10 results.
  • Random baseline: the best of 10 random samples, averaged over 10,000 repetitions.
  • GENIUS runs: 10 iterations per run, repeated 3 times, at GPT-4 temperatures 0 and 1.
  • Stated limitations:
    • The opaque OpenAI API and nonzero variance even at temperature 0 hurt reproducibility.
    • Possible benchmark contamination of GPT-4's training data cannot be ruled out.
    • On NAS-Bench-201, later iterations sometimes perform worse than earlier ones, for unclear reasons.
  • Code: available at github.com/mingkai-zheng/GENIUS.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of CambridgeCambridge · United Kingdom287 works · 125,174 citations🎓The University of SydneySydney · Australia52 works · 68,450 citations🏢SensetimeBeijing · China14 works · 2,199 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksComputer Vision and Pattern Recognition · 3,621 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 4,145 works

Cite

Cite as Mingkai Zheng, Xiu Su, Shan You, Fei Wang, Chen Qian, Chang Xu and Samuel Albanie. 2023. Can GPT-4 Perform Neural Architecture Search?. arXiv:2304.10970.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2304.10970 · arXiv 2304.10970

@misc{zheng2023can,
  title = {{Can GPT-4 Perform Neural Architecture Search?}},
  author = {Mingkai Zheng, Xiu Su, Shan You, Fei Wang, Chen Qian, Chang Xu and Samuel Albanie},
  year = {2023},
  doi = {10.48550/arxiv.2304.10970},
  eprint = {2304.10970},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2304.10970}
}

References

85 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Boiko, D.A., MacKnight, R., Gomes, G.: Emergent autonomous scientific research capabilities of large language models. arXiv preprint arXiv:2304.05332 (2023) 3, 9
  2. Bordt, S., von Luxburg, U.: Chatgpt participates in a computer science exam. arXiv preprint arXiv:2303.09461 (2023) 3 — [arXiv:2303.09461](https://arxiv.org/abs/2303.09461) find
  3. Bran, A.M., Cox, S., White, A.D., Schwaller, P.: Chemcrow: Augmenting large-language models with chemistry tools. arXiv preprint arXiv:2304.05376 (2023) 3, 9
  4. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems 33, 1877–1901 (2020) 3
  5. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems 33, 1877–1901 (2020) 9
  6. Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y.T., Li, Y., Lundberg, S., et al.: Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712 (2023) 1, 3
  7. Cai, H., Zhu, L., Han, S.: Proxylessnas: Direct neural architecture search on target task and hardware. arXiv preprint arXiv:1812.00332 (2018) 2, 6
  8. Chen, A., Dohan, D.M., So, D.R.: Evoprompting: Language models for code-level neural architecture search. arXiv preprint arXiv:2302.14838 (2023) 2 — [arXiv:2302.14838](https://arxiv.org/abs/2302.14838) find
  9. Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Xu, J., Zhang, Z., Cheng, D., Zhu, C., Cheng, T., Zhao, Q., Li, B., Lu, X., Zhu, R., Wu, Y., Dai, J., Wang, J., Shi, J., Ouyang, W., Loy, C.C., Lin, D.: MMDetection: Open mmlab detection toolbox and benchmark. arXi… — [arXiv:1906.07155](https://arxiv.org/abs/1906.07155) find
  10. Chen, X., Wang, R., Cheng, M., Tang, X., Hsieh, C.J.: Dr{nas}: Dirichlet neural architecture search. In: International Conference on Learning Representations (2021), https://openreview.net/forum? id=9FWas6YbmB3 5 ↗
  11. Chu, X., Li, X., Lu, S., Zhang, B., Li, J.: Mixpath: A unified approach for one-shot neural architecture search. arXiv preprint arXiv:2001.05887 (2020) 8 — [arXiv:2001.05887](https://arxiv.org/abs/2001.05887) find
  12. Chu, X., Zhang, B., Li, J., Li, Q., Xu, R.: Scarletnas: Bridging the gap between scalability and fairness in neural architecture search. arXiv preprint arXiv:1908.06022 4(6) (2019) 6 — [arXiv:1908.06022](https://arxiv.org/abs/1908.06022) find
  13. Chu, X., Zhang, B., Xu, R.: Fairnas: Rethinking evaluation fairness of weight sharing neural architecture search. In: Proceedings of the IEEE/CVF International Conference on computer vision. pp. 12239–12248 (2021) 2, 6, 8 find
  14. Ci, Y., Lin, C., Sun, M., Chen, B., Zhang, H., Ouyang, W.: Evolving search space for neural architecture search. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 6659–6669 (2021) 6 find
  15. Cubuk, E.D., Zoph, B., Shlens, J., Le, Q.V.: Randaugment: Practical automated data augmentation with a reduced search space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 702–703 (2020) 7 find
  16. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A Large-Scale Hierarchical Image Database. In: CVPR09 (2009) 1, 6 find
  17. Dong, X., Yang, Y.: Searching for a robust neural architecture in four gpu hours. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1761–1770 (2019) 5 find
  18. Dong, X., Yang, Y.: Nas-bench-201: Extending the scope of reproducible neural architecture search. In: International Conference on Learning Representations (2020), https://openreview.net/forum? id=HJxyZkBKDr 5, 6 ↗
  19. Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. International journal of computer vision 88(2), 303–338 (2010) 1 find
  20. Fang, M., Wang, Q., Zhong, Z.: Betanas: Balanced training and selective drop for neural architecture search. arXiv preprint arXiv:1912.11191 (2019) 6 — [arXiv:1912.11191](https://arxiv.org/abs/1912.11191) find
  21. Guo, R., Lin, C., Li, C., Tian, K., Sun, M., Sheng, L., Yan, J.: Powering one-shot topological nas with stabilized share-parameter proxy. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIV. pp. 625–641. Springer (2020) 6 find
  22. Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., Sun, J.: Single path one-shot neural architecture search with uniform sampling. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16. pp. 544–560. Springer (2020) 2, 6, 8 find
  23. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016) 1
  24. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016) 2, 4
  25. Hendrycks, D.: Natural selection favors ais over humans. arXiv preprint arXiv:2303.16200 (2023) 9 — [arXiv:2303.16200](https://arxiv.org/abs/2303.16200) find
  26. Hendrycks, D., Mazeika, M.: X-risk analysis for ai research. arXiv preprint arXiv:2206.05862 (2022) 9 — [arXiv:2206.05862](https://arxiv.org/abs/2206.05862) find
  27. Hovy, D., Spruit, S.L.: The social impact of natural language processing. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 591–598 (2016) 3
  28. Howard, A., Sandler, M., Chu, G., Chen, L.C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., et al.: Searching for mobilenetv3. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 1314–1324 (2019) 2, 6, 8 find
  29. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017) 2
  30. Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E.: Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 42, 2011–2023 (2017) 2 find
  31. Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E.: Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 42, 2011–2023 (2017) 6 find
  32. Hu, S., Xie, S., Zheng, H., Liu, C., Shi, J., Liu, X., Lin, D.: Dsnas: Direct neural architecture search without parameter retraining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 12084–12092 (2020) 5 find
  33. Hu, Y., Liang, Y., Guo, Z., Wan, R., Zhang, X., Wei, Y., Gu, Q., Sun, J.: Angle-based search space shrinking for neural architecture search. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIX 16. pp. 119–134. Springer (2020) 6 find
  34. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4700–4708 (2017) 2 find
  35. Huang, J., Gu, S.S., Hou, L., Wu, Y., Wang, X., Yu, H., Han, J.: Large language models can self-improve. arXiv preprint arXiv:2210.11610 (2022) 9
  36. Huang, T., You, S., Wang, F., Qian, C., Zhang, C., Wang, X., Xu, C.: Greedynasv2: Greedier search with a greedy path filter. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11902–11911 (2022) 2, 6, 8 find
  37. Huang, Y., Cheng, Y., Bapna, A., Firat, O., Chen, D., Chen, M., Lee, H., Ngiam, J., Le, Q.V., Wu, Y., et al.: Gpipe: Efficient training of giant neural networks using pipeline parallelism. Advances in neural information processing systems 32 (2019) 8 find
  38. Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al.: Highly accurate protein structure prediction with alphafold. Nature 596(7873), 583–589 (2021) 1
  39. Kandasamy, K., Neiswanger, W., Schneider, J., Poczos, B., Xing, E.P.: Neural architecture search with bayesian optimisation and optimal transport. Advances in neural information processing systems 31 (2018) 2 find
  40. Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009) 1, 3, 4 find
  41. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems. vol. 25. Curran Associates, Inc. (2012), https://proceedings.neurips.cc/pape… ↗
  42. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998) 2 find
  43. Li, C., Peng, J., Yuan, L., Wang, G., Liang, X., Lin, L., Chang, X.: Block-wisely supervised neural architecture search with knowledge distillation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1989–1998 (2020) 6 find
  44. Li, C., Tang, T., Wang, G., Peng, J., Wang, B., Liang, X., Chang, X.: Bossnas: Exploring hybrid cnn-transformers with block-wisely self-supervised neural architecture search. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 12281–12291 (2021) 6 find
  45. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision. pp. 2980–2988 (2017) 8
  46. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European conference on computer vision. pp. 740–755. Springer (2014) 1, 8
  47. Liu, H., Simonyan, K., Yang, Y.: Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055 (2018) 2, 5 find
  48. Loshchilov, I., Hutter, F.: Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 (2016) 7 find
  49. Ma, N., Zhang, X., Zheng, H.T., Sun, J.: Shufflenet v2: Practical guidelines for efficient cnn architecture design. In: Proceedings of the European conference on computer vision (ECCV). pp. 116–131 (2018) 2 find
  50. Maynez, J., Narayan, S., Bohnet, B., McDonald, R.: On faithfulness and factuality in abstractive summarization. arXiv preprint arXiv:2005.00661 (2020) 3
  51. Movahedi, S., Adabinejad, M., Imani, A., Keshavarz, A., Dehghani, M., Shakery, A., Araabi, B.N.: $\łambda$-DARTS: Mitigating performance collapse by harmonizing operation selection among cells. In: The Eleventh International Conference on Learning Representations (2023), https://openreview. net/foru… find
  52. OpenAI: Gpt-4 technical report (2023) 1, 2, 9
  53. Peng, H., Du, H., Yu, H., Li, Q., Liao, J., Fu, J.: Cream of the crop: Distilling prioritized paths for one-shot neural architecture search. Advances in Neural Information Processing Systems 33 (2020) 6, 7, 8 find
  54. Pham, H., Guan, M., Zoph, B., Le, Q., Dean, J.: Efficient neural architecture search via parameters sharing. In: International conference on machine learning. pp. 4095–4104. PMLR (2018) 2 find
  55. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PMLR (2021) 9
  56. Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016) 1
  57. Real, E., Moore, S., Selle, A., Saxena, S., Suematsu, Y.L., Tan, J., Le, Q.V., Kurakin, A.: Large-scale evolution of image classifiers. In: International Conference on Machine Learning. pp. 2902–2911. PMLR (2017) 2 find
  58. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2018) 2, 4, 6, 8 find
  59. Shallue, C.J., Vanderburg, A.: Identifying exoplanets with deep learning: A five-planet resonant chain around kepler-80 and an eighth planet around kepler-90. The Astronomical Journal 155(2), 94 (2018) 1 find
  60. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015) 2
  61. Stokes, J.M., Yang, K., Swanson, K., Jin, W., Cubillos-Ruiz, A., Donghia, N.M., MacNair, C.R., French, S., Carfrae, L.A., Bloom-Ackermann, Z., et al.: A deep learning approach to antibiotic discovery. Cell 180(4), 688–702 (2020) 1 find
  62. Su, X., Huang, T., Li, Y., You, S., Wang, F., Qian, C., Zhang, C., Xu, C.: Prioritized architecture sampling with monto-carlo tree search. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10968–10977 (2021) 2, 3, 6 find
  63. Su, X., You, S., Xie, J., Wang, F., Qian, C., Zhang, C., Xu, C.: Searching for network width with bilaterally coupled network. IEEE Transactions on Pattern Analysis and Machine Intelligence pp. 1–17 (2022). https://doi.org/10.1109/TPAMI.2022.3226777 4, 6 — [doi:10.1109/tpami.2022.3226777](https://doi.org/10.1109/tpami.2022.3226777) ↗
  64. Su, X., You, S., Xie, J., Zheng, M., Wang, F., Qian, C., Zhang, C., Wang, X., Xu, C.: Vitas: Vision transformer architecture search. In: Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXI. pp. 139–157. Springer (2022) 2 find
  65. Su, X., You, S., Zheng, M., Wang, F., Qian, C., Zhang, C., Xu, C.: K-shot nas: Learnable weight-sharing for nas with k-shot supernets. In: International Conference on Machine Learning. pp. 9880–9890. PMLR (2021) 2, 6 find
  66. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 1–9 (2015). https://doi.org/10.1109/CVPR.2015.7298594 2 ↗
  67. Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., Le, Q.V.: Mnasnet: Platform-aware neural architecture search for mobile. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 2820–2828 (2019) 2, 6, 8 find
  68. Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International conference on machine learning. pp. 6105–6114. PMLR (2019) 2, 6, 7, 8
  69. Tan, M., Le, Q.V.: Mixconv: Mixed depthwise convolutional kernels. arXiv preprint arXiv:1907.09595 (2019) 8 find
  70. Turpin, M., Michael, J., Perez, E., Bowman, S.R.: Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. arXiv preprint arXiv:2305.04388 (2023) 8
  71. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) 1, 2
  72. Wan, A., Dai, X., Zhang, P., He, Z., Tian, Y., Xie, S., Wu, B., Yu, M., Xu, T., Chen, K., et al.: Fbnetv2: Differentiable neural architecture search for spatial and channel dimensions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 12965–12974 (2020) 6 find
  73. Wiener, N.: Some moral and technical consequences of automation. Science 131 3410, 1355–8 (1960) 9 find
  74. Wightman, R.: Pytorch image models. https://github.com/rwightman/ pytorch-image-models (2019). https://doi.org/10.5281/zenodo.4414861 6 — [doi:10.5281/zenodo.4414861](https://doi.org/10.5281/zenodo.4414861) ↗
  75. Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1492– 1500 (2017) 2 find
  76. Xie, S., Zheng, H., Liu, C., Lin, L.: Snas: stochastic neural architecture search. arXiv preprint arXiv:1812.09926 (2018) 5 — [arXiv:1812.09926](https://arxiv.org/abs/1812.09926) find
  77. Xu, Y., Xie, L., Zhang, X., Chen, X., Qi, G.J., Tian, Q., Xiong, H.: Pc-darts: Partial channel connections for memory-efficient architecture search. arXiv preprint arXiv:1907.05737 (2019) 5 find
  78. Ye, P., Li, B., Li, Y., Chen, T., Fan, J., Ouyang, W.: b-darts: Beta-decay regularization for differentiable architecture search. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10874–10883 (2022) 5 find
  79. You, S., Huang, T., Yang, M., Wang, F., Qian, C., Zhang, C.: Greedynas: Towards fast one-shot nas with greedy supernet. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1999–2008 (2020) 2, 6, 7 find
  80. Yu, J., Jin, P., Liu, H., Bender, G., Kindermans, P.J., Tan, M., Huang, T., Song, X., Pang, R., Le, Q.: Bignas: Scaling up neural architecture search with big single-stage models. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VII 16. pp.… find
  81. Zhang, H., Wu, C., Zhang, Z., Zhu, Y., Zhang, Z., Lin, H., Sun, Y., He, T., Muller, J., Manmatha, R., Li, M., Smola, A.: Resnest: Split-attention networks. arXiv preprint arXiv:2004.08955 (2020) 2 — [arXiv:2004.08955](https://arxiv.org/abs/2004.08955) find
  82. Zhang, M., Su, S.W., Pan, S., Chang, X., Abbasnejad, E.M., Haffari, R.: idarts: Differentiable architecture search with stochastic implicit gradients. In: International Conference on Machine Learning. pp. 12557– 12566. PMLR (2021) 5 find
  83. Zhang, X., Zhou, X., Lin, M., Sun, J.: Shufflenet: An extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 6848–6856 (2018) 2 find
  84. Zoph, B., Le, Q.: Neural architecture search with reinforcement learning. In: International Conference on Learning Representations (2017), https://openreview.net/forum?id=r1Ue8Hcxg 2 ↗
  85. Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 8697–8710 (2018) 2, 8

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zheng-2023-gpt-4-perform-neural-architecture/