Elamur Research⌕
☰
home / papers / 2017 / Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

Paper · 2017 · Chelsea Finn, Pieter Abbeel and Sergey Levine

Summary

The paper proposes a way to train a neural network to quickly master a new task from just a few examples, i.e., in the few-shot regime. The authors present the MAML (Model-Agnostic Meta-Learning) algorithm. It is architecture-independent and suitable for any model trained with gradient descent. The model is given many related tasks and is trained not to solve each of them, but to find initial parameters θ from which one or a few gradient steps on a small sample yield good performance on a new task. In other words, MAML prepares a model that is easy to fine-tune. This requires neither new parameters nor a special architecture, such as a recurrent or Siamese network. One only needs to compute a "gradient of a gradient". The method was evaluated on three types of problems: regression, image classification, and reinforcement learning. On two few-shot classification benchmarks it achieved state-of-the-art results, and in RL it adapted considerably faster than standard pretraining. The paper is important because it offers a simple and universal foundation for meta-learning that can be applied to almost any model.

Key points

  • Method: the inner step adapts the parameters to a specific task (step α), while the outer step updates the shared initialization θ based on performance after adaptation (step β, SGD). The meta-gradient requires Hessian-vector products, so the authors also compare the method with a first-order approximation.
  • RL setup: the policy is a network with two hidden layers of 100 units each and ReLU. The inner update uses REINFORCE, the meta-optimizer is TRPO, and Hessian-vector products are computed with finite differences.
  • 2D Navigation: the agent was meta-trained for a single policy gradient step from 20 trajectories. At test time it performed up to 4 updates of 40 samples each and, already after the first step, clearly outperformed pretraining and random initialization.
  • Locomotion in MuJoCo (half-cheetah and 3D "ant"): the agent had to run at a target velocity (0–2 for cheetah, 0–3 for ant) or in a target direction. Horizon H = 200, 20 rollouts per step. MAML adapts within 2–3 steps, whereas pretraining is sometimes worse than even random initialization.
  • Publication: Proceedings of ICML 2017 (PMLR 70), UC Berkeley and OpenAI. Videos of the policies are available at sites.google.com/view/maml.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California, BerkeleyBerkeley · United States209 works · 244,782 citations🏢OpenAISan Francisco · United States23 works · 27,874 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Few-shot learning · 45 works

Cite

Cite as Chelsea Finn, Pieter Abbeel and Sergey Levine. 2017. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. arXiv:1703.03400.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1703.03400 · arXiv 1703.03400

@misc{finn2017model,
  title = {{Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks}},
  author = {Chelsea Finn, Pieter Abbeel and Sergey Levine},
  year = {2017},
  doi = {10.48550/arxiv.1703.03400},
  eprint = {1703.03400},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1703.03400}
}

References

41 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abadi, Martı́n, Agarwal, Ashish, Barham, Paul, Brevdo, Eugene, Chen, Zhifeng, Citro, Craig, Corrado, Greg S, Davis, Andy, Dean, Jeffrey, Devin, Matthieu, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467, 2016.
  2. Andrychowicz, Marcin, Denil, Misha, Gomez, Sergio, Hoffman, Matthew W, Pfau, David, Schaul, Tom, and de Freitas, Nando. Learning to learn by gradient descent by gradient descent. In Neural Information Processing Systems (NIPS), 2016. find
  3. Bengio, Samy, Bengio, Yoshua, Cloutier, Jocelyn, and Gecsei, Jan. On the optimization of a synaptic learning rule. In Optimality in Artificial and Biological Neural Networks, pp. 6–8, 1992. find
  4. Bengio, Yoshua, Bengio, Samy, and Cloutier, Jocelyn. Learning a synaptic learning rule. Université de Montréal, Département d’informatique et de recherche opérationnelle, 1990. find
  5. Donahue, Jeff, Jia, Yangqing, Vinyals, Oriol, Hoffman, Judy, Zhang, Ning, Tzeng, Eric, and Darrell, Trevor. Decaf: A deep convolutional activation feature for generic visual recognition. In International Conference on Machine Learning (ICML), 2014.
  6. Duan, Yan, Chen, Xi, Houthooft, Rein, Schulman, John, and Abbeel, Pieter. Benchmarking deep reinforcement learning for continuous control. In International Conference on Machine Learning (ICML), 2016a. find
  7. Duan, Yan, Schulman, John, Chen, Xi, Bartlett, Peter L, Sutskever, Ilya, and Abbeel, Pieter. Rl2: Fast reinforcement learning via slow reinforcement learning. arXiv preprint arXiv:1611.02779, 2016b.
  8. Edwards, Harrison and Storkey, Amos. Towards a neural statistician. International Conference on Learning Representations (ICLR), 2017. find
  9. Goodfellow, Ian J, Shlens, Jonathon, and Szegedy, Christian. Explaining and harnessing adversarial examples. International Conference on Learning Representations (ICLR), 2015.
  10. Ha, David, Dai, Andrew, and Le, Quoc V. Hypernetworks. International Conference on Learning Representations (ICLR), 2017. find
  11. Hochreiter, Sepp, Younger, A Steven, and Conwell, Peter R. Learning to learn using gradient descent. In International Conference on Artificial Neural Networks. Springer, 2001. find
  12. Husken, Michael and Goerick, Christian. Fast learning for problem classes using knowledge based network initialization. In Neural Networks, 2000. find
  13. IJCNN 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on, volume 6, pp. 619–624. find
  14. Ioffe, Sergey and Szegedy, Christian. Batch normalization: Accelerating deep network training by reducing internal covariate shift. International Conference on Machine Learning (ICML), 2015. find
  15. Kaiser, Lukasz, Nachum, Ofir, Roy, Aurko, and Bengio, Samy. Learning to remember rare events. International Conference on Learning Representations (ICLR), 2017. find
  16. Kingma, Diederik and Ba, Jimmy. Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR), 2015.
  17. Kirkpatrick, James, Pascanu, Razvan, Rabinowitz, Neil, Veness, Joel, Desjardins, Guillaume, Rusu, Andrei A, Milan, Kieran, Quan, John, Ramalho, Tiago, Grabska-Barwinska, Agnieszka, et al. Overcoming catastrophic forgetting in neural networks. arXiv preprint arXiv:1612.00796, 2016. find
  18. Koch, Gregory. Siamese neural networks for one-shot image recognition. ICML Deep Learning Workshop, 2015. find
  19. Krähenbühl, Philipp, Doersch, Carl, Donahue, Jeff, and Darrell, Trevor. Data-dependent initializations of convolutional neural networks. International Conference on Learning Representations (ICLR), 2016. find
  20. Lake, Brenden M, Salakhutdinov, Ruslan, Gross, Jason, and Tenenbaum, Joshua B. One shot learning of simple visual concepts. In Conference of the Cognitive Science Society (CogSci), 2011. find
  21. Li, Ke and Malik, Jitendra. Learning to optimize. International Conference on Learning Representations (ICLR), 2017. find
  22. Maclaurin, Dougal, Duvenaud, David, and Adams, Ryan. Gradient-based hyperparameter optimization through reversible learning. In International Conference on Machine Learning (ICML), 2015. find
  23. Munkhdalai, Tsendsuren and Yu, Hong. Meta networks. International Conferecence on Machine Learning (ICML), 2017. find
  24. Naik, Devang K and Mammone, RJ. Meta-neural networks that learn by learning. In International Joint Conference on Neural Netowrks (IJCNN), 1992. find
  25. Parisotto, Emilio, Ba, Jimmy Lei, and Salakhutdinov, Ruslan. Actor-mimic: Deep multitask and transfer reinforcement learning. International Conference on Learning Representations (ICLR), 2016. find
  26. Ravi, Sachin and Larochelle, Hugo. Optimization as a model for few-shot learning. In International Conference on Learning Representations (ICLR), 2017. find
  27. Rei, Marek. Online representation learning in recurrent neural language models. arXiv preprint arXiv:1508.03854, 2015. find
  28. Rezende, Danilo Jimenez, Mohamed, Shakir, Danihelka, Ivo, Gregor, Karol, and Wierstra, Daan. One-shot generalization in deep generative models. International Conference on Machine Learning (ICML), 2016. find
  29. Salimans, Tim and Kingma, Diederik P. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In Neural Information Processing Systems (NIPS), 2016. find
  30. Santoro, Adam, Bartunov, Sergey, Botvinick, Matthew, Wierstra, Daan, and Lillicrap, Timothy. Meta-learning with memory-augmented neural networks. In International Conference on Machine Learning (ICML), 2016. find
  31. Saxe, Andrew, McClelland, James, and Ganguli, Surya. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. International Conference on Learning Representations (ICLR), 2014.
  32. Schmidhuber, Jurgen. Evolutionary principles in self-referential learning. On learning how to learn: The meta-meta-... hook.) Diploma thesis, Institut f. Informatik, Tech. Univ. Munich, 1987. find
  33. Schmidhuber, Jürgen. Learning to control fast-weight memories: An alternative to dynamic recurrent networks. Neural Computation, 1992. find
  34. Schulman, John, Levine, Sergey, Abbeel, Pieter, Jordan, Michael I, and Moritz, Philipp. Trust region policy optimization. In International Conference on Machine Learning (ICML), 2015. find
  35. Shyam, Pranav, Gupta, Shubham, and Dukkipati, Ambedkar. Attentive recurrent comparators. International Conferecence on Machine Learning (ICML), 2017. find
  36. Snell, Jake, Swersky, Kevin, and Zemel, Richard S. Prototypical networks for few-shot learning. arXiv preprint arXiv:1703.05175, 2017. find
  37. Thrun, Sebastian and Pratt, Lorien. Learning to learn. Springer Science & Business Media, 1998. find
  38. Todorov, Emanuel, Erez, Tom, and Tassa, Yuval. Mujoco: A physics engine for model-based control. In International Conference on Intelligent Robots and Systems (IROS), 2012. find
  39. Vinyals, Oriol, Blundell, Charles, Lillicrap, Tim, Wierstra, Daan, et al. Matching networks for one shot learning. In Neural Information Processing Systems (NIPS), 2016.
  40. Wang, Jane X, Kurth-Nelson, Zeb, Tirumala, Dhruva, Soyer, Hubert, Leibo, Joel Z, Munos, Remi, Blundell, Charles, Kumaran, Dharshan, and Botvinick, Matt. Learning to reinforcement learn. arXiv preprint arXiv:1611.05763, 2016. — [arXiv:1611.05763](https://arxiv.org/abs/1611.05763) find
  41. Williams, Ronald J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229–256, 1992. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/finn-2017-model-agnostic-meta-learning-fast/