Elamur Research⌕
☰
home / papers / 2017 / PathNet: Evolution Channels Gradient Descent in Super Neural Networks

PathNet: Evolution Channels Gradient Descent in Super Neural Networks

Paper · 2017 · Chrisantha Fernando, Dylan Banarse, Charles Blundell et al.

Summary

The paper asks how one very large neural network could learn many tasks one after another. The goal is to reuse earlier knowledge without overwriting it, which is known as catastrophic forgetting. The authors propose PathNet, a modular network of several layers, each with many small modules. A population of "pathways" is defined, where each pathway selects a few modules per layer. Only the selected modules take part in forward and backward passes. A tournament-selection genetic algorithm copies and mutates the better-performing pathways, while gradient descent (SGD, or A3C for reinforcement learning) trains the weights inside them. Once a task is learned, the winning pathway is frozen, the remaining parameters are reset, and fresh pathways evolve for the next task. Experiments cover binary MNIST, CIFAR, SVHN, Atari games and Labyrinth levels. In all of these domains, PathNet transferred knowledge to the second task better than learning from scratch or fine-tuning. It also made A3C less sensitive to hyperparameter choices. The work suggests that evolving which parameters to train is a practical route to continual and transfer learning.

Key points

  • Supervised architecture: For the binary MNIST tasks, PathNet has L = 3 layers with M = 10 modules each, and each module has 20 ReLU neurons. A pathway may use at most 3 modules per layer. Each task gets its own linear readout. CIFAR and SVHN use a larger version with M = 20 modules per layer.
  • Evolution setup: The population holds 64 genotypes, and selection is a binary tournament. During mutation, each genotype element changes with probability 1/(N×L) by an integer in [−2, 2]. For MNIST, evaluating one pathway means training it with SGD (learning rate 0.0001) on 50 mini-batches of 16 examples.
  • Harder MNIST variant: The binary MNIST tasks add 50% salt-and-pepper noise. Each task is trained until it reaches perfect accuracy on the training set.
  • Reinforcement learning setup: All 64 A3C workers evaluate pathways in parallel, so each worker updates only its own subset of parameters. The illustrative experiment trains Pong and then Alien, for 80M timesteps each.
  • Labyrinth results: Performance was measured as area under the learning curve, compared against a de novo baseline. Fine-tuning averaged a ratio of 1.00, while PathNet averaged 1.26. PathNet also outperformed the controls on seekavoid arena across a sweep of 243 hyperparameter settings.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

Cite

Cite as Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel and Daan Wierstra. 2017. PathNet: Evolution Channels Gradient Descent in Super Neural Networks. arXiv:1701.08734.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1701.08734 · arXiv 1701.08734

@misc{fernando2017pathnet,
  title = {{PathNet: Evolution Channels Gradient Descent in Super Neural Networks}},
  author = {Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel and Daan Wierstra},
  year = {2017},
  doi = {10.48550/arxiv.1701.08734},
  eprint = {1701.08734},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1701.08734}
}

References

22 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. J. E. Auerbach, C. Fernando, and D. Floreano. Online extreme evolutionary learning machines. In Artificial Life 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems, number EPFL-CONF-200273, pages 465–472. The MIT Press, 2014. find
  2. R. Caruana. Multitask learning. In Learning to learn, pages 95–133. Springer, 1998. find
  3. T. Chen, I. Goodfellow, and J. Shlens. Net2net: Accelerating learning via knowledge transfer. arXiv preprint arXiv:1511.05641, 2015. — [arXiv:1511.05641](https://arxiv.org/abs/1511.05641) find
  4. C. Fernando, V. Vasas, E. Szathmáry, and P. Husbands. Evolvable neuronal paths: a novel basis for information and search in the brain. PloS one, 6(8):e23534, 2011. find
  5. C. T. Fernando, E. Szathmary, and P. Husbands. Selectionist and evolutionary approaches to brain function: a critical appraisal. Frontiers in computational neuroscience, 6:24, 2012. find
  6. M. J. Frank, B. Loughry, and R. C. OâĂŹReilly. Interactions between frontal cortex and basal ganglia in working memory: a computational model. Cognitive, Affective, & Behavioral Neuroscience, 1(2):137–160, 2001. find
  7. I. Harvey. The microbial genetic algorithm. In Advances in artificial life. Darwin Meets von Neumann, pages 126–133. Springer, 2011. find
  8. T. E. Hazy, M. J. Frank, and R. C. O’Reilly. Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1485):1601–1613, 2007. find
  9. G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
  10. M. Jaderberg, V. Mnih, W. M. Czarnecki, T. Schaul, J. Z. Leibo, D. Silver, and K. Kavukcuoglu. Reinforcement learning with unsupervised auxiliary tasks. arXiv preprint arXiv:1611.05397, 2016.
  11. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998. find
  12. V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. P. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu. Asynchronous methods for deep reinforcement learning. arXiv preprint arXiv:1602.01783, 2016. find
  13. Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng. Reading digits in natural images with unsupervised feature learning. find
  14. R. C. O’Reilly and M. J. Frank. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural computation, 18(2):283–328, 2006. find
  15. I. Osband, C. Blundell, A. Pritzel, and B. Van Roy. Deep exploration via bootstrapped dqn. arXiv preprint arXiv:1602.04621, 2016.
  16. M. B. Ring. Continual Learning in Reinforcement Environments. PhD thesis, University of Texas at Austin, 1994. find
  17. A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016. find
  18. S. Saxena and J. Verbeek. Convolutional neural fabrics. arXiv preprint arXiv:1606.02492, 2016. — [arXiv:1606.02492](https://arxiv.org/abs/1606.02492) find
  19. N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean. Outrageously large neural networks: A sparsely-gated mixture-of-experts layer. 2017. [ICAR submission]. find
  20. S. Singh, D. Hoiem, and D. Forsyth. Swapout: Learning an ensemble of deep architectures. arXiv preprint arXiv:1605.06465, 2016. — [arXiv:1605.06465](https://arxiv.org/abs/1605.06465) find
  21. N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929–1958, 2014.
  22. M. E. Taylor and P. Stone. An introduction to intertask transfer for reinforcement learning. AI Magazine, 32(1):15, 2011. Score Pong find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/fernando-2017-pathnet-evolution-channels-gradient-descent/