Elamur Research
☰
home / papers / 2018 / Continual Lifelong Learning with Neural Networks: A Review

Continual Lifelong Learning with Neural Networks: A Review

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/1802.07569

Summary

Neural networks trained on data that arrives over time, rather than as one fixed batch, tend to lose earlier knowledge when they learn new tasks. This is called catastrophic forgetting or interference. Retraining on all data each time works but is slow and rules out real-time adaptation. This makes it a serious obstacle for robots and agents in changing environments. Parisi and colleagues survey this problem through the stability-plasticity dilemma. They first review biological mechanisms of lifelong learning: Hebbian and homeostatic plasticity, and the complementary learning systems (CLS) theory of how hippocampus and neocortex share memory work. They then compare three groups of computational methods: regularizing plasticity, growing the network, and replaying experiences through complementary memory systems. They also discuss how to evaluate these methods. Their conclusion is that current models are mostly supervised and tested in controlled settings, so they still lack the robustness and scale of biological learners. They argue that future work should bring together developmental stages, curricula, knowledge transfer between tasks, self-driven exploration and the integration of multiple senses.

Key points

  • Authors are from Universität Hamburg, Rochester Institute of Technology and Heriot-Watt University. The review was published in Neural Networks (2019), arXiv:1802.07569, and was partly funded by DFG project Transregio Crossmodal Learning (TRR 169).
  • Methods against forgetting fall into three groups:
    • regulating synaptic plasticity (e.g., Kirkpatrick et al. 2017; Zenke, Poole & Ganguli 2017);
    • dynamic architectures that add neurons or layers;
    • CLS-style dual-memory models with experience replay.
  • Most methods target supervised learning on finite labelled datasets. Unsupervised lifelong learning has been explored mainly with self-organizing networks.
  • The authors call for quantitative metrics of catastrophic forgetting tested on large-scale datasets. They note that rigorous evaluation has lagged behind the design of new methods.
  • Section 4 covers biologically inspired directions:
    • critical developmental periods and curriculum learning;
    • transfer learning;
    • intrinsically motivated reinforcement learning;
    • crossmodal and multisensory learning (e.g., audio-visual transfer by Moon et al. 2015, Barros et al. 2017).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Cited by

130 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 126 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1016/j.neunet.2019.01.012. · arXiv 1802.07569

@misc{parisi2018continual,
  title = {{Continual Lifelong Learning with Neural Networks: A Review}},
  author = {Parisi, German I. and Kemker, Ronald and Part, Jose L. and Kanan, Christopher and Wermter, Stefan},
  year = {2018},
  doi = {10.1016/j.neunet.2019.01.012.},
  eprint = {1802.07569},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1016/j.neunet.2019.01.012.}
}

References

192 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abbott, L. F. & Nelson, S. B. (2000), ‘Synaptic plasticity: taming the beast’, Nature Neuroscience 3, 1178–1183. find
  2. Abraham, W. C. & Robins, A. (2005), ‘Memory retention and weight plasticity in ANN simulations’, Trends in Neurosciences 28(2), 73–78. find
  3. Achille, A., Rovere, M. & Soatto, S. (2017), Critical learning periods in deep neural networks, arXiv:1711.08856. — [arXiv:1711.08856](https://arxiv.org/abs/1711.08856) find
  4. Adams, R. A., Friston, K. J. & Bastos, A. M. (2015), ‘Active inference, predictive coding and cortical architecture’, In M. F. Casanova and I. Opris (Eds.), Recent Advances on the Modular Organization of the Cortex. Springer Netherlands . find
  5. Aimone, J. B., Wiles, J. & Gage, F. H. (2006), ‘Potential role for adult neurogenesis in the encoding of time in new memories.’, Nature Neuroscience 9, 723–727. find
  6. Aimone, J. B., Wiles, J. & Gage, F. H. (2009), ‘Computational influence of adult neurogenesis on memory encoding.’, Neuron 61, 187–202. find
  7. Altman, J. (1963), ‘Autoradiographic investigation of cell proliferation in the brains of rats and cats’, The Anatomical Record 145(4), 573–591. find
  8. Aström, K. J. & Murray, R. M. (2010), Feedback Systems: An Introduction for Scientists and Engineers, Princeton University Press. find
  9. Baldassarre, G. & Mirolli, M. (2013), Intrinsically motivated learning in natural and artificial systems, Springer-Verlag, Berlin. find
  10. Baranes, A. & Oudeyer, P.-Y. (2013), ‘Active learning of inverse models with intrinsically motivated goal exploration in robots’, Robotics and Autonomous Systems 61(1), 49–73. find
  11. Barnett, S. & Ceci, S. (2002), ‘When and where do we apply what we learn? a taxonomy for far transfer’, Psychological Bulletin 128, 612–637. find
  12. Barros, P., Parisi, G. I., Fu, D., Liu, X. & Wermter, S. (2017), Expectation learning for adaptive crossmodal stimuli association, EUCog Meeting Proceedings, Zurich, Switzerland. find
  13. Barto, A. (2013), Intrinsic motivation and reinforcement learning, Baldassarre, G., Mirolli, M. (Eds.), Intrinsically Motivated Learning in Natural and Artificial Systems. find
  14. Springer. Bellemare, M., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D. & Munos, R. (2016), Unifying count-based exploration and intrinsic motivation. find
  15. Bengio, Y., Louradour, J., Collobert, R. & Weston, J. (2009), Curriculum learning, pp. 41–48. find
  16. Benna, M. K. & Fusi, S. (2016), ‘Computational principles of synaptic memory consolidation’, Nature Neuroscience 19(12), 1697–1708. find
  17. Bienenstock, E., Cooper, L. & Munro, P. (1982), ‘Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex’, The Journal of Neuroscience 2, 32–48. Boldrini, M., Fulmore, C., Tartt, A., Simeon, L. & Pavlova, I. e. a. (2018), ‘Human hippocampa… find
  18. Bontempi, B., Laurent-Demir, C., Destrade, C. & Jaffard, R. (1999), ‘Time-dependent reorganization of brain circuitry underlying long-term memory storage’, Nature 400, 671–675. find
  19. Braun, C., Heinz, U., Schweizer, R., Weich, K., Birbaumer, N. & Topka, H. (2001), ‘Dynamic organization of the somato-sensory cortex induced by motor activity’, Brain 124, 2259–2267. find
  20. Bremner, A., Lewkowicz, D. & Spence, C. (2012), ‘Multisensory development’, Oxford University Press, Oxford, UK . Burgess, N., Shapiro, J. L. & Moore, M. A. (1991), ‘Neural network models of list learning’, Networks 2, 399–422. find
  21. Calvert, G., Spence, C. & Stein, B. (2004), The Handbook of Multisensory Processes, Cambridge, MA: The MIT Press. find
  22. Cameron, H. A., Woolley, C. S., McEwen, B. S. & Gould, E. (1993), ‘Differentiation of newly born neurons and glia in the dentate gyrus of the adult rat’, Neuroscience 56(2), 337–344. find
  23. Cangelosi, A. & Schlesinger, M. (2015), Developmental robotics: From babies to robots, MIT Press. find
  24. Chen, Y.-C. & Spence, C. (2017), ‘Assessing the role of the unity assumption on multisensory integration: A review’, Frontiers in Psychology 8(445). find
  25. Cichon, J. & Gan, W. (2015), ‘Branch-specific dendritic ca(2+) spikes cause persistent synaptic plasticity’, Nature 520, 180–185. find
  26. Coop, R., Mishtal, A. & Arel, I. (2013), ‘Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting’, IEEE Transactions on Neural Networks and Learning Systems 24(10), 1623–1634. find
  27. Cortes, C., Gonzalvo, X., Kuznetsov, V., Mohri, M. & Yang, S. (2016), AdaNet: Adaptive structural learning of artificial neural networks, arXiv:1607.01097. — [arXiv:1607.01097](https://arxiv.org/abs/1607.01097) find
  28. Dai, W., Yang, Q., Xue, G.-R. & Yu, Y. (2007), Boosting for transfer learning, ICML’17, Sydney, Australia, pp. 193–200. find
  29. Davis, G. W. (2006), ‘Homeostatic control of neural activity: from phenomenology to molecular design’, Annual Review of Neuroscience 29, 307–323. find
  30. Deng, W., Aimone, J. B. & Gage, F. H. (2010), ‘New neurons and new memories: how does adult hippocampal neurogenesis affect learning and memory?’, Nature Reviews Neuroscience 11(5), 339–350. find
  31. Ditzler, G., Roveri, M., Alippi, C. & Polikar, R. (2015), ‘Learning in nonstationary environments: A survey’, IEEE Computational Intelligence Magazine 10(4), 12–25. find
  32. Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E. & Darrell, T. (2014), Decaf: A deep convolutional activation feature for generic visual recognition, ICML’14, Beijing, China. find
  33. Douglas, R. J., Koch, C., Mahowald, M., Martin, K. A. & Suarez, H. H. (1995), ‘Recurrent excitation in neocortical circuits’, Science 269, 981–985. find
  34. Doumas, L., Hummel, J. & Sandhofer, C. (2008), ‘A theory of the discovery and predication of relational concepts’, Psychological Review 115, 1–43. find
  35. Draelos, T. J., Miner, N. E., Lamb, C. C., Vineyard, C. M., Carlson, K. D., James, C. D. & Aimone, J. B. (2017), Neurogenesis deep learning, IJCNN’17, Anchorage, Alaska, pp. 526–533. find
  36. Elfaramawy, N., Barros, P., Parisi, G. I. & Wermter, S. (2017), Emotion recognition from body expressions with a neural network architecture, Proceedings of the International Conference on Human Agent Interaction (HAI’17), Bielefeld, Germany, pp. 143–149. find
  37. Elman, J. L. (1993), ‘Learning and development in neural networks: The importance of starting small’, Cognition 48(1), 71–99. find
  38. Eriksson, P. S., Perfilieva, E., Bjork-Eriksson, T., Alborn, A.-M., Nordborg, C., Peterson, D. A. & Gage, F. H. (1998), ‘Neurogenesis in the adult human hippocampus’, Nature medicine 4(11), 1313–1317. find
  39. Ernst, M. O. & Bülthoff, H. (2004), ‘Merging the senses into a robust percept’, Trends in Cognitive Sciences 8(4), 162–169. find
  40. Fei-Fei, L., Fergus, R. & Perona, P. (2003), A bayesian approach to unsupervised one-shot learning of object categories, ICCV’03, Nice, France. find
  41. Fernando, C., Banarse, D., Blundell, C., Zwols, Y., Ha, D., Rusu, A. A., Pritzel, A. & Wierstra, D. (2017), Pathnet: Evolution channels gradient descent in super neural networks, arXiv:1701.08734. find
  42. Forestier, S., Mollard, Y. & Oudeyer, P.-Y. (2017), Intrinsically motivated goal exploration processes with automatic curriculum learning, arXiv:1708.02190. find
  43. Forestier, S. & Oudeyer, P.-Y. (2016), Curiosity-driven development of tool use precursors: a computational model, Proceedings of the Annual Conference of the Cognitive Science Society. find
  44. French, R. M. (1997), ‘Pseudo-recurrent connectionist networks: An approach to the sensitivity-stability dilemma’, Connection Science 9(4), 353–380. find
  45. French, R. M. (1999), ‘Catastrophic forgetting in connectionist networks’, Trends in Cognitive Sciences 3(4), 128–135. find
  46. Friston, K., Rigoli, F., Ognibene, D., Mathys, D., Fitzgerald, T. & Pezzulo, G. (2015), ‘Active inference and epistemic value’, Cognitive neuroscience 6, 187–214. find
  47. Fritzke, B. (1992), A growing neural gas network learns topologies, Vol. 7, NIPS’95, Denver, CO, pp. 625–632. find
  48. Fusi, S., Drew, P. J. & Abbott, L. F. (2005), ‘Cascade models of synaptically stored memories’, Neuron 45(4), 599–611. find
  49. Gage, F. H. (2000), ‘Mammalian neural stem cells’, Science 287, 1433–1438. find
  50. Gais, S., Albouy, G., Boly, M., Dang-Vu, T. T., Darsaud, A., Desseilles, M., Rauchs, G., Schabus, M., Sterpenich, V. & Vandewalle, G. e. a. (2007), ‘Sleep transforms the cerebral trace of declarative memories’, Proceedings of the National Academy of Sciences 104(47), 18778–18783. find
  51. Gelbard-Sagiv, H., Mukamel, R., Harel, M., Malach, R. & Fried, I. (2008), ‘Internally generated reactivation of single neurons in human hippocampus during free recall’, Science 322, 96–101. find
  52. Gemmeke, J. F., Ellis, D. P. W., Freedman, D., Jansen, A., Lawrence, W., Moore, R. C., Plakal, M. & Ritter, M. (2017), Audio set: An ontology and human-labeled dataset for audio events, ICASSP’17, New Orleans, LA. find
  53. Gepperth, A. & Karaoguz, C. (2015), ‘A bio-inspired incremental learning architecture for applied perceptual problems’, Cognitive Computation 8(5), 924–934. find
  54. Goodfellow, I. J., Mirza, M., Xiao, D., Courville, A. & Bengio, Y. (2013), An empirical investigation of catastrophic forgetting in gradient-based neural networks, arXiv:1312.6211. find
  55. Gopnik, A., Meltzoff, A. N. & Kuhl, P. K. (1999), The scientist in the crib: Minds, brains, and how children learn, William Morrow & Co. find
  56. Gorelick, L., Blank, M., Shechtman, E., Irani, M. & Basri, R. (2005), Actions as space-time shapes, ICCV’05, Beijing, China, pp. 1395–1402. find
  57. Gottlieb, J., Oudeyer, P.-Y., Lopes, M. & Baranes, A. (2013), ‘Information seeking, curiosity and attention: Computational and neural mechanisms’, Trends in Cognitive Science 17(11), 585–596. find
  58. Grant, W. S., Tanner, J. & Itti, L. (2017), ‘Biologically plausible learning in neural networks with modulatory feedback’, Neural Networks 88, 32–48. find
  59. Graves, A., Bellemare, M. G., Menick, J., Munos, R. & Kavukcuoglu, K. (2017), Automated curriculum learning for neural networks, arXiv:1704.03003. Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwinska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T. & Agapiou, J. e.… find
  60. Grossberg, S. (1980), ‘How does a brain build a cognitive code?’, Psychol. Rev. 87, 1–51. find
  61. Grossberg, S. (2012), ‘Adaptive resonance theory: how a brain learns to consciously attend, learn, and recognize a changing world.’, Neural Networks 37, 1–41. find
  62. Guo, Y., Liu, Y., Oerlemans, A., Lao, S., Wu, S. & Lew, M. (2016), ‘Deep learning for visual understanding: A review’, Neurocomputing 187(8), 27–48. find
  63. Hassabis, D., Kumaran, D., Summerfield, C. & Botvinick, M. (2017), ‘Neuroscience-inspired artificial intelligence’, Neuron Review 95(2), 245–258. find
  64. Hasson, U., Yang, E., Vallines, I., Heeger, D. J. & Rubin, N. (2008), ‘A hierarchy of temporal receptive windows in human cortex’, The Journal of Neuroscience 28(10), 2539–2550. find
  65. Hebb, D. (1949), The Organization of Behavior, John Wiley & Sons. find
  66. Heeger, D. J. (2017), ‘Theory of cortical function’, Proceedings of the National Academy of Sciences of the United States of America 114(8), 1773–1782. find
  67. Hensch, T. (2004), ‘Critical period regulation’, Annual Review of Neuroscience 27, 549–579. find
  68. Hensch, T. K., Fagiolini, M., Mataga, N., Stryker, M. P., Baekkeskov, S. & Kash, S. F. (1998), ‘Local gaba circuit control of experience-dependent plasticity in developing visual cortex’, Science 282, 1504–1508. find
  69. Hertz, J., Krogh, A. & Palmer, R. G. (1991), Introduction to the Theory of Neural Computation, Redwood City CA: Addison-Wesley. find
  70. Hinton, G. E. & Plaut, D. C. (1987), Using fast weights to deblur old memories, Proceedings of the Annual Conference of the Cognitive Science Society, pp. 177–186. find
  71. Hinton, G., Vinyals, O. & Dean, J. (2014), Distilling the knowledge in a neural network, NIPS’14, Workshop on Deep Learning and Representation, Montreal, Canada. find our card
  72. Hirsch, H. & Spinelli, D. (1970), ‘Visual experience modifies distribution of horizontally and vertically oriented receptive fields in cats’, Science 168, 869–871. find
  73. Holyoak, K. & Thagard, P. (1997), ‘The analogical mind’, American Psychologist 52, 35–44. find
  74. Hubel, D. H. & Wiesel, T. H. (1962), ‘Receptive fields, binocular and functional architecture in the cats visual cortex’, Journal of Physiology 160, 106–154. find
  75. Hubel, D. H. & Wiesel, T. H. (1967), ‘Cortical and callosal connections concerned with the vertical meridian of visual fields in the cat’, Journal of Neurophysiology 30, 1561–1573. find
  76. Hubel, D. H. & Wiesel, T. H. (1970), ‘The period of susceptibility to the psychological effects of unilateral eye closure in kittens’, Journal of Physiology 206, 419–436. find
  77. Hubel, D. H., Wiesel, T. H. & LeVay, S. (1977), ‘Plasticity of ocular dominance columns in monkey striate cortex’, Philosophical Transactions of the Royal Society of London, Series B: Biological Sciences 278, 377–409. find
  78. Jung, H., Ju, J., Jung, M. & Kim, J. (2018), Less-forgetting learning in deep neural networks, AAAI’18, New Orleans, LA. find
  79. Kamra, N., Gupta, U. & Liu, Y. (2018), Deep generative dual memory network for continual learning, arXiv:1710.10368. — [arXiv:1710.10368](https://arxiv.org/abs/1710.10368) find
  80. Kemker, R. & Kanan, C. (2018), Fearnet: Brain-inspired model for incremental learning, ICLR’18, Vancouver, Canada. find
  81. Kemker, R., McClure, M., Abitino, A., Hayes, T. & Kanan, C. (2018), Measuring catastrophic forgetting in neural networks, AAAI’18, New Orleans, LA. find
  82. Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., Hassabis, D., Clopath, C., Kumaran, D. & Hadsell, R. (2017), ‘Overcoming catastrophic forgetting in neural networks’, Proceedings of the National Academy of… find
  83. Kitamura, T., Ogawa, S. K., Roy, D. S., Okuyama, T., Morrissey, M. D., Smith, L. M., Redondo, R. L. & Tonegawa, S. (2017), ‘Engrams and circuits crucial for systems consolidation of a memory’, Science 356, 73–78. find
  84. Kiyota, T. (2017), Neurogenesis and brain repair, Neuroimmune Pharmacology. Springer. find
  85. Knoblauch, A. . (2017), Impact of structural plasticity on memory formation and decline, Rewiring the Brain: A Computational Approach to Structural Plasticity in the Adult Brain, eds A. van Ooyen and M. Butz, Elsevier, Academic Press. find
  86. Knoblauch, A., Körner, E., Krner, U. & Sommer, F. T. (2014), ‘Structural plasticity has high memory capacity and can explain graded amnesia, catastrophic forgetting, and the spacing effect.’, PLoS ONE 9:e96485. find
  87. Kohonen, T. (1982), ‘Self-organized formation of topologically correct feature maps’, Biological Cybernetics 43, 59–69. find
  88. Kohonen, T. (1995), Self-organizing maps, New York: Springer. find
  89. Krizhevsky, A. (2009), Learning multiple layers of features from tiny images, Master’s thesis, University of Toronto. find
  90. Krueger, K. A. & Dayan, P. (2009), ‘Flexible shaping: how learning in small steps helps’, Cognition 110, 380–394. find
  91. Kulkarni, T., Narasimhan, K., Saeedi, A. & Tenenbaum, J. (2016), Hierarchical deep reinforcement learning: integrating temporal abstraction and intrinsic motivation, arXiv:1604.06057. — [arXiv:1604.06057](https://arxiv.org/abs/1604.06057) find
  92. Kumaran, D., Hassabis, D. & McClelland, J. L. (2016), ‘What learning systems do intelligent agents need? complementary learning systems theory updated’, Trends in Cognitive Sciences 20(7), 512– 534. find
  93. Kumaran, D. & McClelland, J. L. (2012), ‘Generalization through the recurrent interaction of episodic memories: A model of the hippocampal system’, Psychological Review 119, 573–616. find
  94. Lambon Ralph, M. & Ehsan, S. (2006), ‘Age of acquisition effects depend on the mapping between representations and the frequency of occurrence: empirical and computational evidence’, Visual Cognition 13(7–8), 928–948. find
  95. Lampert, C., Nickisch, H. & Harmeling, S. (2009), Learning to detect unseen object classes by between-class attribute transfer, CVPR’09, Miami Beach, Florida. find
  96. LeCun, Y., Bengio, Y. & Hinton, G. (2015), ‘Deep learning’, Nature 521, 436–444. find
  97. LeCun, Y., Bottou, L., Bengio, Y. & Haffner, P. (1998), Gradient-based learning applied to document recognition, In Proceedings of the IEEE. find
  98. Lenneberg, E. (1967), Biological foundations of language, New York: Wiley. find
  99. Lewandowsky, S. & Li, S. (1994), Catastrophic Interference in Neural Networks: Causes, Solutions, and Data, Dempster, F.N. & Brainerd,C. (Eds.) New Perspectives on Interference and Inhibition in Cognition. Academic Press, New York. find
  100. Lewkowicz, D. J. (2014), ‘Early experience & multisensory perceptual narrowing’, Developmental Psychobiology 56(2), 292–315. find
  101. Li, P., Farkas, I. & McWhinney, B. (2004), ‘Early lexical development in a self-organising neural network’, Neural Networks 17, 1345–1362. find
  102. Li, Z. & Hoiem, D. (2016), Learning without forgetting, ECCV’16, Amsterdam, The Netherlands, pp. 614–629. find
  103. Lomonaco, V. & Maltoni, D. (2017), CORe50: A new dataset and benchmark for continuous object recognition, CoRL, Mountain View, CA. find
  104. Lopez-Paz, D. & Ranzato, M. (2017), Gradient episodic memory for continual learning, NIPS’17, Long Beach, CA. Lüders, B., Schläger, M. & Risi, S. (2016), Continual learning through evolvable neural turing machines, NIPS’16, Workshop on Continual Learning and Deep Networks, Barcelona, Spain. find
  105. Maltoni, D. & Lomonaco, V. (2018), Continuous learning in single-incremental-task scenarios., arXiv:1806.08568. — [arXiv:1806.08568](https://arxiv.org/abs/1806.08568) find
  106. Marchman, V. A. (1993), ‘Constraints on plasticity in a connectionist model of English past tense’, Journal of Cognitive Neuroscience 5(2), 215–234. find
  107. Mareschal, D., Johnson, M., Sirios, S., Spratling, M., Thomas, M. & Westermann, G. (2007), Neu-roconstructivism: How the brain constructs cognition, Oxford: Oxford University Press. find
  108. Marsland, S., Shapiro, J. & Nehmzow, U. (2002), ‘A self-organising network that grows when required’, Neural Networks 15(8–9), 1041–1058. find
  109. Martinetz, T., Berkovich, S. & Schulten, K. (1993), ‘Neural-gas network for vector quantization and its application to time-series prediction’, IEEE Transactions on Neural Networks 4(4), 558–569. find
  110. McClelland, J. L., McNaughton, B. L. & O’Reilly, R. C. (1995), ‘Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory’, Psychological Review 102, 419–457. find
  111. McCloskey, M. & Cohen, N. J. (1989), ‘Catastrophic interference in connectionist networks: The sequential learning problem’, The Psychology of Learning and Motivation 24, 104–169. find
  112. Mermillod, M., Bugaiska, A. & Bonin, P. (2013), ‘The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects’, Frontiers in Psychology 4(504). find
  113. Mici, L., Parisi, G. I. & Wermter, S. (2017), An incremental self-organizing architecture for sensorimotor learning and prediction, arXiv:1712.08521. — [arXiv:1712.08521](https://arxiv.org/abs/1712.08521) find
  114. Mici, L., Parisi, G. I. & Wermter, S. (2018), ‘A self-organizing neural network architecture for learning human-object interactions’, Neurocomputing 307, 14–24. find
  115. Miikkulainen, R. (1997), ‘Dyslexic and category-specific aphasic impairments in a self-organizing feature map model of the lexicon’, Brain and Language 59, 334–366. find
  116. Miller, K., Keller, J. & Stryker, M. (1989), ‘Occular dominance column development: Analysis and simulation’, Science 245, 605–615. find
  117. Miller, K. & MacKay, D. J. (1994), ‘The role of constraints in Hebbian learning’, Neural Computation 6, 100–126. find
  118. Ming, G. L. & Song, H. (2011), ‘Adult neurogenesis in the mammalian brain: Significant answers and significant questions.’, Neuron 70, 687–702. find
  119. Moon, S., Kim, S. & Wang, H. (2015), Multimodal transfer deep learning with applications in audio-visual recognition, NIPS’15, Montreal, Canada. find
  120. Murray, M. M., Lewkowicz, D. J., Amedi, A. & Wallace, M. T. (2016), ‘Multisensory processes: A balancing act across the lifespan’, Trends in Neurosciences 39, 567–579. find
  121. Nadal, J. P., Toulouse, G., Changeux, J. P. & Dehaene, S. (1986), ‘Networks of formal neurons and memory palimpsets’, Europhysics Letters 1, 535–543. find
  122. Nelson, C. A. (2000), ‘Neural plasticity and human development: The role of early experience in sculpting memory systems’, Developmental Science 3(2), 115–136. find
  123. Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B. & Ng, A. Y. (2011), Reading digits in natural images with unsupervised feature learning, NIPS’11, Workshop on deep learning and unsupervised feature learning, Granada, Spain. find
  124. O’Neill, J., Pleydell-Bouverie, B., Dupret, D. & Csicsvari, J. (2010), ‘Play it again: Reactivation of waking experience and memory’, Trends in Neuroscience 33, 220–229. find
  125. O’Reilly, R. C. (2004), The Division of Labor Between the Neocortex and Hippocampus. Connectionist Modeling in Cognitive (Neuro-)Science, George Houghton, Ed., Psychology Press. find
  126. O’Reilly, R. C. and. Norman, K. A. (2002), ‘Hippocampal and neocortical contributions to memory: Advances in the complementary learning systems framework’, Trends in Cognitive Sciences 6(12), 505–510. find
  127. O’Reilly, R. C. & Rudy, J. W. (2000), ‘Computational principles of learning in the neocortex and hippocampus’, Hippocampus 10, 389–397. find
  128. Oudeyer, P.-Y., Kaplan, F. & Hafner, V. (2007), ‘Intrinsic motivation systems for autonomous mental development’, IEEE Transactions on Evolutionary Computation 11(2), 265–286. find
  129. Palatucci, M., Pomerleau, D. A., Hinton, G. E. & Mitchell, T. (2009), Zero-shot learning with semantic output codes, NIPS’09, Vancouver, Canada. find
  130. Pallier, C., Dehaene, S., Poline, J.-B., LeBihan, D., Argenti, A.-M., Dupoux, E. & Mehler, J. (2003), ‘Brain imaging of language plasticity in adopted adults: can a second language replace a first?’, Cerebral Cortex 13, 155–161. find
  131. Pan, S. J. & Yang, Q. (2010), ‘A survey on transfer learning’, IEEE Transactions on Knowledge and Data Engineering 22(10), 1345–1359. find
  132. Parisi, G., Barros, P., Fu, D., Magg., S., Wu, H., Liu, X. & Wermter, S. (2018), A neurorobotic experiment for crossmodal conflict resolution in complex environments, arXiv:1802.10408. — [arXiv:1802.10408](https://arxiv.org/abs/1802.10408) find
  133. Parisi, G. I., Barros, P., Kerzel, M., Wu, H., Yang, G., Li, Z., Liu, X. & Wermter, S. (2017), A computational model of crossmodal processing for conflict resolution, ICDL-EPIROB’17, Lisbon, Portugal, pp. 33–38. find
  134. Parisi, G. I., Magg, S. & Wermter, S. (2016), Human motion assessment in real time using recurrent self-organization, Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication, New York, NY, pp. 71–79. find
  135. Parisi, G. I., Tani, J., Weber, C. & Wermter, S. (2017), ‘Lifelong learning of humans actions with deep neural network self-organization’, Neural Networks 96, 137–149. find
  136. Parisi, G., Ji, X. & Wermter, S. (2018), On the role of neurogenesis in overcoming catastrophic forgetting., NIPS’18, Workshop on Continual Learning, Montreal, Canada. find
  137. Parisi, G., Tani, J., Weber, C. & Wermter, S. (2018), Lifelong learning of spatiotemporal representations with dual-memory recurrent self-organization, arXiv:1805.10966. — [arXiv:1805.10966](https://arxiv.org/abs/1805.10966) find
  138. Part, J. L. & Lemon, O. (2016), Incremental on-line learning of object classes using a combination of self-organizing incremental neural networks and deep convolutional neural networks, IROS’16, Workshop on Bio-inspired Social Robot Learning in Home Scenarios, Daejeon, South Korea. find
  139. Part, J. L. & Lemon, O. (2017), Incremental online learning of objects for robots operating in real environments, ICDL-EPIROB’17, Lisbon, Portugal. find
  140. Pathak, D., Agrawal, P., Efros, A. A. & Darrell, F. (2017), Curiosity-driven exploration by self-supervised prediction, ICML’17, Sydney, Australia. find
  141. Polikar, R., Upda, L., Upda, S. S. & Honavar, V. (2001), ‘Learn++: An incremental learning algorithm for supervised neural networks’, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 31(4), 497–508. find
  142. Power, J. D. & Schlaggar, B. L. (2016), ‘Neural plasticity across the lifespan’, Wiley Interdisciplinary Reviews: Developmental Biology 6(216). find
  143. Quadrato, G., Elnaggar, M. Y. & Di Giovanni, S. (2014), ‘Adult neurogenesis in brain repair: Cellular plasticity vs. cellular replacement’, Frontiers in Neuroscience 8(17). find
  144. Rao, R. P. N. & Ballard, D. H. (1999), ‘Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects’, Nature Neuroscience 2(1), 79–87. find
  145. Ratcliff, R. (1990), ‘Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions’, Psychological Review 97(2), 285–308. find
  146. Razavian, A. S., Azizpour, H., Sullivan, J. & Carlsson, S. (2014), CNN features off-the-shelf: An astounding baseline for recognition, CVPR’14, Columbus, OH, pp. 806–813. find
  147. Rebuffi, S.-A., Kolesnikov, A., Sperl, G. & Lampert, C. H. (2016), iCaRL: Incremental classifier and representation learning, arXiv:1611.07725. find
  148. Reed, S. & de Freitas, N. (2015), Neural programmer interpreters, arXiv:1511.06279. find
  149. Ren, B., Wang, H., Li, J. & Gao, H. (2017), ‘Life-long learning based on dynamic combination model’, Applied Soft Computing 56, 398–404. find
  150. Richardson, F. M. & Thomas, M. S. C. (2008), ‘Critical periods and catastrophic interference effects in the development of self-organising feature maps’, Developmental Science 11(3), 371–389. find
  151. Ring, M. B. (1997), ‘CHILD: A first step towards continual learning’, Machine Learning 28, 77– 104. find
  152. Robins, A. V. (1993), Catastrophic forgetting in neural networks: The role of rehearsal mechanisms, Proceedings of the First New Zealand International Two-stream Conference on Artificial Neural Networks and Expert Systems. IEEE Computer Society Press. find
  153. Robins, A. V. (1995), ‘Catastrophic forgetting, rehearsal and pseudorehearsal’, Connection Science 7(2), 123–146. find
  154. Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R. & Hadsell, R. (2016), Progressive neural networks, arXiv:1606.04671. find
  155. Rusu, A. A., Vecerik, M., Rothörl, T., Heess, N., Pascanu, R. & Hadsell, R. (2017), Sim-to-real robot learning from pixels with progressive nets, CoRL’17, Mountain View, CA. find
  156. Sanger, T. D. (1994), ‘Neural network learning control of robot manipulators using gradually increasing task difficulty’, IEEE Transactions on Robotics and Automation 10. find
  157. Schmidhuber, J. (1991), Curious model-building control systems. find
  158. Schuldt, C., Laptev, I. & Caputo, B. (2004), Recognizing human actions: A local SVM approach, ICPR’04, Cambridge, UK, pp. 32–36. find
  159. Seidenberg, M. & Zevin, J. (2006), Connectionist models in developmental cognitive neuroscience: Critical periods and the paradox of success, In Y. Munakata & M.H. Johnson (Eds.) Processes of change in brain and cognitive development: Attention and Performance XXI. Oxford: Oxford University Press. find
  160. Senghas, A., Kita, S. & Özyürek, A. (2004), ‘Children creating core properties of language: Evidence from an emerging sign language in Nicaragua’, Science 305, 1779–1782. find
  161. Shatz, C. J. (1996), ‘Emergence of order in visual system development’, Proceedings of the National Academy of Sciences 93, 602–608. find
  162. Shin, H., Lee, J. K., Kim, J. & Kim, J. (2017), Continual learning with deep generative replay, NIPS’17, Long Beach, CA. find
  163. Skinner, B. F. (1958), ‘Reinforcement today’, American Psychologist 13, 94–99. find
  164. Soltoggio, A. (2015), ‘Short-term plasticity as cause-effect hypothesis testing is distal reward learning’, Biological Cybernetics 109, 75–94. find
  165. Soltoggio, A., Stanley, K. O. & Risi, S. (2017), Born to learn: the inspiration, progress, and future of evolved plastic artificial neural networks, arXiv:1703.10371. — [arXiv:1703.10371](https://arxiv.org/abs/1703.10371) find
  166. Song, S., Miller, K. D. & Abbott, L. F. (2000), ‘Competitive Hebbian learning through spike-timing-dependent synaptic plasticity’, Nature Neuroscience 3, 919–926. find
  167. Sorrells, S. F., Paredes, M. F., Arantxa, C.-S., Sandoval, K., Dashi, Q. & Kelley, K. W. e. a. (2018), ‘Human hippocampal neurogenesis drops sharply in children to undetectable levels in adults’, Nature 555, 377–381. find
  168. Spence, C. (2010), ‘Crossmodal spatial attention’, Annals of the New York Academy of Sciences 1191, 182–200. find
  169. Spence, C. (2014), ‘Orienting attention: A crossmodal perspective’, The Oxford Handbook of Attention. Oxford, UK: Oxford University Press pp. 446–471. find
  170. Stein, B. E. & Meredith, M. A. (1993), The merging of the senses, The MIT Press, Cambridge, MA, US. find
  171. Stein, B. E., Stanford, T. R. & Rowland, B. A. (2014), ‘Development of multisensory integration from the perspective of the individual neuron’, Nature Reviews Neuroscience 15(8), 520–535. find
  172. Sur, M. & Leamey, C. A. (2001), ‘Development and plasticity of cortical areas and networks’, Nature Reviews Neuroscience 2, 251–262. find
  173. Tani, J. (2016), Exploring Robotic Minds: Actions, Symbols, and Consciousness a Self-Organizing Dynamic Phenomena, Oxford University Press. find
  174. Tanneberg, D., Peters, J. & Rueckert, E. (2017), Online learning with stochastic recurrent neural networks using intrinsic motivation signals, CoRL’17, Mountain View, CA. find
  175. Taupin, P. & Gage, F. H. (2002), ‘Adult neurogenesis and neural stem cells of the central nervous system in mammals’, Journal of neuroscience research 69(6), 745–749. find
  176. Taylor, P., Hobbs, J. N., Burroni, J. & Siegelmann, H. T. (2015), ‘The global landscape of cognition: hierarchical aggregation as an organizational principle of human cortical networks and functions’, Scientific Reports 5(18112). find
  177. Tessler, C., Givony, S., Zahavy, T., Mankowitz, D. J. & Mannor, S. (2017), A deep hierarchical approach to lifelong learning in Minecraft, AAAI’17, San Francisco, CA. find
  178. Thomas, M. & Johnson, M. (2006), ‘The computational modelling of sensitive periods’, Developmental Psychobiology 48(4), 337–344. find
  179. Thrun, S. & Mitchell, T. (1995), ‘Lifelong robot learning’, Robotics and Autonomous Systems 15, 25–46. find
  180. Tse, D., Takeuchi, T., Kakeyama, M., Kajii, Y., Okuno, H., Tohyama, C., Bito, H. & Morris, R. G. (2011), ‘Schema-dependent gene activation and memory encoding in neocortex’, Science 333, 891–895. find
  181. Turrigiano, G. (2011), ‘Too many cooks? intrinsic and synaptic homeostatic mechanisms in cortical circuit refinement’, Annual Review of Neuroscience 34, 89–103. find
  182. Uylings, H. (2006), ‘Development of the human cortex and the concept of critical or sensitive periods’, Language Learning 56, 59–90. find
  183. Vinyals, O., Blundell, C., Lillicrap, T. & Wierstra, D. (2016), Matching networks for one shot learning, NIPS’16, Barcelona, Spain. find
  184. Wah, C., Branson, S., Welinder, P., Perona, P. & Belongie, S. (2011), Caltech-UCSD Birds-200-2011 dataset, Tech Report: CNS-TR-2011-001. find
  185. Weiss, K., Khoshgoftaar, T. M. & Wang, D.-D. (2016), ‘A survey of transfer learning’, Journal of Big Data 3(9). find
  186. Wermter, S., Palm, G. & Elshaw, M. (2005), Biomimetic Neural Learning for Intelligent Robots, Intelligent Systems, Cognitive Robotics, and Neuroscience. find
  187. Springer, Berlin, Heidelberg. Willshaw, D. J. & von der Malsburg, C. (1976), ‘How patterned neural connections can be set up by self-organization’, Proceedings of the Royal Society of London B: Biological Sciences 194(1117), 431–445. find
  188. Xiao, T., Zhang, J., Yang, K., Peng, Y. & Zhang, Z. (2014), Error-driven incremental learning in deep convolutional neural network for large-scale image classification, Proceedings of the ACM International Conference on Multimedia, Orlando, FL, pp. 177–186. find
  189. Yoon, J., Yang, E. & Lee, J. Hwang, S. J. (2018), Lifelong learning with dynamically expandable networks, ICLR’18, Vancouver, Canada. find
  190. Zenke, F., Gerstner, W. & Ganguli, S. (2017), ‘The temporal paradox of hebbian learning and homeostatic plasticity’, Neurobiology 43, 166–176. find
  191. Zenke, F., Poole, B. & Ganguli, S. (2017), Continual learning through synaptic intelligence, ICML’17, Sydney, Australia. find
  192. Zhou, G., Sohn, K. & Lee, H. (2012), Online incremental feature learning with denoising autoencoders, International Conference on Artificial Intelligence and Statistics, pp. 1453–1461. find