Elamur Research⌕
☰
home / papers / 2020 / The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence

The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence

Paper · 2020 · Gary Marcus

Summary

Marcus argues that a decade of progress built on generic learning algorithms, ever-bigger datasets and more compute has not produced AI that behaves dependably outside its training conditions. He defines "robust AI" as a modest but essential target: systems that apply what they know systematically across many problems, combine information from varied sources, and carry lessons into new contexts much as an ordinary adult would. This is a position paper, not an empirical study. It introduces no new corpus, model or benchmark scores. Instead, it gathers evidence of brittleness, such as misclassified images and language models derailed by distractors or simple tracking questions. It sets this evidence against Rich Sutton's "Bitter Lesson" and DeepMind's goal of learning without human input. The proposed remedy revives classical AI priorities using modern techniques. The paper's significance lies in its research agenda, which treats reliability and trust as the central yardsticks for AI's next decade.

Key points

  • The agenda consists of four steps: a hybrid neural–symbolic architecture, large-scale knowledge (partly abstract and causal), explicit reasoning, and internal cognitive models of the world.
  • GPT-2 is the main language example. Given that a person left clothes at the dry cleaners, it placed them at "my mom's house." Asked about six frogs on a log where two leave and three join, it answered "seventeen." Both examples come from a benchmark Marcus presented at NeurIPS 2019.
  • Other cited failures include a vision classifier misreading an overturned school bus on a snowy road (Alcorn et al., 2018) and a reading model misled by inserted distractor sentences (Jia & Liang, 2017).
  • The paper contrasts its view with Sutton's "The Bitter Lesson" and with AlphaGo Zero, which learned "tabula rasa" (Silver et al., 2017).
  • It draws on classical precedents: McCarthy's 1959 "Programs with Common Sense," Lenat's commonsense knowledge project, and Winograd's SHRDLU blocks-world system (1971).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Artificial intelligence · 49 works

Cite

Cite as Gary Marcus. 2020. The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence. arXiv:2002.06177.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2002.06177 · arXiv 2002.06177

@misc{marcus2020next,
  title = {{The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence}},
  author = {Gary Marcus},
  year = {2020},
  doi = {10.48550/arxiv.2002.06177},
  eprint = {2002.06177},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2002.06177}
}

References

107 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Adiwardana, D., Luong, M.-T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R. et al. (2020). Towards a Human-like Open-Domain Chatbot. cs.CL.
  2. Alcorn, M. A., Li, Q., Gong, Z., Wang, C., Mai, L., Ku, W.-S. et al. (2018). Strike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects. arXiv, 1811.11553v3. find
  3. Arabshahi, F., Lu, Z., Singh, S., & Anandkumar, A. (2019). Memory Augmented Recursive Neural Networks. cs.LG. find
  4. Bakken, T. E., Miller, J. A., Ding, S. L., Sunkin, S. M., Smith, K. A., Ng, L. et al. (2016). A comprehensive transcriptional map of primate brain development. Nature, 535(7612), 367-375. find
  5. Banino, A., Badia, A. P., Köster, R., Chadwick, M. J., Zambaldi, V., Hassabis, D. et al. (2020). MEMO: A Deep Network for Flexible Combination of Episodic Memories. cs.LG. find
  6. Bender, E. M., & Lascarides, A. (2019). Linguistic Fundamentals for Natural Language Processing II: 100 Essentials from Semantics and Pragmatics. Synthesis Lectures on Human Language Technologies, 12(3), 1-268. find
  7. Bengio, Y. (2019). From System 1 Deep Learning to System 2 Deep Learning. Proceedings from NeuripS 2019. find
  8. Bengio, Y., Deleu, T., Rahaman, N., Ke, R., Lachapelle, S., Bilaniuk, O. et al. (2019). A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms. cs.LG. find
  9. Berent, I., Marcus, G. F., Shimron, J., & Gafos, A. I. (2002). The scope of linguistic generalizations: Evidence from Hebrew word formation. Cognition, 83(2), 113-139. find
  10. Berent, I., Vaknin, V., & Marcus, G. F. (2007). Roots, stems, and the universality of lexical representations: Evidence from Hebrew. Cognition, 104(2), 254-286. find
  11. Besold, T. R., Garcez, A. D., Stenning, K., van der Torre, L., & van Lambalgen, M. (2017). Reasoning in non-probabilistic uncertainty: Logic programming and neural-symbolic computing as examples. Minds and Machines, 27(1), 37-77. find
  12. Bingham, E., Chen, J. P., Jankowiak, M., Obermeyer, F., Pradhan, N., Karaletsos, T. et al. (2019). Pyro: Deep universal probabilistic programming. The Journal of Machine Learning Research, 20(1), 973-978. find
  13. Bordes, A., Usunier, N., Chopra, S., & Weston, J. (2015). Large-scale Simple Question Answering with Memory Networks. arXiv. find
  14. Burgess, C. P., Matthey, L., Watters, N., Kabra, R., Higgins, I., Botvinick, M. et al. (2019). MONet: Unsupervised Scene Decomposition and Representation. arXiv, 1901.11390v1. find
  15. Carey, S. (2009). The origin of concepts. Oxford university press. find
  16. Chollet, F. (2019). On the Measure of Intelligence. cs.AI. find
  17. Clark, P., Etzioni, O., Khashabi, D., Khot, T., Mishra, B. D., Richardson, K. et al. (2019). From ‘F’ to ‘A’ on the N.Y. Regents Science Exams: An Overview of the Aristo Project. cs.CL. find
  18. Cranmer, M. D., Xu, R., Battaglia, P., & Ho, S. (2019). Learning Symbolic Physics with Graph Networks. arXiv preprint arXiv:1909.05862. — [arXiv:1909.05862](https://arxiv.org/abs/1909.05862) find
  19. Cropper, A., Morel, R., & Muggleton, S. (2019). Learning higher-order logic programs. Machine Learning, 1-34. find
  20. D’Avila Garcez, A. S., Lamb, L. C., & Gabbay, D. M. (2009). Neural-symbolic cognitive reasoning. Springer Science & Business Media. find
  21. Davis, E. (2019). The Use of Deep Learning for Symbolic Integration: A Review of (Lample and Charton, 2019). cs.LG. find
  22. Davis, E., Marcus, G., & Frazier-Logue, N. (2017). Commonsense reasoning about containers using radically incomplete information. Artificial intelligence, 248, 46-84. find
  23. Dietterich, T. G. (2017). Steps toward robust artificial intelligence. AI Magazine, 38(3), 3-24. find
  24. Dyer, F. C., & Dickinson, J. A. (1994). Development of sun compensation by honeybees: how partially experienced bees estimate the sun’s course. Proceedings of the National Academy of Sciences, 91(10), 4471-4474. find
  25. Engelcke, M., Kosiorek, A. R., Jones, O. P., & Posner, I. (2019). GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations. arXiv, 1907.13052v3. find
  26. Evans, R., & Grefenstette, E. (2017). Learning Explanatory Rules from Noisy Data. arXiv, cs.NE. find
  27. Fawzi, A., Malinowski, M., Fawzi, H., & Fawzi, O. (2019). Learning dynamic polynomial proofs. cs.LG. find
  28. Fodor, J. A., & Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: a critical analysis. Cognition, 28(1-2), 3-71. find
  29. Frankland, S. M., & Greene JD. (2019). Concepts and Compositionality: In Search of the Brain’s Language of Thought. Annual review of psychology. find
  30. Gallistel, C. R. (1990). The organization of learning. The MIT Press. find
  31. Gallistel, C. R., & King, A. P. (2010). Memory and the computational brain: Why cognitive science will transform neuroscience. John Wiley & Sons. find
  32. George, D., Lehrach, W., Kansky, K., Lázaro-Gredilla, M., Laan, C., Marthi, B. et al. (2017). A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs. Science, 358(6368). find
  33. Gidon, A., Zolnik, T. A., Fidzinski, P., Bolduan, F., Papoutsi, A., Poirazi, P. et al. (2020). Dendritic action potentials and computation in human layer 2/3 cortical neurons. Science, 367(6473), 83-87. find
  34. Gopnik, A., & Sobel, D. M. (2000). Detecting blickets: How young children use information about novel causal powers in categorization and induction. Child development, 71(5), 1205-1222. find
  35. Goyal, A., Lamb, A., Hoffmann, J., Sodhani, S., Levine, S., Bengio, Y. et al. (2019). Recurrent Independent Mechanisms. cs.LG. find
  36. Gregor, K., Rezende, D. J., Besse, F., Wu, Y., Merzic, H., & Oord, A. V. D. (2019). Shaping Belief States with Generative Environment Models for RL. arXiv, 1906.09237v2. find
  37. Gupta, N., Lin, K., Roth, D., Singh, S., & Gardner, M. (2019). Neural Module Networks for Reasoning over Text. arXiv, 1912.04971v1. find
  38. Ha, D., & Schmidhuber, J. (2018). World Models. arXiv, 1803.10122v4. find
  39. Henaff, M., Weston, J., Szlam, A., Bordes, A., & LeCun, Y. (2016). Tracking the World State with Recurrent Entity Networks. arXivICLR 2017, 1612.03969v3. find
  40. Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2017). Deep Reinforcement Learning that Matters. arXiv, cs.LG. find
  41. Hill, F., Lampinen, A., Schneider, R., Clark, S., Botvinick, M., McClelland, J. L. et al. (2019). Emergent systematic generalization in a situated agent. arXiv preprint arXiv:1910.00571. — [arXiv:1910.00571](https://arxiv.org/abs/1910.00571) find
  42. Hinton, G. E. (1990). Preface to the special issue on connectionist symbol processing. Artificial Intelligence, 46(1-2), 1-4. find
  43. Janner, M., Levine, S., Freeman, W. T., Tenenbaum, J. B., Finn, C., & Wu, J. (2018). Reasoning About Physical Interactions with Object-Oriented Prediction and Planning. cs.LG. find
  44. Jia, R., & Liang, P. (2017). Adversarial Examples for Evaluating Reading Comprehension Systems. arXiv.
  45. Johnson-Laird, P. N. (1983). Mental models: Towards a cognitive science of language, inference, and consciousness ((6)). Harvard University Press. find
  46. Kansky, K., Silver, T., Mély, D. A., Eldawy, M., Lázaro-Gredilla, M., Lou, X. et al. (2017). Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics. arXIv, cs.AI. find
  47. Keil, F. C. (1992). Concepts, kinds, and cognitive development. mit Press. find
  48. Knudsen, E. I., & Konishi, M. (1979). Mechanisms of sound localization in the barn owl (Tyto alba). Journal of Comparative Physiology, 133(1), 13-21. find
  49. Koul, A., Greydanus, S., Fern - arXiv preprint arXiv:1811.12530, A., & 2018. Learning finite state representations of recurrent policy networks. arxiv.org. — [arXiv:1811.12530](https://arxiv.org/abs/1811.12530) find
  50. Lake, B. M., & Baroni, M. (2017). Still not systematic after all these years: On the compositional skills of sequence-to-sequence recurrent networks. arXiv. find
  51. Lample, G., & Charton, F. (2019). Deep Learning for Symbolic Mathematics. arXiv, 1912.01412v1. find
  52. Landau, B., Gleitman, L. R., & Landau, B. (2009). Language and experience: Evidence from the blind child (8). Harvard University Press. find
  53. LeCun, Y. (1989). Generalization and network design strategies. Technical Report CRG-TR-89-4. find
  54. Legenstein, R., Papadimitriou, C. H., Vempala, S., & Maass, W. (2016). Assembly pointers for variable binding in networks of spiking neurons. arXiv, 1611.03698v1. find
  55. Lenat, D. (2019). What AI Can Learn From Romeo & Juliet. Forbes. find
  56. Lenat, D. B., Prakash, M., & Shepherd, M. (1985). CYC: Using common sense knowledge to overcome brittleness and knowledge acquisition bottlenecks. AI magazine, 6(4), 65-65. find
  57. Leslie, A. M. (1982). The perception of causality in infants. Perception, 11(2), 173-186. find
  58. Maier, A., Schebesch, F., Syben, C., Würfl, T., Steidl, S., Choi, J.-H. et al. (2017). Precision Learning: Towards Use of Known Operators in Neural Networks. A. Maier, F. Schebesch, C. Syben, T. W\\”urfl, S. Steidl, J.-H. Choi, R. Fahrig, Precision Learning: Towards Use of Known Operators in Neural N… find
  59. Mandler, J. M. (1992). How to build a baby: II. Conceptual primitives. Psychological review, 99(4), 587. find
  60. Mao, J., Gan, C., Kohli, P., Tenenbaum, J. B., & Wu, J. (2019). The Neuro-Symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. arXiv preprint arXiv:1904.12584. find
  61. Marcus, G. (2019). Deep Understanding: The Next Challenge for AI. Proceedings from NeurIPS 2019. find
  62. Marcus, G. (2020). GPT-2 and the Nature of Intelligence. The Gradient. find
  63. Marcus, G., Marblestone, A., & Dean, T. (2014). The atoms of neural computation. Science, 346(6209), 551-552. find
  64. Marcus, G. (2018). Deep Learning: A Critical Appraisal. arXiv.
  65. Marcus, G., & Davis, E. (2019). Rebooting AI: building artificial intelligence we can trust. Pantheon. find
  66. Marcus, G. F. (2008). Kluge : the haphazard construction of the human mind. Boston: Houghton Mifflin. find
  67. Marcus, G. F. (2001). The Algebraic Mind: Integrating Connectionism and cognitive science. Cambridge, Mass.: MIT Press. find
  68. Marcus, G. F. (2004). The Birth of the Mind : how a tiny number of genes creates the complexities of human thought. Basic Books. find
  69. Marcus, G. F. (1998). Rethinking eliminative connectionism. Cogn Psychol, 37(3), 243-282. find
  70. Marcus, G. F., Pinker, S., Ullman, M., Hollander, M., Rosen, T. J., & Xu, F. (1992). Overregularization in language acquisition. Monogr Soc Res Child Dev, 57(4), 1-182. find
  71. Marcus, G. F., Vijayan, S., Bandi Rao, S., & Vishton, P. M. (1999). Rule learning by seven-month-old infants. Science, 283(5398), 77-80. find
  72. Marr, D. (1982). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. San Francisco: WH Freeman and Co. find
  73. McClelland, J. L. (2019). Integrating New Knowledge into a Neural Network without Catastrophic Interference: Computational and Theoretical Investigations in a Hierarchically Structured Environment. find
  74. McCloskey, M., & Cohen, N. J. (1989). Catastrophic interference in connectionist networks: The sequential learning problem. Elsevier, 24, 109-165. find
  75. Miller, J. A., Ding, S. L., Sunkin, S. M., Smith, K. A., Ng, L., Szafer, A. et al. (2014). Transcriptional landscape of the prenatal human brain. Nature, 508(7495), 199-206. find
  76. Minervini, P., Bošnjak, M., Rocktäschel, T., Riedel, S., & Grefenstette, E. (2019). Differentiable Reasoning on Large Knowledge Bases and Natural Language. cs.LG. find
  77. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G. et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529-533. find
  78. Newell, A. (1980). Physical symbol systems. Cognitive science, 4(2), 135-183. find
  79. Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., & Kiela, D. (2019). Adversarial NLI: A New Benchmark for Natural Language Understanding. cs.CL.
  80. Norvig, P. (1986). Unified theory of inference for text understanding. find
  81. OpenAI, Akkaya, I., Andrychowicz, M., Chociej, M., Litwin, M., McGrew, B. et al. (2019). Solving Rubik’s Cube with a Robot Hand. arXiv, 1910.07113v1. find
  82. Pasupat, P., & Liang, P. (2015). Compositional semantic parsing on semi-structured tables. arXiv preprint arXiv:1508.00305.
  83. Pearl, J., & Mackenzie, D. (2018). The book of why: the new science of cause and effect. Basic Books. find
  84. Polozov, O., & Gulwani, S. (2015). FlashMeta: a framework for inductive program synthesis. find
  85. Rabinowitz, N. C., Perbet, F., Song, H. F., Zhang, C., Eslami, S. M. A., & Botvinick, M. (2018). Machine Theory of Mind. arXiv, 1802.07740v2. find
  86. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI Blog, 1(8). find
  87. Raedt, L. D., Kersting, K., Natarajan, S., & Poole, D. (2016). Statistical relational artificial intelligence: Logic, probability, and computation. Synthesis Lectures on Artificial Intelligence and Machine Learning, 10(2), 1-189. find
  88. Richardson, M., & Domingos, P. (2006). Markov logic networks. Machine learning, 62(1), 107-136. find
  89. Schank, R. C., & Abelson, R. P. (1977). Scripts, plans, goals, and understanding: An inquiry into human knowledge structures. Erlbaum. find
  90. Schlag, I., Smolensky, P., Fernandez, R., Jojic, N., Schmidhuber, J., & Gao, J. (2019). Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving. cs.LG. find
  91. Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S. et al. (2019). Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. arXiv, 1911.08265v1. find
  92. Serafini, L., & Garcez, A. D. (2016). Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and Knowledge. arXiv, 1606.04422v2. find
  93. Shavlik, J. W. (1994). Combining symbolic and neural learning. Machine Learning, 14(3), 321-331. find
  94. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A. et al. (2017). Mastering the game of Go without human knowledge. Nature, 550(7676), 354-359. find
  95. Smolensky, P., Lee, M., He, X., Yih, W.-t., Gao, J., & Deng, L. (2016). Basic Reasoning with Tensor Product Representations. arXiv, cs.AI. find
  96. Spelke, E. (1994). Initial knowledge: six suggestions. Cognition, 50(1-3), 431-445. find
  97. Sun, R. (1996). Hybrid Connectionist-Symbolic Modules: A Report from the IJCAI-95 Workshop on Connectionist-Symbolic Integration. AI Magazine, 17(2), 99-99. find
  98. Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217), 1632-1634. find
  99. Van den Broeck, G. (2019). IJCAI-19 Computers and Thought Award Lecture. Proceedings from IJCAI-19. find
  100. Van Essen, D. C., Donahue, C., Dierker, D. L., & Glasser, M. F. (2016). Parcellations and connectivity patterns in human and macaque cerebral cortex. In Micro-, Meso-and Macro-Connectomics of the Brain (pp. 89-106). Springer, Cham. find
  101. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N. et al. (2017). Attention Is All You Need. cs.CL.
  102. Veerapaneni, R., Co-Reyes, J. D., Chang, M., Janner, M., Finn, C., Wu, J. et al. (2019). Entity Abstraction in Visual Model-Based Reinforcement Learning. cs.LG. find
  103. Vergari, A., Di Mauro, N., & Van den Broek, G. (2019). Tutorial slides on tractable probabilistic models. find
  104. Wayne, G., Hung, C.-C., Amos, D., Mirza, M., Ahuja, A., Grabska-Barwinska, A. et al. (2018). Unsupervised Predictive Memory in a Goal-Directed Agent. arXiv, 1803.10760v1. find
  105. Winograd, T. (1971). Procedures as a representation for data in a computer program for understanding natural language. find
  106. Yang, F., Yang, Z., & Cohen, W. W. (2017). Differentiable Learning of Logical Rules for Knowledge Base Reasoning. cs.AI. find
  107. Zhang, R., Wu, J., Zhang, C., Freeman, W. T., & Tenenbaum, J. B. (2016). A Comparative Evaluation of Approximate Probabilistic Simulation and Deep Neural Networks as Accounts of Human Physical Scene Understanding. arXiv, 1605.01138v2. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/marcus-2020-next-decade-ai-four/