Relational inductive biases, deep learning, and graph networks

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Cited by

136 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 132 not yet.

• Fast Graph Representation Learning with PyTorch Geometric
• StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Texts
• AMR-To-Text Generation with Graph Transformer
• A Comprehensive Survey on Graph Neural Networks

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1806.01261 · arXiv 1806.01261

@misc{battaglia2018relational,
  title = {{Relational inductive biases, deep learning, and graph networks}},
  author = {Battaglia, Peter W. and Hamrick, Jessica B. and Bapst, Victor and Sanchez-Gonzalez, Alvaro and Zambaldi, Vinicius and Malinowski, Mateusz and Tacchetti, Andrea and Raposo, David and Santoro, Adam and Faulkner, Ryan and Gulcehre, Caglar and Song, Francis and Ballard, Andrew and Gilmer, Justin and Dahl, George and Vaswani, Ashish and Allen, Kelsey and Nash, Charles and Langston, Victoria and Dyer, Chris and Heess, Nicolas and Wierstra, Daan and Kohli, Pushmeet and Botvinick, Matt and Vinyals, Oriol and Li, Yujia and Pascanu, Razvan},
  year = {2018},
  doi = {10.48550/arxiv.1806.01261},
  eprint = {1806.01261},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1806.01261}
}

References

192 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Allamanis, M., Brockschmidt, M., and Khademi, M. (2018). Learning to represent programs with graphs. In Proceedings of the International Conference on Learning Representations (ICLR). find
2. Allamanis, M., Chanthirasegaran, P., Kohli, P., and Sutton, C. (2017). Learning continuous semantic representations of symbolic expressions. In Proceedings of the International Conference on Machine Learning (ICML). find
3. Anderson, J. R. (1982). Acquisition of cognitive skill. Psychological Review, 89(4):369. find
4. Andreas, J., Klein, D., and Levine, S. (2017). Modular multitask reinforcement learning with policy sketches. In Proceedings of the International Conference on Machine Learning (ICML). find
5. Andreas, J., Rohrbach, M., Darrell, T., and Klein, D. (2016). Neural module networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 39–48. find our card
6. Ba, J. L., Kiros, J. R., and Hinton, G. E. (2016). arXiv:1607.06450. find our card
7. Bahdanau, D., Cho, K., and Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations (ICLR). find our card
8. Battaglia, P., Pascanu, R., Lai, M., Rezende, D. J., et al. (2016). Interaction networks for learning about objects, relations and physics. In Advances in Neural Information Processing Systems, pages 4502–4510. find
9. Battaglia, P. W., Hamrick, J. B., and Tenenbaum, J. B. (2013). Simulation as an engine of physical scene understanding. Proceedings of the National Academy of Sciences, 110(45):18327–18332. find
10. Bello, I., Pham, H., Le, Q. V., Norouzi, M., and Bengio, S. (2016). Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940. — [arXiv:1611.09940](https://arxiv.org/abs/1611.09940) find
11. Bobrow, D. G. and Hinton, G. E., editors (1990). Artificial Intelligence, volume 46. Elsevier Science Publishers Ltd., Essex, UK. Special Issue 1-2: On Connectionist Symbol Processing. find
12. Layer normalization. arXiv preprint Bojchevski, A., Shchur, O., Zügner, D., and Günnemann, S. (2018). Netgan: Generating graphs via random walks. arXiv preprint arXiv:1803.00816. — [arXiv:1803.00816](https://arxiv.org/abs/1803.00816) find
13. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., and Yakhnenko, O. (2013). Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems, pages 2787–2795. find
14. Botvinick, M. M. (2008). Hierarchical models of behavior and prefrontal function. Trends in Cognitive Sciences, 12(5):201–208. find
15. Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., and Vandergheynst, P. (2017). Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18–42. find
16. Bruna, J., Zaremba, W., Szlam, A., and LeCun, Y. (2014). Spectral networks and locally connected networks on graphs. In Proceedings of the International Conference on Learning Representations (ICLR). find
17. Chang, M. B., Ullman, T., Torralba, A., and Tenenbaum, J. B. (2017). A compositional object-based approach to learning physical dynamics. In Proceedings of the International Conference on Learning Representations (ICLR). find
18. Chen, X., Li, L., Fei-Fei, L., and Gupta, A. (2018a). Iterative visual reasoning beyond convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). find
19. Chen, X., Liu, C., and Song, D. (2018b). Tree-to-tree neural networks for program translation. In Workshops of the International Conference on Learning Representations (ICLR). find
20. Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using rnn encoder-decoder for statistical machine translation. In Proceeding of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
21. Chomsky, N. (1957). Syntactic Structures. Mouton & Co. find
22. Chomsky, N. (1965). Aspects of the Theory of Syntax. MIT Press. find
23. Cohen, T. and Welling, M. (2016). Group equivariant convolutional networks. In International Conference on Machine Learning, pages 2990–2999. find
24. Craik, K. J. W. (1943). The Nature of Explanation. Cambridge University Press. find
25. Cui, Z., Henrickson, K., Ke, R., and Wang, Y. (2018). High-order graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting. arXiv preprint arXiv:1802.07007. — [arXiv:1802.07007](https://arxiv.org/abs/1802.07007) find
26. Dai, H., Dai, B., and Song, L. (2016). Discriminative embeddings of latent variable models for structured data. In Proceedings of the International Conference on Machine Learning (ICML). find
27. Dai, H., Khalil, E. B., Zhang, Y., Dilkina, B., and Song, L. (2017). Learning combinatorial optimization algorithms over graphs. In Advances in Neural Information Processing Systems. find
28. De Cao, N. and Kipf, T. (2018). MolGAN: An implicit generative model for small molecular graphs. arXiv preprint arXiv:1805.11973. — [arXiv:1805.11973](https://arxiv.org/abs/1805.11973) find
29. Defferrard, M., Bresson, X., and Vandergheynst, P. (2016). Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems, pages 3844–3852. find our card
30. Denil, M., Colmenarejo, S. G., Cabi, S., Saxton, D., and de Freitas, N. (2017). Programmable agents. arXiv preprint arXiv:1706.06383. — [arXiv:1706.06383](https://arxiv.org/abs/1706.06383) find
31. Devlin, J., Uesato, J., Singh, R., and Kohli, P. (2017). Semantic code repair using neuro-symbolic transformation networks. arXiv preprint arXiv:1710.11054. — [arXiv:1710.11054](https://arxiv.org/abs/1710.11054) find
32. Duvenaud, D. K., Maclaurin, D., Iparraguirre, J., Bombarell, R., Hirzel, T., Aspuru-Guzik, A., and Adams, R. P. (2015). Convolutional networks on graphs for learning molecular fingerprints. In Advances in Neural Information Processing Systems, pages 2224–2232. find
33. Dyer, C., Ballesteros, M., Ling, W., Matthews, A., and Smith, N. A. (2015). Transition-based dependency parsing with stack long short-term memory. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find our card
34. Džeroski, S., De Raedt, L., and Driessens, K. (2001). Relational reinforcement learning. Machine Learning, 43(1-2):7–52. find
35. Edwards, H. and Storkey, A. (2016). Towards a neural statistician. arXiv preprint arXiv:1606.02185. — [arXiv:1606.02185](https://arxiv.org/abs/1606.02185) find
36. Eliasmith, C. (2013). How to build a brain: A neural architecture for biological cognition. Oxford University Press. find
37. Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14(2):179–211. find
38. Elman, J. L. (1991). Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7(2-3):195–225. find
39. Eslami, S. A., Heess, N., Weber, T., Tassa, Y., Szepesvari, D., Hinton, G. E., et al. (2016). Attend, infer, repeat: Fast scene understanding with generative models. In Advances in Neural Information Processing Systems, pages 3225–3233. find
40. Evans, R. and Grefenstette, E. (2018). Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research, 61:1–64. find
41. Evans, R., Saxton, D., Amos, D., Kohli, P., and Grefenstette, E. (2018). Can neural networks understand logical entailment? In Proceedings of the International Conference on Learning Representations (ICLR). find
42. Farquhar, G., Rocktäschel, T., Igl, M., and Whiteson, S. (2018). TreeQN and ATreeC: Differentiable tree planning for deep reinforcement learning. In Proceedings of the International Conference on Learning Representations (ICLR). find
43. Finn, C., Abbeel, P., and Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. arXiv preprint arXiv:1703.03400. find
44. Fodor, J. A. (1975). The Language of Thought. Harvard University Press. find
45. Fodor, J. A. and Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1-2):3–71. find
46. Foerster, J., Assael, I. A., de Freitas, N., and Whiteson, S. (2016). Learning to communicate with deep multi-agent reinforcement learning. In Advances in Neural Information Processing Systems, pages 2137–2145. find
47. Fukushima, K. (1980). Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36:193–202. find
48. Garcia, V. and Bruna, J. (2018). Few-shot learning with graph neural networks. In Proceedings of the International Conference on Learning Representations (ICLR). find
49. Garcı́a-Durán, A. and Niepert, M. (2017). Learning graph representations with embedding propagation. arXiv preprint arXiv:1710.03059. — [arXiv:1710.03059](https://arxiv.org/abs/1710.03059) find
50. Garnelo, M., Arulkumaran, K., and Shanahan, M. (2016). Towards deep symbolic reinforcement learning. arXiv preprint arXiv:1609.05518. — [arXiv:1609.05518](https://arxiv.org/abs/1609.05518) find
51. Gaunt, A. L., Brockschmidt, M., Kushman, N., and Tarlow, D. (2016). Differentiable programs with neural libraries. arXiv preprint arXiv:1611.02109. — [arXiv:1611.02109](https://arxiv.org/abs/1611.02109) find
52. Geman, S., Bienenstock, E., and Doursat, R. (1992). Neural networks and the bias/variance dilemma. Neural Computation, 4(1):1–58. find
53. Gentner, D. and Markman, A. B. (1997). Structure mapping in analogy and similarity. American Psychologist, 52(1):45. find
54. Getoor, L. and Taskar, B. (2007). Introduction to Statistical Relational Learning. MIT press. find
55. Probabilistic machine learning and artificial intelligence. find
56. Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E. (2017). Neural message passing for quantum chemistry. arXiv preprint arXiv:1704.01212. find our card
57. Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y. (2016). Deep Learning. MIT Press. find
58. Goodman, N. (1955). The new riddle of induction. In Fact, Fiction, and Forecast, pages 59–83. Harvard University Press. find
59. Goodman, N., Mansinghka, V., Roy, D. M., Bonawitz, K., and Tenenbaum, J. B. (2012). Church: a language for generative models. arXiv preprint arXiv:1206.3255. find
60. Goodman, N. D., Tenenbaum, J. B., and Gerstenberg, T. (2015). Concepts in a probabilistic language of thought. In Margolis, E. and Laurence, S., editors, The Conceptual Mind: New Directions in the Study of Concepts. MIT Press. find
61. Goodwin, G. P. and Johnson-Laird, P. (2005). Reasoning about relations. Psychological Review, 112(2):468. find
62. Gori, M., Monfardini, G., and Scarselli, F. (2005). A new model for learning in graph domains. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), volume 2, pages 729–734. IEEE. find
63. Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., et al. (2016). Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626):471. find
64. Nature, Grefenstette, E., Hermann, K. M., Suleyman, M., and Blunsom, P. (2015). Learning to transduce with unbounded memory. In Advances in Neural Information Processing Systems, pages 1828–1836. find
65. Griffiths, T. L., Chater, N., Kemp, C., Perfors, A., and Tenenbaum, J. B. (2010). Probabilistic models of cognition: Exploring representations and inductive biases. Trends in Cognitive Sciences, 14(8):357–364. find
66. Grover, A. and Leskovec, J. (2016). node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pages 855–864. ACM. find
67. Guez, A., Weber, T., Antonoglou, I., Simonyan, K., Vinyals, O., Wierstra, D., Munos, R., and Silver, D. (2018). Learning to search with MCTSnets. arXiv preprint arXiv:1802.04697. find
68. Gulcehre, C., Denil, M., Malinowski, M., Razavi, A., Pascanu, R., Hermann, K. M., Battaglia, P., Bapst, V., Raposo, D., Santoro, A., and de Freitas, N. (2018). Hyperbolic attention networks. arXiv preprint arXiv:1805.09786. — [arXiv:1805.09786](https://arxiv.org/abs/1805.09786) find
69. Hamaguchi, T., Oiwa, H., Shimbo, M., and Matsumoto, Y. (2017). Knowledge transfer for out-of-knowledge-base entities: A graph neural network approach. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). find
70. Hamilton, W., Ying, Z., and Leskovec, J. (2017). Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, pages 1025–1035. find our card
71. Hamrick, J., Allen, K., Bapst, V., Zhu, T., McKee, K., Tenenbaum, J., and Battaglia, P. (2018). Relational inductive bias for physical construction in humans and machines. In Proceedings of the 40th Annual Conference of the Cognitive Science Society. find
72. Hamrick, J. B., Ballard, A. J., Pascanu, R., Vinyals, O., Heess, N., and Battaglia, P. W. (2017). Metacontrol for adaptive imagination-based optimization. In Proceedings of the International Conference on Learning Representations (ICLR). find
73. Hartford, J., Graham, D. R., Leyton-Brown, K., and Ravanbakhsh, S. (2018). Deep models of interactions across sets. arXiv preprint arXiv:1803.02879. — [arXiv:1803.02879](https://arxiv.org/abs/1803.02879) find
74. Hay, N., Stark, M., Schlegel, A., Wendelken, C., Park, D., Purdy, E., Silver, T., Phoenix, D. S., and George, D. (2018). Behavior is everything–towards representing concepts with sensorimotor contingencies. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). find
75. Henaff, M., Bruna, J., and LeCun, Y. (2015). Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163. — [arXiv:1506.05163](https://arxiv.org/abs/1506.05163) find
76. Hinton, G. E. (1990). Mapping part-whole hierarchies into connectionist networks. Artificial Intelligence, 46(1-2):47–75. find
77. Hjort, N. L., Holmes, C., Müller, P., and Walker, S. G. (2010). Bayesian Nonparametrics. Cambridge University Press. find
78. Hoshen, Y. (2017). Vain: Attentional multi-agent predictive modeling. In Advances in Neural Information Processing Systems, pages 2698–2708. find
79. Hu, H., Gu, J., Zhang, Z., Dai, J., and Wei, Y. (2017). Relation networks for object detection. arXiv preprint arXiv:1711.11575. — [arXiv:1711.11575](https://arxiv.org/abs/1711.11575) find
80. Hudson, D. A. and Manning, C. D. (2018). Compositional attention networks for machine reasoning. In Proceedings of the International Conference on Learning Representations (ICLR). find
81. Humboldt, W. (1999/1836). On Language: On the diversity of human language construction and its influence on the mental development of the human species. Cambridge University Press. find
82. Hummel, J. E. and Holyoak, K. J. (2003). A symbolic-connectionist theory of relational inference and generalization. Psychological Review, 110(2):220. find
83. Ioffe, S. and Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning (ICML). find
84. Johnson, D. D. (2017). Learning graphical state transitions. Proceedings of the International Conference on Learning Representations (ICLR). find
85. Joulin, A. and Mikolov, T. (2015). Inferring algorithmic patterns with stack-augmented recurrent nets. In Advances in Neural Information Processing Systems, pages 190–198. find
86. Kansky, K., Silver, T., Mély, D. A., Eldawy, M., Lázaro-Gredilla, M., Lou, X., Dorfman, N., Sidor, S., Phoenix, S., and George, D. (2017). Schema networks: Zero-shot transfer with a generative causal model of intuitive physics. In Proceedings of the International Conference on Machine Learning (IC… find
87. Kearnes, S., McCloskey, K., Berndl, M., Pande, V., and Riley, P. (2016). Molecular graph convolutions: moving beyond fingerprints. Journal of computer-aided molecular design, 30(8):595– 608. find
88. Kemp, C. and Tenenbaum, J. B. (2008). The discovery of structural form. Proceedings of the National Academy of Sciences, 105(31):10687–10692. find
89. Kipf, T., Fetaya, E., Wang, K.-C., Welling, M., and Zemel, R. (2018). Neural relational inference for interacting systems. In Proceedings of the International Conference on Machine Learning (ICML). find
90. Kipf, T. N. and Welling, M. (2017). Semi-supervised classification with graph convolutional networks. In Proceedings of the International Conference on Learning Representations (ICLR). find our card
91. Koller, D. and Friedman, N. (2009). Probabilistic Graphical Models: Principles and Techniques. MIT press. find
92. Kondor, R., Son, H. T., Pan, H., Anderson, B., and Trivedi, S. (2018). Covariant compositional networks for learning graphs. arXiv preprint arXiv:1801.02144. — [arXiv:1801.02144](https://arxiv.org/abs/1801.02144) find
93. Kondor, R. and Trivedi, S. (2018). On the generalization of equivariance and convolution in neural networks to the action of compact groups. arXiv preprint arXiv:1802.03690. — [arXiv:1802.03690](https://arxiv.org/abs/1802.03690) find
94. Konidaris, G., Kaelbling, L. P., and Lozano-Perez, T. (2018). From skills to symbols: Learning symbolic representations for abstract high-level planning. Journal of Artificial Intelligence Research, 61:215–289. find
95. Kool, W. and Welling, M. (2018). Attention solves your TSP. arXiv preprint arXiv:1803.08475. find
96. Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, pages 1097–1105. find
97. Kurach, K., Andrychowicz, M., and Sutskever, I. (2016). Neural random-access machines. In Proceedings of the International Conference on Learning Representations (ICLR). find
98. Lake, B. M. and Baroni, M. (2018). Still not systematic after all these years: On the compositional skills of sequence-to-sequence recurrent networks. In Proceedings of the International Conference on Machine Learning (ICML). find
99. Lake, B. M., Salakhutdinov, R., and Tenenbaum, J. B. (2015). Human-level concept learning through probabilistic program induction. Science, 350(6266):1332–1338. find
100. Lake, B. M., Ullman, T. D., Tenenbaum, J. B., and Gershman, S. J. (2017). Building machines that learn and think like people. Behavioral and Brain Sciences, 40. find
101. LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. Nature, 521(7553):436. find
102. LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., and Jackel, L. D. (1989). Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541–551. find
103. Li, Y., Tarlow, D., Brockschmidt, M., and Zemel, R. (2016). Gated graph sequence neural networks. In Proceedings of the International Conference on Learning Representations (ICLR). find
104. Li, Y., Vinyals, O., Dyer, C., Pascanu, R., and Battaglia, P. (2018). Learning deep generative models of graphs. In Workshops at the International Conference on Learning Representations (ICLR). find
105. Li, Y., Yu, R., Shahabi, C., and Liu, Y. (2017). Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926. — [arXiv:1707.01926](https://arxiv.org/abs/1707.01926) find
106. Lin, Z., Feng, M., Santos, C. N. d., Yu, M., Xiang, B., Zhou, B., and Bengio, Y. (2017). A structured self-attentive sentence embedding. In Proceedings of the International Conference on Learning Representations (ICLR). find our card
107. Liu, H., Simonyan, K., Vinyals, O., Fernando, C., and Kavukcuoglu, K. (2018). Hierarchical representations for efficient architecture search. In Proceedings of the International Conference on Learning Representations (ICLR). find
108. Luong, M.-T., Pham, H., and Manning, C. D. (2015). Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025. find
109. Marcus, G. (2001). The algebraic mind. find
110. Marcus, G. (2018a). Deep learning: A critical appraisal. arXiv preprint arXiv:1801.00631. find
111. McClelland, J. L. (1994). The interaction of nature and nurture in development: A parallel distributed processing perspective. International perspectives on psychological science, 1:57–88. arXiv preprint McClelland, J. L. and Rumelhart, D. E. (1981). An interactive activation model of context effect… find
112. Mikolov, T., Yih, W.-t., and Zweig, G. (2013). Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746–751. find
113. Mitchell, T. M. (1980). The need for biases in learning generalizations. Department of Computer Science, Laboratory for Computer Science Research, Rutgers Univ. New Jersey. find
114. Mnih, V., Heess, N., Graves, A., et al. (2014). Recurrent models of visual attention. In Advances in neural information processing systems, pages 2204–2212. find
115. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540):529. find
116. Monti, F., Boscaini, D., Masci, J., Rodola, E., Svoboda, J., and Bronstein, M. M. (2017). Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). find
117. Moravčı́k, M., Schmid, M., Burch, N., Lisỳ, V., Morrill, D., Bard, N., Davis, T., Waugh, K., Johanson, M., and Bowling, M. (2017). Deepstack: Expert-level artificial intelligence in heads-up no-limit poker. Science, 356(6337):508–513. find
118. Narayanan, A., Chandramohan, M., Chen, L., Liu, Y., and Saminathan, S. (2016). subgraph2vec: Learning distributed representations of rooted sub-graphs from large graphs. In Workshops at the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. find
119. Narayanan, A., Chandramohan, M., Venkatesan, R., Chen, L., Liu, Y., and Jaiswal, S. (2017). graph2vec: Learning distributed representations of graphs. arXiv preprint arXiv:1707.05005. — [arXiv:1707.05005](https://arxiv.org/abs/1707.05005) find
120. Navon, D. (1977). Forest before trees: The precedence of global features in visual perception. Cognitive Psychology, 9(3):353–383. find
121. Niepert, M., Ahmed, M., and Kutzkov, K. (2016). Learning convolutional neural networks for graphs. In Proceedings of the International Conference on Machine Learning (ICML), pages 2014–2023. find
122. Nilsson, N. J. and Fikes, R. E. (1970). Strips: A new approach to the application of theorem proving to problem solving. Technical report, SRI International, Menlo Park, CA Artificial Intelligence Center. find
123. Nowak, A., Villar, S., Bandeira, A. S., and Bruna, J. (2017). A note on learning algorithms for quadratic assignment with graph neural networks. In Proceedings of the Principled Approaches to Deep Learning Workshop (PADL) at the International Conference of Machine Learning (ICML). find
124. Nowak, M. A. (2006). Five rules for the evolution of cooperation. science, 314(5805):1560–1563. find
125. Ohtsuki, H., Hauert, C., Lieberman, E., and Nowak, M. A. (2006). A simple rule for the evolution of cooperation on graphs and social networks. Nature, 441(7092):502. find
126. Oñoro-Rubio, D., Niepert, M., Garcı́a-Durán, A., González-Sánchez, R., and López-Sastre, R. J. (2017). Representation learning for visual-relational knowledge graphs. arXiv preprint arXiv:1709.02314. — [arXiv:1709.02314](https://arxiv.org/abs/1709.02314) find
127. Parisotto, E., Mohamed, A.-r., Singh, R., Li, L., Zhou, D., and Kohli, P. (2017). Neuro-symbolic program synthesis. In Proceedings of the International Conference on Learning Representations (ICLR). find
128. Pascanu, R., Li, Y., Vinyals, O., Heess, N., Buesing, L., Racanière, S., Reichert, D., Weber, T., Wierstra, D., and Battaglia, P. (2017). Learning model-based planning from scratch. arXiv preprint arXiv:1707.06170. — [arXiv:1707.06170](https://arxiv.org/abs/1707.06170) find
129. Pearl, J. (1986). Fusion, propagation, and structuring in belief networks. Artificial intelligence, 29(3):241–288. find
130. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. Morgan Kaufmann. find
131. Pearl, J. (2009). Causality: Models, Reasoning and Inference. Cambridge University Press, New York, NY, USA, 2nd edition. find
132. Pearl, J. (2018). Theoretical impediments to machine learning with seven sparks from the causal revolution. arXiv preprint arXiv:1801.04016. find
133. Pennington, J., Socher, R., and Manning, C. (2014). Glove: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. find
134. Perozzi, B., Al-Rfou, R., and Skiena, S. (2014). Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 701–710. ACM. find
135. Pevnỳ, T. and Somol, P. (2017). Using neural network formalism to solve multiple-instance problems. In International Symposium on Neural Networks, pages 135–142. Springer. find
136. Pinker, S. and Prince, A. (1988). On language and connectionism: Analysis of a parallel distributed processing model of language acquisition. Cognition, 28(1-2):73–193. find
137. Plate, T. A. (1995). Holographic reduced representations. IEEE Transactions on Neural Networks, 6(3):623–641. find
138. Plaut, D. C., McClelland, J. L., Seidenberg, M. S., and Patterson, K. (1996). Understanding normal and impaired word reading: computational principles in quasi-regular domains. Psychological Review, 103(1):56. find
139. Pollack, J. B. (1990). Recursive distributed representations. Artificial Intelligence, 46(1-2):77–105. find
140. Qi, C. R., Su, H., Mo, K., and Guibas, L. J. (2017). Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). find
141. Raposo, D., Santoro, A., Barrett, D., Pascanu, R., Lillicrap, T., and Battaglia, P. (2017). Discovering objects and their relations from entangled scene representations. In Workshops at the International Conference on Learning Representations (ICLR). find
142. Reed, S. and De Freitas, N. (2016). Neural programmer-interpreters. In Proceedings of the International Conference on Learning Representations (ICLR). find
143. Ritchie, D., Horsfall, P., and Goodman, N. D. (2016). Deep amortized inference for probabilistic programs. arXiv preprint arXiv:1610.05735. — [arXiv:1610.05735](https://arxiv.org/abs/1610.05735) find
144. Rosenblatt, F. (1961). Principles of neurodynamics. perceptrons and the theory of brain mechanisms. Technical report, Cornell Aeronautical Lab Inc., Buffalo, NY. find
145. Rumelhart, D. E., McClelland, J. L., Group, P. R., et al. (1987). Parallel Distributed Processing, volume 1. MIT Press. find
146. Russell, S. J. and Norvig, P. (2009). Artificial Intelligence: A Modern Approach (3rd Edition). Pearson. find
147. Sabour, S., Frosst, N., and Hinton, G. E. (2017). Dynamic routing between capsules. In Advances in Neural Information Processing Systems, pages 3859–3869. find
148. Sanchez-Gonzalez, A., Heess, N., Springenberg, J. T., Merel, J., Riedmiller, M., Hadsell, R., and Battaglia, P. (2018). Graph networks as learnable physics engines for inference and control. In Proceedings of the 35th International Conference on Machine Learning (ICLR). find
149. Santoro, A., Raposo, D., Barrett, D. G., Malinowski, M., Pascanu, R., Battaglia, P., and Lillicrap, T. (2017). A simple neural network module for relational reasoning. In Advances in Neural Information Processing Systems. find our card
150. Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G. (2009a). Computational capabilities of graph neural networks. IEEE Transactions on Neural Networks, 20(1):81–102. find
151. Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G. (2009b). The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61–80. find
152. Scarselli, F., Yong, S. L., Gori, M., Hagenbuchner, M., Tsoi, A. C., and Maggini, M. (2005). Graph neural networks for ranking web pages. In Proceedings of the 2005 IEEE/WIC/ACM International Conference on Web Intelligence, pages 666–672. IEEE. find
153. Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61:85–117. find
154. Selsam, D., Lamm, M., Bunz, B., Liang, P., de Moura, L., and Dill, D. L. (2018). Learning a sat solver from single-bit supervision. arXiv preprint arXiv:1802.03685. find
155. Shalev-Shwartz, S., Shamir, O., and Shammah, S. (2017). Failures of gradient-based deep learning. arXiv preprint arXiv:1703.07950. — [arXiv:1703.07950](https://arxiv.org/abs/1703.07950) find
156. Shaw, P., Uszkoreit, J., and Vaswani, A. (2018). Self-attention with relative position representations. In Proceedings of the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. find our card
157. Shervashidze, N., Schweitzer, P., Leeuwen, E. J. v., Mehlhorn, K., and Borgwardt, K. M. (2011). Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12(Sep):2539–2561. find
158. Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016). Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484–489. find
159. Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46(1-2):159–216. find
160. Socher, R., Huval, B., Manning, C. D., and Ng, A. Y. (2012). Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing (EMNLP) and Computational Natural Language Learning (CNLL), pages 1201–1211. Assoc… find
161. Socher, R., Lin, C. C., Manning, C., and Ng, A. Y. (2011a). Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML), pages 129–136. find
162. Socher, R., Pennington, J., Huang, E. H., Ng, A. Y., and Manning, C. D. (2011b). Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 151–161. Association for Computational L… find our card
163. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., and Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1631–1642. find
164. Spelke, E. S., Breinlinger, K., Macomber, J., and Jacobson, K. (1992). Origins of knowledge. Psychological review, 99(4):605. find
165. Spelke, E. S. and Kinzler, K. D. (2007). Core knowledge. Developmental Science, 10(1):89–96. find
166. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929–1958. find
167. Sukhbaatar, S., Fergus, R., et al. (2016). Learning multiagent communication with backpropagation. In Advances in Neural Information Processing Systems, pages 2244–2252. find
168. Sukhbaatar, S., Weston, J., Fergus, R., et al. (2015). End-to-end memory networks. In Advances in Neural Information Processing Systems, pages 2440–2448. find
169. Sutskever, I., Vinyals, O., and Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, pages 3104–3112. find
170. Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. A. (2017). Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume 4, page 12. find
171. Tai, K. S., Socher, R., and Manning, C. D. (2015). Improved semantic representations from tree-structured long short-term memory networks. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
172. Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., and Mei, Q. (2015). Line: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web, pages 1067–1077. International World Wide Web Conferences Steering Committee. find
173. Tenenbaum, J. B., Griffiths, T. L., and Kemp, C. (2006). Theory-based bayesian models of inductive learning and reasoning. Trends in Cognitive Sciences, 10(7):309–318. find
174. Tenenbaum, J. B., Kemp, C., Griffiths, T. L., and Goodman, N. D. (2011). How to grow a mind: Statistics, structure, and abstraction. Science, 331(6022):1279–1285. find
175. Toyer, S., Trevizan, F., Thiebaux, S., and Xie, L. (2017). Action schema networks: Generalised policies with deep learning. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). find
176. Ullman, T. D., Spelke, E., Battaglia, P., and Tenenbaum, J. B. (2017). Mind games: Game engines as an architecture for intuitive physics. Trends in Cognitive Sciences, 21(9):649–665. van Steenkiste, S., Chang, M., Greff, K., and Schmidhuber, J. (2018). Relational neural expectation maximization: Uns… find
177. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems. find
178. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y. (2018). Graph attention networks. In Proceedings of the International Conference on Learning Representations (ICLR). find
179. Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., and Botvinick, M. (2018a). Prefrontal cortex as a meta-reinforcement learning system. Nature neuroscience, page 1. find
180. Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., and Botvinick, M. (2016). Learning to reinforcement learn. arXiv preprint arXiv:1611.05763. — [arXiv:1611.05763](https://arxiv.org/abs/1611.05763) find
181. Wang, T., Liao, R., Ba, J., and Fidler, S. (2018b). Nervenet: Learning structured policy with graph neural networks. In Proceedings of the International Conference on Learning Representations (ICLR). find
182. Wang, X., Girshick, R., Gupta, A., and He, K. (2018c). Non-local neural networks. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR). find
183. Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., and Solomon, J. M. (2018d). Dynamic graph cnn for learning on point clouds. arXiv preprint arXiv:1801.07829. — [arXiv:1801.07829](https://arxiv.org/abs/1801.07829) find
184. Watters, N., Zoran, D., Weber, T., Battaglia, P., Pascanu, R., and Tacchetti, A. (2017). Visual interaction networks: Learning a physics simulator from video. In Advances in Neural Information Processing Systems, pages 4542–4550. find
185. Wu, J., Lu, E., Kohli, P., Freeman, B., and Tenenbaum, J. (2017). Learning to see physics via visual de-animation. In Advances in Neural Information Processing Systems, pages 152–163. find
186. Yoon, K., Liao, R., Xiong, Y., Zhang, L., Fetaya, E., Urtasun, R., Zemel, R., and Pitkow, X. (2018). Inference in probabilistic graphical models by graph neural networks. In Workshops at the International Conference on Learning Representations (ICLR). find
187. You, J., Ying, R., Ren, X., Hamilton, W. L., and Leskovec, J. (2018). GraphRNN: A deep generative model for graphs. arXiv preprint arXiv:1802.08773. — [arXiv:1802.08773](https://arxiv.org/abs/1802.08773) find
188. Yuille, A. L. and Liu, C. (2018). Deep nets: What have they ever done for vision? arXiv preprint arXiv:1805.04025. — [arXiv:1805.04025](https://arxiv.org/abs/1805.04025) find
189. Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R. R., and Smola, A. J. (2017). Deep sets. In Advances in Neural Information Processing Systems, pages 3394–3404. find
190. Zambaldi, V., Raposo, D., Santoro, A., Bapst, V., Li, Y., Babuschkin, I., Tuyls, K., Reichert, D., Lillicrap, T., Lockhart, E., Shanahan, M., Langston, V., Pascanu, R., Botvinick, M., Vinyals, O., and Battaglia, P. (2018). Relational deep reinforcement learning. arXiv preprint arXiv. find
191. Zhang, A., Lerer, A., Sukhbaatar, S., Fergus, R., and Szlam, A. (2018). Composable planning with attributes. arXiv preprint arXiv:1803.00512. — [arXiv:1803.00512](https://arxiv.org/abs/1803.00512) find
192. Zügner, D., Akbarnejad, A., and Günnemann, S. (2018). Adversarial Attacks on Neural Networks for Graph Data. arXiv preprint arXiv:1805.07984. — [arXiv:1805.07984](https://arxiv.org/abs/1805.07984) find