Elamur Research⌕
☰
home / papers / 2015 / Convolutional Networks on Graphs for Learning Molecular Fingerprints

Convolutional Networks on Graphs for Learning Molecular Fingerprints

Paper · 2015 · David Duvenaud, Dougal Maclaurin, Jorge Aguilera-Iparraguirre et al.

Summary

Predicting molecular properties with machine learning is hard because molecules vary in size and structure, while most models expect fixed-length inputs. The usual workaround is to compute hand-designed circular fingerprints (ECFP) with existing software and feed these frozen vectors into a regressor or deep network. Duvenaud and colleagues instead make the fingerprint itself learnable. They treat a molecule as a graph, with atoms as nodes and bonds as edges, and rebuild the ECFP pipeline from differentiable parts. The hash becomes a single neural layer applied to an atom plus the sum of its neighbours. Bit-indexing becomes a softmax whose outputs are summed into the fingerprint. The whole pipeline can then be trained end to end with gradient descent. With large random weights the model behaves much like ECFP, and ECFP can be seen as a limiting special case of it. When trained, the neural fingerprints match or outperform standard ones on solubility, drug efficacy and photovoltaic efficiency tasks. They are also more compact and easier to interpret. The work is an early, influential step toward graph neural networks for chemistry.

Key points

  • Atom neighbourhoods are combined by summation rather than sorting, which gives permutation invariance simply and scales well. Each layer has a separate hidden weight matrix for each atom degree (up to 5 bonds in organic molecules).
  • On the solubility dataset, with 2048-length fingerprints, pairwise distances from randomly weighted neural fingerprints correlate with ECFP distances at r = 0.823. Random networks with small weights already predict better than ECFP when paired with linear regression.
  • Learned features are interpretable without manual effort:
    • For solubility, one feature captures hydrophilic R-OH groups and another captures non-polar repeated rings.
    • For the toxicity datasets SR-MMP and NR-AHR, features pick out sulphur attached to aromatic rings and polycyclic aromatic hydrocarbons.
  • Learned features avoid the huge vectors of fixed fingerprints; earlier work needed about 43,000 dimensions.
  • The model has two main limitations:
    • How far information travels across the graph is bounded by network depth.
    • It cannot yet distinguish stereoisomers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Harvard UniversityCambridge · United States126 works · 45,130 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksMathematics · 189 worksComputational Theory and Mathematics · 321 worksGeometry and Topology · 4 works

Cite

Cite as David Duvenaud, Dougal Maclaurin, Jorge Aguilera-Iparraguirre, Rafael Gómez-Bombarelli, Timothy Hirzel, Alán Aspuru-Guzik and Ryan P. Adams. 2015. Convolutional Networks on Graphs for Learning Molecular Fingerprints. arXiv:1509.09292.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1509.09292 · arXiv 1509.09292

@misc{duvenaud2015convolutional,
  title = {{Convolutional Networks on Graphs for Learning Molecular Fingerprints}},
  author = {David Duvenaud, Dougal Maclaurin, Jorge Aguilera-Iparraguirre, Rafael Gómez-Bombarelli, Timothy Hirzel, Alán Aspuru-Guzik and Ryan P. Adams},
  year = {2015},
  doi = {10.48550/arxiv.1509.09292},
  eprint = {1509.09292},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1509.09292}
}

References

30 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, James Bergstra, Ian J. Goodfellow, Arnaud Bergeron, Nicolas Bouchard, and Yoshua Bengio. Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop, 2012.
  2. Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203, 2013. find
  3. George E. Dahl, Navdeep Jaitly, and Ruslan Salakhutdinov. Multi-task neural networks for QSAR predictions. arXiv preprint arXiv:1406.1231, 2014. find
  4. John S. Delaney. ESOL: Estimating aqueous solubility directly from molecular structure. Journal of Chemical Information and Computer Sciences, 44(3):1000–1005, 2004. find
  5. Francisco-Javier Gamo, Laura M Sanz, Jaume Vidal, Cristina de Cozar, Emilio Alvarez, Jose-Luis Lavandera, Dana E Vanderwall, Darren VS Green, Vinod Kumar, Samiul Hasan, et al. Thousands of chemical starting points for antimalarial lead identification. Nature, 465(7296):305–310, 2010. find
  6. Robert C. Glem, Andreas Bender, Catrin H. Arnby, Lars Carlsson, Scott Boyer, and James Smith. Circular fingerprints: flexible molecular descriptors with applications from physical chemistry to ADME. IDrugs: the investigational drugs journal, 9(3):199–204, 2006. find
  7. Alex Graves, Greg Wayne, and Ivo Danihelka. Neural Turing machines. arXiv preprint arXiv:1410.5401, 2014. find
  8. Johannes Hachmann, Roberto Olivares-Amaya, Sule Atahan-Evrenk, Carlos Amador-Bedolla, Roel S Sánchez-Carrera, Aryeh Gold-Parker, Leslie Vogt, Anna M Brockway, and Alán Aspuru-Guzik. The Harvard clean energy project: large-scale computational screening and design of organic photovoltaics on the wor… find
  9. John R Hershey, Jonathan Le Roux, and Felix Weninger. Deep unfolding: Model-based inspiration of novel deep architectures. arXiv preprint arXiv:1409.2574, 2014. — [arXiv:1409.2574](https://arxiv.org/abs/1409.2574) find
  10. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997. find
  11. Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015. find
  12. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. A convolutional neural network for modelling sentences. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, June 2014.
  13. Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  14. Yann LeCun and Yoshua Bengio. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361, 1995. find
  15. Alessandro Lusci, Gianluca Pollastri, and Pierre Baldi. Deep architectures and deep learning in chemoinformatics: the prediction of aqueous solubility for drug-like molecules. Journal of chemical information and modeling, 53(7):1563–1575, 2013. find
  16. Alessio Micheli. Neural network for graphs: A contextual constructive approach. Neural Networks, IEEE Transactions on, 20(3):498–511, 2009. find
  17. H.L. Morgan. The generation of a unique machine description for chemical structure. Journal of Chemical Documentation, 5(2):107–113, 1965. find
  18. Travis E Oliphant. Python for scientific computing. Computing in Science & Engineering, 9(3):10–20, 2007. find
  19. Bharath Ramsundar, Steven Kearnes, Patrick Riley, Dale Webster, David Konerding, and Vijay Pande. Massively multitask networks for drug discovery. arXiv:1502.02072, 2015. — [arXiv:1502.02072](https://arxiv.org/abs/1502.02072) find
  20. RDKit: Open-source cheminformatics. www.rdkit.org. [accessed 11-April-2013]. find
  21. David Rogers and Mathew Hahn. Extended-connectivity fingerprints. Journal of Chemical Information and Modeling, 50(5):742–754, 2010. find
  22. F. Scarselli, M. Gori, Ah Chung Tsoi, M. Hagenbuchner, and G. Monfardini. The graph neural network model. Neural Networks, IEEE Transactions on, 20(1):61–80, Jan 2009. find
  23. Richard Socher, Eric H Huang, Jeffrey Pennin, Christopher D Manning, and Andrew Y Ng. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In Advances in Neural Information Processing Systems, pages 801–809, 2011. find
  24. Richard Socher, Jeffrey Pennington, Eric H Huang, Andrew Y Ng, and Christopher D Manning. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 151–161. Association for Computational…
  25. Kai Sheng Tai, Richard Socher, and Christopher D Manning. Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075, 2015. find
  26. Tox21 Challenge. National center for advancing translational sciences. http://tripod.nih.gov/tox21/challenge, 2014. [Online; accessed 2-June-2015]. ↗
  27. Thomas Unterthiner, Andreas Mayr, Günter Klambauer, and Sepp Hochreiter. Toxicity prediction using deep learning. arXiv preprint arXiv:1503.01445, 2015. — [arXiv:1503.01445](https://arxiv.org/abs/1503.01445) find
  28. Thomas Unterthiner, Andreas Mayr, G ünter Klambauer, Marvin Steijaert, Jörg Wenger, Hugo Ceulemans, and Sepp Hochreiter. Deep learning as an opportunity in virtual screening. In Advances in Neural Information Processing Systems, 2014. find
  29. Li Wan, Matthew Zeiler, Sixin Zhang, Yann L. Cun, and Rob Fergus. Regularization of neural networks using dropconnect. In International Conference on Machine Learning, 2013. find
  30. David Weininger. SMILES, a chemical language and information system. Journal of chemical information and computer sciences, 28(1):31–36, 1988. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/duvenaud-2015-convolutional-networks-graphs-learning-molecular/