Elamur Research
☰
home / papers / 2017 / Neural Message Passing for Quantum Chemistry

Neural Message Passing for Quantum Chemistry

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1704.01212

Summary

Predicting quantum-mechanical properties of molecules with density functional theory (DFT) is accurate but slow. Gilmer et al. ask whether neural networks that read molecular graphs directly, without hand-crafted descriptors, can replace such calculations. They show that at least eight earlier graph models are special cases of one scheme, Message Passing Neural Networks (MPNNs). In an MPNN, node states are refined over several rounds using messages from neighbouring atoms, and a permutation-invariant readout then turns those states into a molecule-level prediction. Within this framework they test new variants, including an edge-network message function, a set2set readout, virtual edges, a master node and a multi-tower design that keeps cost down for wide node representations. On the QM9 benchmark (about 130k small organic molecules, 13 DFT-computed regression targets), their best model beats all baselines on every target and matches DFT within chemical accuracy on 11 of them. The work made MPNNs a standard reference point for molecular machine learning. The authors argue that future benchmarks should use larger molecules or more accurate labels.

Key points

  • Data: QM9 contains 13 properties per molecule plus 3D geometry, so the authors could test models both with and without spatial input. Trained networks run roughly 300,000× faster than the DFT calculation they approximate.
  • Best model: The enn-s2s model (edge network plus set2set) reaches an average error ratio of 0.68 relative to chemical accuracy, and a 5-model ensemble reaches 0.52. The strongest earlier baselines scored about 1.35 (HDAD) and 1.36 (GG-NN).
  • Graph-only input: Without spatial information, adding long-range mechanisms to GG-NN lowers the average error ratio from 3.47 to 2.90 with virtual edges, 2.62 with a master node and 2.57 with set2set. The set2set version hits chemical accuracy on 5 of 13 targets.
  • Towers variant: Splitting node states into 8 towers improves scalability and slightly improves accuracy, with error ratio falling from 1.53 to 1.37 under per-target training.
  • Message functions: Under joint training, the edge network outperforms the pair-message function (1.53 vs 3.98). Tying weights across time steps while increasing the hidden size worked better than untied weights.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

259 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 252 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1704.01212 · arXiv 1704.01212

@misc{gilmer2017neural,
  title = {{Neural Message Passing for Quantum Chemistry}},
  author = {Gilmer, Justin and Schoenholz, Samuel S. and Riley, Patrick F. and Vinyals, Oriol and Dahl, George E.},
  year = {2017},
  doi = {10.48550/arxiv.1704.01212},
  eprint = {1704.01212},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1704.01212}
}

References

37 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Battaglia, Peter, Pascanu, Razvan, Lai, Matthew, Rezende, Danilo Jimenez, and Kavukcuoglu, Koray. Interaction networks for learning about objects, relations and physics. In Advances in Neural Information Processing Systems, pp. 4502–4510, 2016. find
  2. Becke, Axel D. Density-functional thermochemistry. iii. the role of exact exchange. The Journal of Chemical Physics, 98(7):5648–5652, 1993. doi: 10.1063/1.464913. URL http://dx.doi.org/10.1063/1.464913. — [doi:10.1063/1.464913](https://doi.org/10.1063/1.464913) ↗
  3. Behler, Jörg and Parrinello, Michele. Generalized neural-network representation of high-dimensional potential-energy surfaces. Phys. Rev. Lett., 98: 146401, Apr 2007. doi: 10.1103/PhysRevLett.98.146401. URL http://link.aps.org/doi/10.1103/PhysRevLett.98.146401. — [doi:10.1103/physrevlett.98.146401](https://doi.org/10.1103/physrevlett.98.146401) ↗
  4. Bing, Huang, von Lillenfeld, O. Anatole, and Bakowies, Dirk. personal communication, 2017. find
  5. Bruna, Joan, Zaremba, Wojciech, Szlam, Arthur, and Le-Cun, Yann. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203, 2013. — [arXiv:1312.6203](https://arxiv.org/abs/1312.6203) find
  6. Ceperley, David and Alder, B. Quantum monte carlo. Science, 231, 1986. find
  7. Cho, Kyunghyun, Van Merriënboer, Bart, Bahdanau, Dzmitry, and Bengio, Yoshua. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259, 2014. find our card
  8. Defferrard, Michaël, Bresson, Xavier, and Vandergheynst, Pierre. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems, pp. 3837–3845, 2016. find our card
  9. Duvenaud, David K, Maclaurin, Dougal, Iparraguirre, Jorge, Bombarell, Rafael, Hirzel, Timothy, Aspuru-Guzik, Alán, and Adams, Ryan P. Convolutional networks on graphs for learning molecular fingerprints. In Advances in neural information processing systems, pp. 2224–2232, 2015. find
  10. Faber, Felix, Hutchison, Luke, Huang, Bing, Gilmer, Justin, Schoenholz, Samuel S., Dahl, George E., Vinyals, Oriol, Kearnes, Steven, Riley, Patrick F., and von Lilienfeld, O. Anatole. Fast machine learning models of electronic and energetic properties consistently reach approximation errors better t… — [arXiv:1702.05532](https://arxiv.org/abs/1702.05532) find
  11. Hansen, Katja, Biegler, Franziska, Ramakrishnan, Raghunathan, Pronobis, Wiktor, von Lilienfeld, O. Anatole, Mller, Klaus-Robert, and Tkatchenko, Alexandre. Machine learning predictions of molecular properties: Accurate many-body potentials and nonlocality in chemical space. The journal of physical c… — [doi:10.1021/acs.jpclett.5b00831](https://doi.org/10.1021/acs.jpclett.5b00831) find
  12. Hedin, Lars. New method for calculating the one-particle green’s function with application to the electron-gas problem. Phys. Rev., 139:A796–A823, Aug 1965. doi: 10.1103/PhysRev.139.A796. URL http://link. aps.org/doi/10.1103/PhysRev.139.A796. — [doi:10.1103/physrev.139.a796](https://doi.org/10.1103/physrev.139.a796) find
  13. Hinton, Geoffrey, Deng, Li, Yu, Dong, Dahl, George E., Mohamed, Abdel-rahman, Jaitly, Navdeep, Senior, Andrew, Vanhoucke, Vincent, Nguyen, Patrick, Sainath, Tara N, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processi… find
  14. Hohenberg, P. and Kohn, W. Inhomogeneous electron gas. Phys. Rev., 136:B864–B871, Nov 1964. doi: 10.1103/PhysRev.136.B864. URL http://link.aps.org/doi/10.1103/PhysRev.136.B864. — [doi:10.1103/physrev.136.b864](https://doi.org/10.1103/physrev.136.b864) ↗
  15. Hu, LiHong, Wang, XiuJun, Wong, LaiHo, and Chen, GuanHua. Combined first-principles calculation and neural-network correction approach for heat of formation. The Journal of Chemical Physics, 119(22):11501– 11507, 2003. find
  16. Huang, Bing and von Lilienfeld, O. Anatole. Communication: Understanding molecular representations in machine learning: The role of uniqueness and target similarity. The Journal of Chemical Physics, 145(16): 161102, 2016. doi: 10.1063/1.4964627. URL http: //dx.doi.org/10.1063/1.4964627. — [doi:10.1063/1.4964627](https://doi.org/10.1063/1.4964627) find
  17. Kearnes, Steven, McCloskey, Kevin, Berndl, Marc, Pande, Vijay, and Riley, Patrick. Molecular graph convolutions: Moving beyond fingerprints. Journal of Computer-Aided Molecular Design, 30(8):595–608, 2016. find
  18. Kingma, Diederik and Ba, Jimmy. method for stochastic optimization. arXiv:1412.6980, 2014. find
  19. Adam: A arXiv preprint Kipf, T. N. and Welling, M. Semi-Supervised Classification with Graph Convolutional Networks. ArXiv eprints, September 2016. find our card
  20. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097–1105, 2012. find
  21. Li, Yujia, Tarlow, Daniel, Brockschmidt, Marc, and Zemel, Richard. Gated graph sequence neural networks. ICLR, 2016. find
  22. Lusci, Alessandro, Pollastri, Gianluca, and Baldi, Pierre. Deep architectures and deep learning in chemoinformatics: the prediction of aqueous solubility for drug-like molecules. Journal of chemical information and modeling, 53(7):1563–1575, 2013. find
  23. Marino, Kenneth, Salakhutdinov, Ruslan, and Gupta, Abhinav. The more you know: Using knowledge graphs for image classification. arXiv preprint arXiv:1612.04844, 2016. — [arXiv:1612.04844](https://arxiv.org/abs/1612.04844) find
  24. Merkwirth, Christian and Lengauer, Thomas. Automatic generation of complementary descriptors with molecular graph networks. Journal of chemical information and modeling, 45(5):1159–1168, 2005. find
  25. Micheli, Alessio. Neural network for graphs: A contextual constructive approach. IEEE Transactions on Neural Networks, 20(3):498–511, 2009. find
  26. Montavon, Grégoire, Hansen, Katja, Fazli, Siamac, Rupp, Matthias, Biegler, Franziska, Ziehe, Andreas, Tkatchenko, Alexandre, von Lilienfeld, O. Anatole, and Müller, Klaus-Robert. Learning invariant representations of molecules for atomization energy prediction. In Advances in Neural Information Pr… find
  27. Niepert, Mathias, Ahmed, Mohamed, and Kutzkov, Konstantin. Learning convolutional neural networks for graphs. In Proceedings of the 33rd annual international conference on machine learning. ACM, 2016. find
  28. Ramakrishnan, Raghunathan, Dral, Pavlo O, Rupp, Matthias, and Von Lilienfeld, O Anatole. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1, 2014. find
  29. Rogers, David and Hahn, Mathew. Extended-connectivity fingerprints. Journal of chemical information and modeling, 50(5):742–754, 2010. find
  30. Ruddigkeit, Lars, Van Deursen, Ruud, Blum, Lorenz C, and Reymond, Jean-Louis. Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17. Journal of chemical information and modeling, 52(11):2864–2875, 2012. find
  31. Rupp, Matthias, Tkatchenko, Alexandre haand Müller, Klaus-Robert, and von Lilienfeld, O. Anatole. Fast and accurate modeling of molecular atomization energies with machine learning. Physical review letters, 108 (5):058301, Jan 2012. URL http://dx.doi.org/10.1103/PhysRevLett.108.058301. — [doi:10.1103/physrevlett.108.058301](https://doi.org/10.1103/physrevlett.108.058301) ↗
  32. Scarselli, Franco, Gori, Marco, Tsoi, Ah Chung, Hagenbuchner, Markus, and Monfardini, Gabriele. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61–80, 2009. find
  33. Schoenholz, Samuel S., Cubuk, Ekin D., Sussman, Daniel M, Kaxiras, Efthimios, and Liu, Andrea J. A structural approach to relaxation in glassy liquids. Nature Physics, 2016. find
  34. Schütt, Kristof T, Arbabzadah, Farhad, Chmiela, Stefan, Müller, Klaus R, and Tkatchenko, Alexandre. Quantum-chemical insights from deep tensor neural networks. Nature Communications, 8, 2017. find
  35. Stillinger, Frank H. and Weber, Thomas A. Computer simulation of local order in condensed phases of silicon. Phys. Rev. B, 31:5262–5271, Apr 1985. doi: 10.1103/PhysRevB.31.5262. URL http://link.aps.org/doi/10.1103/PhysRevB.31.5262. — [doi:10.1103/physrevb.31.5262](https://doi.org/10.1103/physrevb.31.5262) ↗
  36. Vinyals, Oriol, Bengio, Samy, and Kudlur, Manjunath. Order matters: Sequence to sequence for sets. arXiv preprint arXiv:1511.06391, 2015. find
  37. Wu, Yonghui, Schuster, Mike, Chen, Zhifeng, Le, Quoc V., Norouzi, Mohammad, Macherey, Wolfgang, Krikun, Maxim, Cao, Yuan, Gao, Qin, Macherey, Klaus, et al. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144, 2016. 10. A… find