MetaGIN: a lightweight framework for molecular property prediction

Abstract

Recent advancements in AI-based synthesis of small molecules have led to the creation of extensive databases, housing billions of small molecules. Given this vast scale, traditional quantum chemistry (QC) methods become inefficient for determining the chemical and physical properties of such an extensive array of molecules. To address this challenge, we present MetaGIN, a lightweight deep learning framework designed for efficient and accurate molecular property prediction.

While traditional GNN models with 1-hop edges (i.e., covalent bonds) are sufficient for abstract graph representation, they are inadequate for capturing 3D features. Our MetaGIN model shows that including 2-hop and 3-hop edges (representing bond and torsion angles, respectively) is crucial to fully comprehend the intricacies of 3D molecules. Moreover, MetaGIN is a streamlined model with fewer than 10 million parameters, making it ideal for fine-tuning on a single GPU. It also adopts the widely acknowledged MetaFormer framework, which has consistently shown high accuracy in many computer vision tasks.

In our experiments, MetaGIN achieved a mean absolute error (MAE) of 0.0851 with just 8.87M parameters on the PCQM4Mv2 dataset, outperforming leading techniques across several datasets in the MoleculeNet benchmark. These results demonstrate MetaGIN’s potential to significantly accelerate drug discovery processes by enabling rapid and accurate prediction of molecular properties for large-scale databases.

Graphical abstract

Keywords

molecule property prediction / quantum chemistry / graph convolution / graph neural network / deep learning

References

[1] Lin X, Li X, Lin X . A review on applications of computational methods in drug screening and design. Molecules, 2020, 25( 6): 1375

[2] Hann M M, Leach A R, Harper G . Molecular complexity and its impact on the probability of finding leads for drug discovery. Journal of Chemical Information and Computer Sciences, 2001, 41( 3): 856–864

[3] Manallack D T, Prankerd R J, Yuriev E, Oprea T I, Chalmers D K . The significance of acid/base properties in drug discovery. Chemical Society Reviews, 2013, 42( 2): 485–496

[4] Geerlings P, De Proft F, Langenaeker W . Conceptual density functional theory. Chemical Reviews, 2003, 103( 5): 1793–1874

[5] Motta M, Zhang S . Ab initio computations of molecular systems by the auxiliary-field quantum Monte Carlo method. WIREs Computational Molecular Science, 2018, 8( 5): e1364

[6] Kümmel H G . A biography of the coupled cluster method. International Journal of Modern Physics B, 2003, 17( 28): 5311–5325

[7] Zhang X, Chen C, Meng Z, Yang Z, Jiang H, Cui X. CoAtGIN: marrying convolution and attention for graph-based molecule property prediction. In: Proceedings of 2022 IEEE International Conference on Bioinformatics and Biomedicine. 2022, 374−379

[8] Wang Z, Wang Y, Zhang X, Meng Z, Yang Z, Zhao W, Cui X. Graph-based reaction classification by contrasting between precursors and products. In: Proceedings of 2022 IEEE International Conference on Bioinformatics and Biomedicine. 2022, 354−359

[9] Hu W, Fey M, Ren H, Nakata M, Dong Y, Leskovec J. OGB-LSC: a large-scale challenge for machine learning on graphs. In: Proceedings of the 35th Conference on Neural Information Processing Systems. 2021

[10] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000−6010

[11] Liu L, He D, Fang X, Zhang S, Wang F, He J, Wu H. GEM-2: next generation molecular property prediction network by modeling full-range many-body interactions. 2022, arXiv preprint arXiv: 2208.05863

[12] Dwivedi V P, Luu A T, Laurent T, Bengio Y, Bresson X. Graph neural networks with learnable structural and positional representations. In: Proceedings of the 10th International Conference on Learning Representations. 2022

[13] Hussain S, Zaki M J, Subramanian D. Global self-attention as a replacement for graph convolution. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2022, 655−665

[14] Park W, Chang W, Lee D, Kim J, Hwang S W. GRPE: relative positional encoding for graph transformer. 2022, arXiv preprint arXiv: 2201.12787

[15] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[16] Xu K, Hu W, Leskovec J, Jegelka S. How powerful are graph neural networks? In: Proceedings of the 7th International Conference on Learning Representations. 2019

[17] Thiel W . Semiempirical quantum–chemical methods. WIREs Computational Molecular Science, 2014, 4( 2): 145–157

[18] Bannwarth C, Caldeweyher E, Ehlert S, Hansen A, Pracht P, Seibert J, Spicher S, Grimme S . Extended tight-binding quantum chemistry methods. WIREs Computational Molecular Science, 2021, 11( 2): e1493

[19] Feng J, Chen Y, Li F, Sarkar A, Zhang M. How powerful are K-hop message passing graph neural networks. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 345

[20] Nikolentzos G, Dasoulas G, Vazirgiannis M . k-hop graph neural networks. Neural Networks, 2020, 130: 195–205

[21] Irwin J J, Tang K G, Young J, Dandarchuluun C, Wong B R, Khurelbaatar M, Moroz Y S, Mayfield J, Sayle R A . ZINC20—a free ultralarge-scale chemical database for ligand discovery. Journal of Chemical Information and Modeling, 2020, 60( 12): 6065–6073

[22] Pence H E, Williams A . ChemSpider: an online chemical information resource. Journal of Chemical Education, 2010, 87( 11): 1123–1124

[23] Hu W, Fey M, Zitnik M, Dong Y, Ren H, Liu B, Catasta M, Leskovec J. Open graph benchmark: Datasets for machine learning on graphs. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 1855

[24] Yu W, Luo M, Zhou P, Si C, Zhou Y, Wang X, Feng J, Yan S. MetaFormer is actually what you need for vision. In: Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 10809−10819

[25] Wu Y, He K. Group normalization. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 3−19

[26] Shazeer N. GLU variants improve transformer. 2020, arXiv preprint arXiv: 2002.05202

[27] Wu Z, Ramsundar B, Feinberg E N, Gomes J, Geniesse C, Pappu A S, Leswing K, Pande V . MoleculeNet: a benchmark for molecular machine learning. Chemical Science, 2018, 9( 2): 513–530

[28] Xie X, Zhou P, Li H, Lin Z, Yan S. Adan: adaptive nesterov momentum algorithm for faster optimizing deep models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, doi: 10.1109/TPAMI.2024.3423382

[29] Rampášek L, Galkin M, Dwivedi V P, Luu A T, Wolf G, Beaini D. Recipe for a general, powerful, scalable graph transformer. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 1054

[30] Ramsundar B, Kearnes S, Riley P, Webster D, Konerding D, Pande V. Massively multitask networks for drug discovery. 2015, arXiv preprint arXiv: 1502.02072

[31] Rogers D, Hahn M . Extended-connectivity fingerprints. Journal of Chemical Information and Modeling, 2010, 50( 5): 742–754

[32] Kearnes S, McCloskey K, Berndl M, Pande V, Riley P . Molecular graph convolutions: Moving beyond fingerprints. Journal of Computer-Aided Molecular Design, 2016, 30( 8): 595–608

[33] Schütt K T, Kindermans P J, Sauceda H E, Chmiela S, Tkatchenko A, Müller K R. SchNet: a continuous-filter convolutional neural network for modeling quantum interactions. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 992−1002

[34] Lu C, Liu Q, Wang C, Huang Z, Lin P, He L. Molecular property prediction: a multilevel quantum interactions modeling perspective. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 1052−1060

[35] Xiong Z, Wang D, Liu X, Zhong F, Wan X, Li X, Li Z, Luo X, Chen K, Jiang H, Zheng M . Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism. Journal of Medicinal Chemistry, 2020, 63( 16): 8749–8760

[36] Liaw R, Liang E, Nishihara R, Moritz P, Gonzalez J E, Stoica I. Tune: a research platform for distributed model selection and training. 2018, arXiv preprint arXiv: 1807.05118

[37] Gilmer J, Schoenholz S S, Riley P F, Vinyals O, Dahl G E. Neural message passing for quantum chemistry. In: Proceedings of the 34th International Conference on Machine Learning. 2017, 1263−1272

[38] Yang K, Swanson K, Jin W, Coley C, Eiden P, Gao H, Guzman-Perez A, Hopper T, Kelley B, Mathea M, Palmer A, Settels V, Jaakkola T, Jensen K, Barzilay R . Analyzing learned molecular representations for property prediction. Journal of Chemical Information and Modeling, 2019, 59( 8): 3370–3388

[39] Hajiabolhassan H, Taheri Z, Hojatnia A, Yeganeh Y T . FunQG: molecular representation learning via quotient graphs. Journal of Chemical Information and Modeling, 2023, 63( 11): 3275–3287

[40] Mastropietro A, Pasculli G, Feldmann C, Rodríguez-Pérez R, Bajorath J . EdgeSHAPer: bond-centric Shapley value-based explanation method for graph neural networks. iScience, 2022, 25( 10): 105043

[41] Mastropietro A, Pasculli G, Bajorath J . Protocol to explain graph neural network predictions using an edge-centric Shapley value-based approach. STAR Protocols, 2022, 3( 4): 101887

RIGHTS & PERMISSIONS

Higher Education Press