MetaGIN: a lightweight framework for molecular property prediction

Abstract

Recent advancements in AI-based synthesis of small molecules have led to the creation of extensive databases, housing billions of small molecules. Given this vast scale, traditional quantum chemistry (QC) methods become inefficient for determining the chemical and physical properties of such an extensive array of molecules. To address this challenge, we present MetaGIN, a lightweight deep learning framework designed for efficient and accurate molecular property prediction.

While traditional GNN models with 1-hop edges (i.e., covalent bonds) are sufficient for abstract graph representation, they are inadequate for capturing 3D features. Our MetaGIN model shows that including 2-hop and 3-hop edges (representing bond and torsion angles, respectively) is crucial to fully comprehend the intricacies of 3D molecules. Moreover, MetaGIN is a streamlined model with fewer than 10 million parameters, making it ideal for fine-tuning on a single GPU. It also adopts the widely acknowledged MetaFormer framework, which has consistently shown high accuracy in many computer vision tasks.

In our experiments, MetaGIN achieved a mean absolute error (MAE) of 0.0851 with just 8.87M parameters on the PCQM4Mv2 dataset, outperforming leading techniques across several datasets in the MoleculeNet benchmark. These results demonstrate MetaGIN’s potential to significantly accelerate drug discovery processes by enabling rapid and accurate prediction of molecular properties for large-scale databases.

Graphical abstract

Keywords

molecule property prediction / quantum chemistry / graph convolution / graph neural network / deep learning

Xuan Zhang received the bachelor degree from Shandong University, China. He is currently pursuing the PhD degree at the School of Computer Science and Technology, Shandong University, China. His research interests include bioinformatics, deep learning, and drug discovery

Cheng Chen received the bachelor, master degrees at Qingdao University of Science and Technology, China. He is currently a PhD student at Shandong University. His research interests include bioinformatics, data mining, and deep learning

Xiaoting Wang is an undergraduate at the Taishan College of Shandong University, China. She has a keen interest in bioinformatics and artificial intelligence

Haitao Jiang received the PhD degree in computer science from Shandong University, China in 2011. He is currently a professor with the School of Computer Science and Technology, Shandong University, China. From 2009 to 2011, he was a visiting scholar with Montana State University. His research interests include the design and analysis of algorithm, bioinformatics, and computational biology

Wei Zhao ia an associate professor at the State Key Laboratory of Microbial Technology, Shandong University, China. He received the PhD degree of Food Nutrition and Safety from Huazhong Agricultural University, China. His research focuses on AI-assisted design and druggability evaluation of small molecule antitumor drugs, the development of specific tumor-targeted allosteric inhibitors and the structural biology of complexes. He also commits to break through the druggability bottleneck problems of the traditional leading compounds with cytotoxicity

Xuefeng Cui received the bachelor’s, master’s and the PhD degrees from the David R. Cheriton School of Computer Science, the University of Waterloo, Canada. He is currently a full professor with the School of Computer Science and Technology, Shandong University, China. His research interests include designing and implementing machine learning algorithms and parallel algorithms to solve computational biology problems

References

[1] Lin X, Li X, Lin X . A review on applications of computational methods in drug screening and design. Molecules, 2020, 25( 6): 1375

[2] Hann M M, Leach A R, Harper G . Molecular complexity and its impact on the probability of finding leads for drug discovery. Journal of Chemical Information and Computer Sciences, 2001, 41( 3): 856–864

[3] Manallack D T, Prankerd R J, Yuriev E, Oprea T I, Chalmers D K . The significance of acid/base properties in drug discovery. Chemical Society Reviews, 2013, 42( 2): 485–496

[4] Geerlings P, De Proft F, Langenaeker W . Conceptual density functional theory. Chemical Reviews, 2003, 103( 5): 1793–1874

[5] Motta M, Zhang S . Ab initio computations of molecular systems by the auxiliary-field quantum Monte Carlo method. WIREs Computational Molecular Science, 2018, 8( 5): e1364

[6] Kümmel H G . A biography of the coupled cluster method. International Journal of Modern Physics B, 2003, 17( 28): 5311–5325

[7] Zhang X, Chen C, Meng Z, Yang Z, Jiang H, Cui X. CoAtGIN: marrying convolution and attention for graph-based molecule property prediction. In: Proceedings of 2022 IEEE International Conference on Bioinformatics and Biomedicine. 2022, 374−379

[8] Wang Z, Wang Y, Zhang X, Meng Z, Yang Z, Zhao W, Cui X. Graph-based reaction classification by contrasting between precursors and products. In: Proceedings of 2022 IEEE International Conference on Bioinformatics and Biomedicine. 2022, 354−359

[9] Hu W, Fey M, Ren H, Nakata M, Dong Y, Leskovec J. OGB-LSC: a large-scale challenge for machine learning on graphs. In: Proceedings of the 35th Conference on Neural Information Processing Systems. 2021

[10] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000−6010

[11] Liu L, He D, Fang X, Zhang S, Wang F, He J, Wu H. GEM-2: next generation molecular property prediction network by modeling full-range many-body interactions. 2022, arXiv preprint arXiv: 2208.05863

[12] Dwivedi V P, Luu A T, Laurent T, Bengio Y, Bresson X. Graph neural networks with learnable structural and positional representations. In: Proceedings of the 10th International Conference on Learning Representations. 2022

[13] Hussain S, Zaki M J, Subramanian D. Global self-attention as a replacement for graph convolution. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2022, 655−665

[14] Park W, Chang W, Lee D, Kim J, Hwang S W. GRPE: relative positional encoding for graph transformer. 2022, arXiv preprint arXiv: 2201.12787

[15] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[16] Xu K, Hu W, Leskovec J, Jegelka S. How powerful are graph neural networks? In: Proceedings of the 7th International Conference on Learning Representations. 2019

[17] Thiel W . Semiempirical quantum–chemical methods. WIREs Computational Molecular Science, 2014, 4( 2): 145–157

[18] Bannwarth C, Caldeweyher E, Ehlert S, Hansen A, Pracht P, Seibert J, Spicher S, Grimme S . Extended tight-binding quantum chemistry methods. WIREs Computational Molecular Science, 2021, 11( 2): e1493

[19] Feng J, Chen Y, Li F, Sarkar A, Zhang M. How powerful are K-hop message passing graph neural networks. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 345

[20] Nikolentzos G, Dasoulas G, Vazirgiannis M . k-hop graph neural networks. Neural Networks, 2020, 130: 195–205

[21] Irwin J J, Tang K G, Young J, Dandarchuluun C, Wong B R, Khurelbaatar M, Moroz Y S, Mayfield J, Sayle R A . ZINC20—a free ultralarge-scale chemical database for ligand discovery. Journal of Chemical Information and Modeling, 2020, 60( 12): 6065–6073

[22] Pence H E, Williams A . ChemSpider: an online chemical information resource. Journal of Chemical Education, 2010, 87( 11): 1123–1124

[23] Hu W, Fey M, Zitnik M, Dong Y, Ren H, Liu B, Catasta M, Leskovec J. Open graph benchmark: Datasets for machine learning on graphs. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 1855

[24] Yu W, Luo M, Zhou P, Si C, Zhou Y, Wang X, Feng J, Yan S. MetaFormer is actually what you need for vision. In: Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 10809−10819

[25] Wu Y, He K. Group normalization. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 3−19

[26] Shazeer N. GLU variants improve transformer. 2020, arXiv preprint arXiv: 2002.05202

[27] Wu Z, Ramsundar B, Feinberg E N, Gomes J, Geniesse C, Pappu A S, Leswing K, Pande V . MoleculeNet: a benchmark for molecular machine learning. Chemical Science, 2018, 9( 2): 513–530

[28] Xie X, Zhou P, Li H, Lin Z, Yan S. Adan: adaptive nesterov momentum algorithm for faster optimizing deep models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, doi: 10.1109/TPAMI.2024.3423382

[29] Rampášek L, Galkin M, Dwivedi V P, Luu A T, Wolf G, Beaini D. Recipe for a general, powerful, scalable graph transformer. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 1054

[30] Ramsundar B, Kearnes S, Riley P, Webster D, Konerding D, Pande V. Massively multitask networks for drug discovery. 2015, arXiv preprint arXiv: 1502.02072

[31] Rogers D, Hahn M . Extended-connectivity fingerprints. Journal of Chemical Information and Modeling, 2010, 50( 5): 742–754

[32] Kearnes S, McCloskey K, Berndl M, Pande V, Riley P . Molecular graph convolutions: Moving beyond fingerprints. Journal of Computer-Aided Molecular Design, 2016, 30( 8): 595–608

[33] Schütt K T, Kindermans P J, Sauceda H E, Chmiela S, Tkatchenko A, Müller K R. SchNet: a continuous-filter convolutional neural network for modeling quantum interactions. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 992−1002

[34] Lu C, Liu Q, Wang C, Huang Z, Lin P, He L. Molecular property prediction: a multilevel quantum interactions modeling perspective. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 1052−1060

[35] Xiong Z, Wang D, Liu X, Zhong F, Wan X, Li X, Li Z, Luo X, Chen K, Jiang H, Zheng M . Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism. Journal of Medicinal Chemistry, 2020, 63( 16): 8749–8760

[36] Liaw R, Liang E, Nishihara R, Moritz P, Gonzalez J E, Stoica I. Tune: a research platform for distributed model selection and training. 2018, arXiv preprint arXiv: 1807.05118

[37] Gilmer J, Schoenholz S S, Riley P F, Vinyals O, Dahl G E. Neural message passing for quantum chemistry. In: Proceedings of the 34th International Conference on Machine Learning. 2017, 1263−1272

[38] Yang K, Swanson K, Jin W, Coley C, Eiden P, Gao H, Guzman-Perez A, Hopper T, Kelley B, Mathea M, Palmer A, Settels V, Jaakkola T, Jensen K, Barzilay R . Analyzing learned molecular representations for property prediction. Journal of Chemical Information and Modeling, 2019, 59( 8): 3370–3388

[39] Hajiabolhassan H, Taheri Z, Hojatnia A, Yeganeh Y T . FunQG: molecular representation learning via quotient graphs. Journal of Chemical Information and Modeling, 2023, 63( 11): 3275–3287

[40] Mastropietro A, Pasculli G, Feldmann C, Rodríguez-Pérez R, Bajorath J . EdgeSHAPer: bond-centric Shapley value-based explanation method for graph neural networks. iScience, 2022, 25( 10): 105043

[41] Mastropietro A, Pasculli G, Bajorath J . Protocol to explain graph neural network predictions using an edge-centric Shapley value-based approach. STAR Protocols, 2022, 3( 4): 101887

Acknowledgment

This research has been facilitated by the National Key R&D Program of China (Nos. 2019YFA0905700 and 2021YFC2101500) and the National Natural Science Foundation of China (Grant No. 62072283).

Competing interests

The authors declare that they have no competing interests or financial conflicts to disclose.

RIGHTS & PERMISSIONS

2025 Higher Education Press