Systematic assessment of various universal machine-learning interatomic potentials

Abstract

Machine-learning interatomic potentials have revolutionized materials modeling at the atomic scale. Thanks to these, it is now indeed possible to perform simulations of ab initio quality over very large time and length scales. More recently, various universal machine-learning models have been proposed as an out-of-box approach avoiding the need to train and validate specific potentials for each particular material of interest. In this paper, we review and evaluate four different universal machine-learning interatomic potentials (uMLIPs), all based on graph neural network architectures which have demonstrated transferability from one chemical system to another. The evaluation procedure relies on data both from a recent verification study of density-functional-theory implementations and from the Materials Project. Through this comprehensive evaluation, we aim to provide guidance to materials scientists in selecting suitable models for their specific research problems, offer recommendations for model selection and optimization, and stimulate discussion on potential areas for improvement in current machine-learning methodologies in materials science.

Keywords

formation energy / geometry optimization / machine learning / phonons / universal machine-learning interatomic potentials / verification

References

1	Frenkel D, Smit B. Understanding Molecular Simulation: From Algorithms to Applications. Elsevier;2023.
2	Hohenberg P, Kohn W. Inhomogeneous electron gas. Phys Rev. 1964;136(3B):B864-B871.
3	Kohn W, Sham LJ. Self-consistent equations including exchange and correlation effects. Phys Rev. 1965;140(4A):A1133-A1138.
4	Payne MC, Teter MP, Allan DC, Arias T, Joannopoulos AJ. Iterative minimization techniques for ab initio total-energy calculations: molecular dynamics and conjugate gradients. Rev Mod Phys. 1992;64(4):1045-1097.
5	Marzari N, Ferretti A, Wolverton C. Electronic-structure methods for materials design. Nat Mater. 2021;20(6):736-749.
6	Goddard I. Classical Force Fields and Methods of Molecular Dynamics. Springer International Publishing;2021:1063-1072.
7	Zuo Y, Chen C, Li X, et al. Performance and cost assessment of machine learning interatomic potentials. J Phys Chem A. 2020;124(4):731-745.
8	Deringer VL, Caro MA, Csányi G. Machine learning interatomic potentials as emerging tools for materials science. Adv Mater. 2019;31(46).
9	Behler J, Parrinello M. Generalized neural-network representation of high-dimensional potential-energy surfaces. Phys Rev Lett. 2007;98(14):146401.
10	Bartók AP, Payne MC, Kondor R, Csányi G. Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons. Phys Rev Lett. 2010;104(13):136403.
11	Thompson A, Swiler L, Trott C, Foiles S, Tucker G. Spectral neighbor analysis method for automated generation of quantum-accurate interatomic potentials. J Comput Phys. 2015;285:316-330.
12	Schütt KT, Sauceda HE, Kindermans PJ, Tkatchenko A, Müller KR. SchNet – a deep learning architecture for molecules and materials. J Chem Phys. 2018;148(24).
13	Drautz R. Atomic cluster expansion for accurate and transferable interatomic potentials. Phys Rev B. 2019;99(1):014104.
14	Lilienfeld VOA, Burke K. Retrospective on a decade of machine learning for chemical discovery. Nat Commun. 2020;11(1):4895.
15	Batzner S, Musaelian A, Sun L, et al. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat Commun. 2022;13(1):2453.
16	Ko TW, Ong SP. Recent advances and outstanding challenges for machine learning interatomic potentials. Nat Comput Sci. 2023;3(12):998-1000.
17	Deringer VL, Bartók AP, Bernstein N, Wilkins DM, Ceriotti M, Csányi G. Gaussian process regression for materials and molecules. Chem Rev. 2021;121(16):10073-10141.
18	Geiger M, Smidt T. e3nn: Euclidean neural networks. arXiv preprint arXiv:2207.09453. 2022.
19	Batatia I, Kovacs DP, Simm G, Ortner C, Csanyi G. MACE: higher order equivariant message passing neural networks for fast and accurate force fields. In: Koyejo S, Mohamed S, Agarwal A, Belgrave D, Cho K, Oh A, eds. Advances in Neural Information Processing Systems 35 (NeurIPS 2022). 35. NeurIPS. Curran Associates, Inc.;2022:11423-11436.
20	Behler J. Neural network potential-energy surfaces in chemistry: a tool for large-scale simulations. Phys Chem Chem Phys. 2011;13(40):17930-17955.
21	Behler J. First principles neural network potentials for reactive simulations of large molecular and condensed systems. Angew Chem Int Ed. 2017;56(42):12828-12840.
22	Behler J. Four generations of high-dimensional neural network potentials. Chem Rev. 2021;121(16):10037-10072.
23	Chen C, Ye W, Zuo Y, Zheng C, Ong SP. Graph networks as a universal machine learning framework for molecules and crystals. Chem Mater. 2019;31(9):3564-3572.
24	Jain A, Ong SP, Hautier G, et al. Commentary: the Materials Project: a materials genome approach to accelerating materials innovation. APL Mater. 2013;1(1).
25	Perdew JP, Burke K, Ernzerhof M. Generalized gradient approximation made simple. Phys Rev Lett. 1996;77(18):3865-3868.
26	Chen C, Ong SP. A universal graph deep learning interatomic potential for the periodic table. Nat Comput Sci. 2022;2(11):718-728.
27	Deng B, Zhong P, Jun K, et al. CHGNet as a pretrained universal neural network potential for charge-informed atomistic modelling. Nat Mach Intell. 2023;5(9):1031-1041.
28	https://github.com/materialsvirtuallab/matgl. Accessed:2024-02-29.
29	Qi J, Ko TW, Wood BC, Pham TA, Ong SP. Robust training of machine learning interatomic potentials with dimensionality reduction and stratified sampling. NPJ Comput Mater. 2024;10(1):43.
30	Choudhary K, DeCost B, Major L, Butler K, Thiyagalingam J, Tavazza F. Unified graph neural network force-field for the periodic table: solid state applications. Digit Discov. 2023;2(2):346-355.
31	Choudhary K, Garrity KF, Reid ACE, et al. The joint automated repository for various integrated simulations (JARVIS) for data-driven materials design. NPJ Comput Mater. 2020;6(1):173.
32	Klimeš J, Bowler DR, Michaelides A. Chemical accuracy for the van der Waals density functional. J Phys Condens Matter. 2009;22(2):022201.
33	Batatia I, Benner P, Chiang Y, et al. A foundation model for atomistic materials chemistry. arXiv preprint arXiv:2401.00096. 2023.
34	Merchant A, Batzner S, Schoenholz SS, Aykol M, Cheon G, Cubuk ED. Scaling deep learning for materials discovery. Nature. 2023;624(7990):80-85.
35	Takamoto S, Shinagawa C, Motoki D, et al. Towards universal neural network potential for material discovery applicable to arbitrary combination of 45 elements. Nat Commun. 2022;13(1):2991.
36	Takamoto S, Okanohara D, Li QJ, Li J. Towards universal neural network interatomic potential. J Materiomics. 2023;9(3):447-454.
37	Takamoto S, Izumi S, Li J. TeaNet: universal neural network interatomic potential inspired by iterative electronic relaxations. Comput Mater Sci. 2022;207:111280.
38	Smith JS, Isayev O, Roitberg AE. ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost. Chem Sci. 2017;8(4):3192-3203.
39	Smith JS, Nebgen BT, Zubatyuk R, et al. Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning. Nat Commun. 2019;10(1):2903.
40	Zubatyuk R, Smith JS, Leszczynski J, Isayev O. Accurate and transferable multitask prediction of chemical properties with an atoms-inmolecules neural network. Sci Adv. 2019;5(8).
41	Lopanitsyna N, Fraux G, Springer MA, De S, Ceriotti M. Modeling high-entropy transition metal alloys with alchemical compression. Phys Rev Mater. 2023;7(4):045802.
42	Riebesell J, Goodall RE, Jain A, Benner P, Persson KA, Lee AA. Matbench Discovery–An evaluation framework for machine learning crystal stability prediction. arXiv preprint arXiv:2308.14920. 2023.
43	Focassio B, Freitas LPM, Schleder GR. Performance assessment of universal machine learning interatomic potentials: challenges and directions for materials’surfaces. arXiv preprint arXiv:2403.04217. 2024.
44	Deng B, Choi Y, Zhong P, et al. Overcoming systematic softening in universal machine learning interatomic potentials by fine-tuning. arXiv preprint arXiv:2405.07105. 2024.
45	Bosoni E, Beal L, Bercx M, et al. How to verify the precision of density-functional-theory implementations via reproducible and universal workflows. Nat Rev Phys. 2023;6(1):45-58.
46	Petretto G, Dwaraknath S, Miranda H PC, et al. High-throughput density-functional perturbation theory phonons for inorganic materials. Sci Data. 2018;5(1):180065.
47	Gonze X, Amadon B, Antonius G, et al. The Abinit project: impact, environment and recent developments. Comput Phys Commun. 2020;248:107042.
48	Romero AH, Allan DC, Amadon B, et al. ABINIT: overview and focus on selected capabilities. J Chem Phys. 2020;152(12).
49	Hamann DR. Optimized norm-conserving Vanderbilt pseudopotentials. Phys Rev B. 2013;88(8):085117.
50	Setten vM, Giantomassi M, Bousquet E, et al. The PseudoDojo: training and grading a 85 element optimized norm-conserving pseudopotential table. Comput Phys Commun. 2018;226:39-54.
51	Perdew JP, Ruzsinszky A, Csonka GI, et al. Restoring the densitygradient expansion for exchange in solids and surfaces. Phys Rev Lett. 2008;100(13):136406.
52	Kresse G, Joubert D. From ultrasoft pseudopotentials to the projector augmented-wave method. Phys Rev B. 1999;59(3):1758-1775.
53	Chen C, Zuo Y, Ye W, Li X, Ong SP. Learning properties of ordered and disordered materials from multi-fidelity data. Nat Comput Sci. 2021;1(1):46-53.
54	Choudhary K, DeCost B, Chen C, et al. Recent advances and applications of deep learning methods in materials science. NPJ Comput Mater. 2022;8(1):59.
55	Batatia I, Batzner S, Kovács DP, et al. The design space of e (3)-equivariant atom-centered interatomic potentials. arXiv preprint arXiv:2205.06643. 2022.
56	Choudhary K, DeCost B. Atomistic line graph neural network for improved materials property predictions. NPJ Comput Mater. 2021;7(1):185.
57	Wu Z, Pan S, Chen F, Long G, Zhang C, Philip SY. A comprehensive survey on graph neural networks. IEEE Transact Neural Networks Learn Syst. 2020;32(1):4-24.
58	Asif NA, Sarker Y, Chakrabortty RK, et al. Graph neural network: a comprehensive review on non-euclidean space. IEEE Access. 2021;9:60588-60606.
59	Mueller T, Hernandez A, Wang C. Machine learning for interatomic potential models. J Chem Phys. 2020;152(5).
60	Tersoff J. New empirical approach for the structure and energy of covalent systems. Phys Rev B. 1988;37(12):6991-7000.
61	Wang A, Kingsbury R, McDermott M, et al. A framework for quantifying uncertainty in DFT energy corrections. Sci Rep. 2021;11(1):15496.
62	Jain A, Hautier G, Ong SP, et al. Formation enthalpies by mixing GGA and GGA calculations. Phys Rev B. 2011;84(4):045115.
63	Materials Project. Accessed:2024-06-15 https://docs.materialsproject.org/methodology/materials-methodology/calculation-details/gga+ucalculations/hubbard-u-values
64	Reed J, Ceder G. Role of electronic structure in the susceptibility of metastable transition-metal oxide structures to transformation. Chem Rev. 2004;104(10):4513-4534.
65	Dusson G, Bachmayr M, Csányi G, et al. Atomic cluster expansion: completeness, efficiency and stability. J Comput Phys. 2022;454:110946.
66	Drautz R. Atomic cluster expansion of scalar, vectorial, and tensorial properties including magnetism and charge transfer. Phys Rev B. 2020;102(2):024104.
67	Wang M, Zheng D, Ye Z, et al. Deep graph library: a graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315. 2019.
68	CHGNet pretrained. Accessed: 2024-06-15. https://github.com/CederGroupHub/chgnet/blob/main/chgnet/pretrained/0.3.0/chgnet_0.3.0_e29f68s314m37.pth.tar
69	Materials Project Trajectory (MPtrj) Dataset. figshare. Accessed:2024-06-15.
70	M3GNet-DIRECT pretrained. Accessed:2024-06-15. https://github.com/materialsvirtuallab/matgl/tree/main/pretrained_models/M3GNet-MP-2021.2.8-DIRECT-PES
71	ACEsuit/mace-mp. Accessed:2024-06-15. https://github.com/ACEsuit/mace-mp/releases/tag/mace_mp_0
72	ACEsuit/mace-mp. Accessed:2024-06-15. https://github.com/ACEsuit/mace/blob/main/mace/calculators/foundations_models.py
73	ALIGNN pretrained. Accessed:2024-06-15. https://github.com/usnistgov/alignn/blob/main/alignn/ff/ff.py
74	ALIGNN pretrained. Accessed:2024-06-15. https://figshare.com/ndownloader/files/41583594
75	ALIGNN pretrained. Accessed:2024-06-15. https://github.com/usnistgov/alignn/blob/main/alignn/models/alignn_atomwise.py
76	Wortmann D, Michalicek G, Hilgers R, et al. FLEUR;2023.
77	Blaha P, Schwarz K, Tran F, Laskowski R, Madsen GKH, Marks LD. WIEN2k: an APW+lo program for calculating the properties of solids. J Chem Phys. 2020;152(7):074101.
78	Blöchl PE. Projector augmented-wave method. Phys Rev B. 1994;50(24):17953-17979.
79	Lejaeghere K, Bihlmayer G, Björkman T, et al. Reproducibility in density functional theory calculations of solids. Science. 2016;351(6280):aad3000.
80	Riebesell J. Pymatviz: visualization toolkit for materials informatics;2022.
81	Ong SP, Richards WD, Jain A, et al. Python Materials Genomics (pymatgen): a robust, open-source python library for materials analysis. Comput Mater Sci. 2013;68:314-319.
82	Pymatgen Compatibility corrections entry. Accessed:2024-06-24. https://github.com/materialsproject/pymatgen/blob/master/pymatgen/entries/compatibility.py
83	Togo A, Chaput L, Tadano T, Tanaka I. Implementation strategies in phonopy and phono3py. J Phys Condens Matter. 2023;35(35):353001.
84	Togo A. First-principles phonon calculations with phonopy and phono3py. J Phys Soc Jpn. 2023;92(1):012001.
85	Gonze X, Lee C. Dynamical matrices, Born effective charges, dielectric permittivity tensors, and interatomic force constants from density-functional perturbation theory. Phys Rev B. 1997;55(16):10355-10368.
86	Baroni S, Gironcoli dS, Dal Corso A, Giannozzi P. Phonons and related crystal properties from density-functional perturbation theory. Rev Mod Phys. 2001;73(2):515-562.
87	Setyawan W, Curtarolo S. High-throughput electronic band structure calculations: challenges and tools. Comput Mater Sci. 2010;49(2):299-312.
88	Hjorth Larsen A, Jørgen Mortensen J, Blomqvist J, et al. The atomic simulation environment—a Python library for working with atoms. J Phys Condens Matter. 2017;29(27):273002.