Generative models for inverse design of inorganic solid materials

Abstract

Overwhelming evidence has been accumulating that materials informatics can provide a novel solution for materials discovery. While the conventional approach to innovation relies mainly on experimentation, the generative models stemming from the field of machine learning can realize the long-held dream of inverse design, where properties are mapped to the chemical structures. In this review, we introduce the general aspects of inverse materials design and provide a brief overview of two generative models, variational autoencoder and generative adversarial network, which can be utilized to generate and optimize inorganic solid materials according to their properties. Reversible representation schemes for generative models are compared between molecular and crystalline structures, and challenges in regard to the latter are also discussed. Finally, we summarize the recent application of generative models in the exploration of chemical space with compositional and configurational degrees of freedom, and potential future directions are speculatively outlined.

Keywords

Inverse design / inorganic solid materials / machine learning / generative model

References

[1] Zunger A.Inverse design in search of materials with target functionalities.Nat Rev Chem2018;2

[2] Sanchez-Lengeling B.Inverse molecular design using machine learning: generative models for matter engineering.Science2018;361:360-5

[3] Butler KT,Cartwright H,Walsh A.Machine learning for molecular and materials science.Nature2018;559:547-55

[4] Hong Y,Jiang H.Machine learning and artificial neural network accelerated computational discoveries in materials science.WIREs Comput Mol Sci2020;10

[5] Li S,Chen D,Nie Z.Encoding the atomic structure for machine learning in materials science.WIREs Comput Mol Sci2021;

[6] Tabor DP,Saikin SK.Accelerating the discovery of materials for clean energy in the era of smart automation.Nat Rev Mater2018;3:5-20

[7] Lilienfeld OA, Müller K, Tkatchenko A. Exploring chemical compound space with quantum-based machine learning.Nat Rev Chem2020;4:347-58

[8] Nie Z,Yang L,Pan F.Construction and application of materials knowledge graph based on author disambiguation: revisiting the evolution of LiFePO₄.Adv Energy Mater2021;11:2003580

[9] Noh J,Kim S.Machine-enabled inverse design of inorganic solid materials: promises and challenges.Chem Sci2020;11:4871-81 PMCID:PMC8159218

[10] Coley CW.Defining and exploring chemical spaces.Trends in Chemistry2021;3:133-45

[11] Kim C,Ramprasad R.Machine learning assisted predictions of intrinsic dielectric breakdown strength of ABX₃ perovskites.J Phys Chem C2016;120:14575-80

[12] Furmanchuk A,Choudhary A.Predictive analytics for crystalline materials: bulk modulus.RSC Adv2016;6:95246-51

[13] Kauwe SK,Vazquez A.Machine learning prediction of heat capacity for solid inorganics.Integr Mater Manuf Innov2018;7:43-51

[14] Li W,Morgan D.Predicting the thermodynamic stability of perovskite oxides using machine learning models.Computational Materials Science2018;150:454-63 PMCID:PMC5992996

[15] Zheng X,Zhang RZ.Machine learning material properties from the periodic table using convolutional neural networks.Chem Sci2018;9:8426-32 PMCID:PMC6244172

[16] Raccuglia P,Adler PD.Machine-learning-assisted materials discovery using failed experiments.Nature2016;533:73-6

[17] Weston L,Dagdelen J.Named entity recognition and normalization applied to large-scale information extraction from the materials science literature.J Chem Inf Model2019;59:3692-702

[18] Tshitoyan V,Weston L.Unsupervised word embeddings capture latent knowledge from materials science literature.Nature2019;571:95-8

[19] Gómez-Bombarelli R,Duvenaud D.Automatic chemical design using a data-driven continuous representation of molecules.ACS Cent Sci2018;4:268-76 PMCID:PMC5833007

[20] Popova M,Tropsha A.Deep reinforcement learning for de novo drug design.Sci Adv2018;4:eaap7885

[21] Long T,Opahle I.Constrained crystals deep convolutional generative adversarial network for the inverse design of crystal structures.npj Comput Mater2021;7

[22] Kingma DP.Auto-encoding variational bayes. Available from: https://arxiv.org/abs/1312.6114 [Last accessed on 13 Sep 2021]

[23] Goodfellow IJ,Mirza M. Generative adversarial nets. Available from: https://papers.nips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf [Last accessed on 13 Sep 2021]

[24] Davies DW,Skelton JM,Oganov AR.Computer-aided design of metal chalcohalide semiconductors: from chemical composition to crystal structure.Chem Sci2018;9:1022-30 PMCID:PMC5883896

[25] Zakutayev A,Nagaraja A.Theoretical prediction and experimental realization of new stable inorganic materials using the inverse design approach.J Am Chem Soc2013;135:10048-54

[26] Acosta CM,Dalpian GM.Inverse design of compounds that have simultaneously ferroelectric and Rashba cofunctionality.Phys Rev B2020;102

[27] Noh J,Gu GH.Unveiling new stable manganese based photoanode materials via theoretical high-throughput screening and experiments.Chem Commun (Camb)2019;55:13418-21

[28] Lazauskas T,Woodley SM.An efficient genetic algorithm for structure prediction at the nanoscale.Nanoscale2017;9:3850-64

[29] Alberi K,Zakutayev A.The 2019 materials by design roadmap.J Phys D Appl Phys2018;52:013001

[30] Bergerhoff G,Sievers R.The inorganic crystal structure data base.J Chem Inf Comput Sci1983;23:66-9

[31] Kirklin S,Meredig B.The Open Quantum Materials Database (OQMD): assessing the accuracy of DFT formation energies.npj Comput Mater2015;1

[32] Jain A,Hautier G.Commentary: the materials project: a materials genome approach to accelerating materials innovation.APL Materials2013;1:011002

[33] Meredig B,Kirklin S.Combinatorial screening for new materials in unconstrained composition space with machine learning.Phys Rev B2014;89

[34] Regmi K.Cross-view image synthesis using geometry-guided conditional GANs.Computer Vision and Image Understanding2019;187:102788

[35] Cao J,Yu B,Sun Z.3D aided duet GANs for multi-view face image synthesis.IEEE Trans Inform Forensic Secur2019;14:2028-42

[36] Santurkar S,Tran B.Image synthesis with a single (robust) classifier. Available from: https://arxiv.org/abs/1906.09453 [Last accessed on 13 Sep 2021]

[37] Lao Q,Pesaranghader A,Di Jorio L.Dual adversarial inference for text-to-image synthesis. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019. p. 7567-76.

[38] Cha M,Kung HT.Adversarial learning of semantic relevance in text to image synthesis.AAAI2019;33:3272-9

[39] Li W,Zhang L.Object-driven text-to-image synthesis via adversarial training. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. p. 12174-82.

[40] Wang TC,Tao A,Kautz J. Few-shot video-to-video synthesis. Available from: https://arxiv.org/abs/1910.12713 [Last accessed on 13 Sep 2021]

[41] Kumar K,de Boissiere T. MelGAN: generative adversarial networks for conditional waveform synthesis. Available from: https://arxiv.org/abs/1910.06711 [Last accessed on 13 Sep 2021]

[42] Donahue C,Puckette M. Adversarial audio synthesis. Available from: https://arxiv.org/abs/1802.04208 [Last accessed on 13 Sep 2021]

[43] Bińkowski M,Dieleman S. High fidelity speech synthesis with adversarial networks. Available from: https://arxiv.org/abs/1909.11646 [Last accessed on 13 Sep 2021]

[44] Blaschke T,Engkvist O,Chen H.Application of generative autoencoder in de novo molecular design.Mol Inform2018;37:1700123 PMCID:PMC5836887

[45] Segler MHS,Tyrchan C.Generating focused molecule libraries for drug discovery with recurrent neural networks.ACS Cent Sci2018;4:120-31 PMCID:PMC5785775

[46] Kadurin A,Khrabrov K,Zhavoronkov A.druGAN: an advanced generative adversarial autoencoder model for de novo generation of new molecules with desired molecular properties in silico.Mol Pharm2017;14:3098-104

[47] Kadurin A,Kazennov A.The cornucopia of meaningful leads: Applying deep adversarial autoencoders for new molecule development in oncology.Oncotarget2017;8:10883-90 PMCID:PMC5355231

[48] Weininger D,Weininger JL.SMILES. 2. Algorithm for generation of unique SMILES notation.J Chem Inf Comput Sci1989;29:97-101

[49] Weininger D.SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules.J Chem Inf Model1988;28:31-6

[50] Kearnes S,Berndl M,Riley P.Molecular graph convolutions: moving beyond fingerprints.J Comput Aided Mol Des2016;30:595-608 PMCID:PMC5028207

[51] Pathak Y,Varma G,Priyakumar UD.Deep learning enabled inorganic material generator.Phys Chem Chem Phys2020;22:26935-43

[52] Kim S,Gu GH,Jung Y.Generative adversarial networks for crystal structure prediction.ACS Cent Sci2020;6:1412-20 PMCID:PMC7453563

[53] Hoffmann J,Sawada Y,Sellier JM. Data-driven approach to encoding and decoding 3-D crystal structures. Available from: https://arxiv.org/abs/1909.00949 [Last accessed on 13 Sep 2021]

[54] Noh J,Stein HS.Inverse design of solid-state materials via a continuous representation.Matter2019;1:1370-84

[55] Gupta A,Huisman BJH,Schneider P.Generative recurrent networks for de novo drug design.Mol Inform2018;37:1700111 PMCID:PMC5836943

[56] Guimaraes GL,Farias PLC. Objective-reinforced generative adversarial networks (ORGAN) for sequence generation models. Available from: https://arxiv.org/abs/1705.10843 [Last accessed on 13 Sep 2021]

[57] Sanchez B,Guimaraes G. Optimizing distributions over molecular space. An objective-reinforced generative adversarial network for inverse-design chemistry (ORGANIC). ChemRxiv. Cambridge: Cambridge Open Engage; 2017.

[58] Lim J,Kim JW.Molecular generative model based on conditional variational autoencoder for de novo molecular design.J Cheminform2018;10:31 PMCID:PMC6041224

[59] Prykhodko O,Kotsias PC.A de novo molecular generation method using latent vector based generative adversarial network.J Cheminform2019;11:74 PMCID:PMC6892210

[60] Li Y,Liu Z.Multi-objective de novo drug design with conditional graph generative model.J Cheminform2018;10:33 PMCID:PMC6057868

[61] Li Y,Dyer C,Battaglia P. Learning deep generative models of graphs. Available from: https://arxiv.org/abs/1803.03324 [Last accessed on 13 Sep 2021]

[62] Simonovsky M.GraphVAE: towards generation of small graphs using variational autoencoders. In: Kůrková V, Manolopoulos Y, Hammer B, Iliadis L, Maglogiannis I, editors. Artificial Neural Networks and Machine Learning - ICANN 2018. Cham: Springer International Publishing; 2018. p. 412-22.

[63] Weng M,Qian G.Identify crystal structures by a new paradigm based on graph theory for building materials big data.Sci China Chem2019;62:982-6

[64] Zhao Q,He B.Identifying descriptors for Li+ conduction in cubic Li-argyrodites via hierarchically encoding crystal structure and inferring causality.Energy Storage Materials2021;40:386-93

[65] Keith JA,Cheng B.Combining machine learning and computational chemistry for predictive insights into chemical systems.Chem Rev2021;121:9816-72 PMCID:PMC8391798

[66] Liu Y,Ju W.Materials discovery and design using machine learning.Journal of Materiomics2017;3:159-77

[67] Chen C,Ye W,Deng Z.A critical review of machine learning of energy materials.Adv Energy Mater2020;10:1903242

[68] Lu Z.Computational discovery of energy materials in the era of big data and machine learning: a critical review.Materials Reports: Energy2021;1:100047

[69] Zhao Q,Chen L.Machine learning prediction of activation energy in cubic Li-argyrodites with hierarchically encoding crystal structure-based (HECS) descriptors.Science Bulletin2021;66:1401-8

[70] Gu GH,Lee Y.Progress in computational and machine-learning methods for heterogeneous small-molecule activation.Adv Mater2020;32:e1907865

[71] Lu S,Ouyang Y,Li Q.Accelerated discovery of stable lead-free hybrid organic-inorganic perovskites via machine learning.Nat Commun2018;9:3405 PMCID:PMC6109147

[72] Nouira A,Crivello JC. CrystalGAN: learning to discover crystallographic structures with generative adversarial networks. Available from: https://arxiv.org/abs/1810.11203 [Last accessed on 13 Sep 2021]

[73] Ren Z,Tian S. Inverse design of crystals using generalized invertible crystallographic representation. Available from: https://arxiv.org/abs/2005.07609 [Last accessed on 13 Sep 2021]

[74] Court CJ,Jain A.3-D inorganic crystal structure generation and property prediction via representation learning.J Chem Inf Model2020;60:4518-35 PMCID:PMC7592118

[75] Kim B,Kim J.Inverse design of porous materials using artificial neural networks.Sci Adv2020;6:eaax9324 PMCID:PMC6941911

[76] Davies DW,Jackson AJ.Computational screening of all stoichiometric inorganic materials.Chem2016;1:617-27 PMCID:PMC5074417

[77] Dan Y,Li X,Hu M.Generative adversarial networks (GAN) based efficient sampling of chemical composition space for inverse design of inorganic materials.npj Comput Mater2020;6

[78] Nguyen P,Gupta S,Venkatesh S. Hybrid Generative-Discriminative Models for inverse materials design. Available from: https://arxiv.org/abs/1811.06060 [Last accessed on 13 Sep 2021]

[79] Yao Z,Bobbitt NS.Inverse design of nanoporous crystalline reticular materials with deep generative models.Nat Mach Intell2021;3:76-86

[80] Dong Y,Zhang C.Inverse design of two-dimensional graphene/h-BN hybrids by a regressional and conditional GAN.Carbon2020;169:9-16

[81] Wang Y,Kwok JT.Generalizing from a few examples: a survey on few-shot learning.ACM Comput Surv2020;53:1-34

[82] Weiss K,Wang D.A survey of transfer learning.J Big Data2016;3

[83] Yamada H,Wu S.Predicting materials properties with little data using shotgun transfer learning.ACS Cent Sci2019;5:1717-30 PMCID:PMC6813555

[84] Sohn K,Yan X. Learning structured output representation using deep conditional generative models. In: Cortes C, Lawrence N, Lee D, Sugiyama M, Garnett R, editors. NIPS: Curran Associates, Inc; 2015.

[85] Mirza M. Conditional generative adversarial nets. Available from: https://arxiv.org/abs/1411.1784 [Last accessed on 13 Sep 2021]

[86] Hochreiter S,Frasconi P. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. A field guide to dynamical recurrent neural networks. IEEE Press;2001. p. 237-43.

[87] Kodali N,Hays J. On convergence and stability of GANs. Available from: https://arxiv.org/abs/1705.07215 [Last accessed on 13 Sep 2021]

[88] Browne M.Convolutional neural networks for image processing: an application in robot vision. In: Gedeon TD, Fung LCC, editors. AI 2003: Advances in Artificial Intelligence. Berlin, Heidelberg: Springer Berlin Heidelberg; 2003, p. 641-52.

[89] Arjovsky M,Bottou L. Wasserstein generative adversarial networks. Proceedings of the 34th International Conference on Machine Learning, PMLR; 2017. p. 214-23.

[90] Denton E,Szlam A. Deep generative image models using a laplacian pyramid of adversarial networks. Available from: https://arxiv.org/abs/1506.05751 [Last accessed on 13 Sep 2021]

[91] Berthelot D,Metz L. BEGAN: boundary equilibrium generative adversarial networks. Available from: https://arxiv.org/abs/1703.10717 [Last accessed on 13 Sep 2021]

[92] Brock A,Simonyan K. Large scale GAN training for high fidelity natural image synthesis. Available from: https://arxiv.org/abs/1809.11096 [Last accessed on 13 Sep 2021]

[93] Jolliffe I. Principal Component Analysis. In: Lovric M, editor. International encyclopedia of statistical science. Berlin, Heidelberg: Springer Berlin Heidelberg; 2011, p. 1094-6.

[94] Borg I. Multidimensional Scaling. In: Lovric M, editor. International encyclopedia of statistical science. Berlin, Heidelberg: Springer Berlin Heidelberg; 2011, p. 875-8.

[95] Maaten LVD.Visualizing data using t-SNE.Journal of Machine Learning Research2008;9:2579-605

[96] Ceriotti M,Parrinello M.From the Cover: Simplifying the representation of complex free-energy landscapes using sketch-map.Proc Natl Acad Sci U S A2011;108:13023-8 PMCID:PMC3156203

[97] Jang J,Noh J,Jung Y.Structure-based synthesizability prediction of crystals using partially supervised learning.J Am Chem Soc2020;142:18836-43

[98] Jung J,Park S.Modelling feasibility constraints for materials design: Application to inverse crystallographic texture problem.Computational Materials Science2019;156:361-7

[99] Johnson L.An inverse design framework for prescribing precipitation heat treatments from a target microstructure.2016;107:7-17

[100] Wang S,Lu X,Zhang C.Simultaneous optimization of fixture and cutting parameters of thin-walled workpieces based on particle swarm optimization algorithm.SIMULATION2017;94:67-76

[101] Olivecrona M,Engkvist O.Molecular de-novo design through deep reinforcement learning.J Cheminform2017;9:48 PMCID:PMC5583141