Discovery of Governing Equations with Recursive Deep Neural Networks

Abstract

Model discovery based on existing data has been one of the major focuses of mathematical modelers for decades. Despite tremendous achievements in model identification from adequate data, how to unravel the models from limited data is less resolved. This paper focuses on the model discovery problem when the data is not efficiently sampled in time, which is common due to limited experimental accessibility and labor/resource constraints. Specifically, we introduce a recursive deep neural network (RDNN) for data-driven model discovery. This recursive approach can retrieve the governing equation efficiently and significantly improve the approximation accuracy by increasing the recursive stages. In particular, our proposed approach shows superior power when the existing data are sampled with a large time lag, from which the traditional approach might not be able to recover the model well. Several examples of dynamical systems are used to benchmark this newly proposed recursive approach. Numerical comparisons confirm the effectiveness of this recursive neural network for model discovery. The accompanying codes are available at https://github.com/c2fd/RDNNs.

Keywords

Deep neural networks (DNNs) / Recursive / Model discovery / Dynamical systems / 92B20 / 37M99 / 37N30

References

[1] Anitescu C, Atroshchenko E, Alajlan N, Rabczuk T. Artificial neural network methods for the solution of second order boundary value problems. Comput. Mater. Contin.. 2019, 59(1): 345-359

[2] Berg J, Nystrom K. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing. 2018, 317: 28-41

[3] Berg J, Nystrom K. Data-driven discovery of PDEs in complex datasets. J. Comput. Phys.. 2019, 284: 239-252

[4] Brunton SL, Proctor JL, Kutz JN. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl. Acad. Sci.. 2016, 113(15): 3932-3937

[5] Chen Z, Xiu D. On generalized residue network for deep learning of unknown dynamical systems. J. Comput. Phys.. 2021, 438 110362

[6] Daniels BC, Nemenman I. Efficient inference of parsimonious phenomenological models of cellular dynamics using S-systems and alternating regression. PLoS One. 2015, 10(3) e0119821

[7] Dockhorn, T.: A discussion on solving partial differential equations using neural networks. arXiv:1904.07200 (2019)

[8] E W, Yu B. The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun. Math. Stat.. 2018, 6: 1-12

[9] Han J, Jentzen A, E W. Solving high-dimensional partial differential equations using deep learning. Proc. Natl. Acad. Sci.. 2018, 115(34)): 8505-8510

[10] Higham C, Higham D. Deep learning: an introduction for applied mathematicians. SIAM Rev.. 2019, 61(4): 860-891

[11] Jin P, Zhu A, Karniadakis GE, Tang Y. Symplectic networks: intrinsic structure-preserving networks for identifying Hamiltonian systems. Neural Netw.. 2020, 132: 166-179

[12] Koopman BO, Neumann JV. Dynamical systems of continuous spectra. Proc. Natl. Acad. Sci. USA. 1932, 18(3): 255-263

[13] Kutz JN, Fu X, Brunton SL. Multi-resolution dynamic mode decomposition. SIAM J. Appl. Dyn. Syst.. 2016, 15(2): 713-735

[14] Li M, Jiang L. Deep learning nonlinear multiscale dynamic problems using Koopman operator. J. Comput. Phys.. 2021, 446 110660

[15] Liu Y, Kutz JN, Brunton SL. Hierarchical deep learning of multiscale differential equation time-steppers. Philos. Trans. A. 2021, 380: 20210200

[16] Long, Z., Lu, Y., Ma, X., Dong, B.: PDE-Net: learning PDEs from data. In: Proceedings of the 35th International Conference on Machine Learning, vol 80, pp. 3208–3216 (2018)

[17] Lu L, Meng X, Mao Z, Karniadakis GE. DeepXDE: a deep learning library for solving differential equations. SIAM Rev.. 2021, 63(1): 208-228

[18] Nguyen-Thanh V, Zhuang X, Rabczuk T. A deep energy method for finite deformation hyperelasticity. Eur. J. Mech.. 2020, 80 103874

[19] Pinkus A. Approximation theory of the MLP model in neural networks. Acta Numer.. 1999, 8: 143-195

[20] Qin T, Chen Z, Jakeman J, Xiu D. Data-driven learning of non-autonomous systems. SIAM J. Sci. Comput.. 2021, 43(3): A1607-A1624

[21] Qin T, Wu K, Xiu D. Data driven governing equations approximation using deep neural networks. J. Comput. Phys.. 2019, 395: 620-635

[22] Raissi M. Deep hidden physics models: deep learning of nonlinear partial differential equations. J. Mach. Learn. Res.. 2018, 19: 1-24

[23] Raissi, M., Perdikaris, P., Karniadakis, G.E.: Multistep neural networks for data-driven discovery of nonlinear dynamical systems. arXiv:1801.01236 (2018)

[24] Raissi M, Perdikaris P, Karniadakis GE. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys.. 2019, 378: 686-707

[25] Sirignano J, Spiliopoulos K. DGM: a deep learning algorithm for solving partial differential equations. J. Comput. Phys.. 2018, 375: 1339-1364

[26] Wang, Y.-J., Lin, C.-T.: Runge-Kutta neural network for identification of continuous systems. In: SMC’98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics, vol. 4, pp. 3277–3282 (1998)

[27] Wang Y, Lin C. Runge-Kutta neural network for identification of dynamical systems in high accuracy. IEEE Trans. Neural Netw.. 1998, 9(2): 294-307

[28] Wight CL, Zhao J. Solving Allen-Cahn and Cahn-Hilliard equations using the adaptive physics-informed neural networks. Commun. Comput. Phys.. 2021, 29: 930-954

[29] Yeung, E., Kundu, S., Hodas, N.: Learning deep neural network representations for Koopman operators of nonlinear dynamical systems. In: IEEE 2019 American Control Conference (ACC), pp. 4832–4839 (2019)

[30] Yu H, Tian X, Li Q. OnsagerNet: learning stable and interpretable dynamics using a generalized Onsager principle. Phys. Rev. Fluids. 2021, 6 114402

[31] Zhao J. Discovering phase field models from image data with the pseudo-spectral physics-informed neural networks. Commun. Appl. Math. Comput.. 2021, 3: 357-369

Funding

Directorate for Mathematical and Physical Sciences(1816783)

RIGHTS & PERMISSIONS

Shanghai University