A multiscale differential-algebraic neural network-based method for learning dynamical systems

Abstract

The objective of dynamical system learning tasks is to forecast the future behavior of a system by leveraging observed data. However, such systems can sometimes exhibit rigidity due to significant variations in component parameters or the presence of slow and fast variables, leading to challenges in learning. To overcome this limitation, we propose a multiscale differential-algebraic neural network (MDANN) method that utilizes Lagrangian mechanics and incorporates multiscale information for dynamical system learning. The MDANN method consists of two main components: the Lagrangian mechanics module and the multiscale module. The Lagrangian mechanics module embeds the system in Cartesian coordinates, adopts a differential-algebraic equation format, and uses Lagrange multipliers to impose constraints explicitly, simplifying the learning problem. The multiscale module converts high-frequency components into low-frequency components using radial scaling to learn subprocesses with large differences in velocity. Experimental results demonstrate that the proposed MDANN method effectively improves the learning of dynamical systems under rigid conditions.

Keywords

dynamical systems learning / multibody system dynamics / differential-algebraic equation / neural networks / multiscale structures

References

[1] Meidani K, Farimani AB. Identification of parametric dynamical systems using integer programming. Expert Syst Appl. 2023;219:119622.

[2] Ding JY, Pan ZK, Chen LQ. Second order adjoint sensitivity analysis of multibody systems described by differential-algebraic equations. Multibody Syst Dyn. 2007;18:599-617.

[3] Zheng M. Lie symmetries and conserved quantities of fractional nonconservative singular systems. Int J Mech Syst Dyn. 2023;3:274-279.

[4] Huang W, Zou H, Pan Y, et al. Numerical simulation and behavior prediction of a space net system throughout the capture process: spread, contact, and close. Int J Mech Syst Dyn. 2023;3:265-273.

[5] Eghbali M, Hosseini SA. A complex solution on the dynamic response of sandwich graphene-reinforced aluminum-based composite beams with copper face sheets under two moving constant loads on an elastic foundation. Int J Mech Syst Dyn. 2023;3:251-264.

[6] Rui X, Zhang J, Wang X, Rong B, He B, Jin Z. Multibody system transfer matrix method: the past, the present, and the future. Int J Mech Syst Dyn. 2022;2(1):3-26.

[7] Legaard C, Schranz T, Schweiger G, et al. Constructing neural network-based models for simulating dynamical systems. Acm Comput Surv. 2023;55(11):1-34.

[8] Wang R, Walters R, Yu R. Approximately equivariant networks for imperfectly symmetric dynamics. Paper presented at:39th International Conference on Machine Learning;July 17-23, 2022;Baltimore, MD. https://proceedings.mlr.press/v162/wang22aa.html

[9] Wang K, Aanjaneya M, Bekris K. A first principles approach for data-efficient system identification of spring-rod systems via differentiable physics engines. Paper presented at:2nd Conference on Learning for Dynamics and Control;June 10-11, 2020;The Cloud. https://proceedings.mlr.press/v120/wang20b.html

[10] Yang TY, Rosca J, Narasimhan K, Ramadge PJ. Learning physics constrained dynamics using autoencoders. Proc Adv Neural Inf Process Syst. 2022;35:17157-17172.

[11] Willard J, Jia X, Xu S, Steinbach M, Kumar V. Integrating scientific knowledge with machine learning for engineering and environmental systems. Acm Comput Surv. 2022;55(4):1-37.

[12] Lutter M, Ritter C, Peters J. Deep Lagrangian networks: Using physics as model prior for deep learning. Paper presented at:7th International Conference on Learning Representations;May 6-9, 2019;New Orleans, LA. https://openreview.net/forum?id=BklHpjCqKm

[13] Yu H, Lu J, Zhang G. An online robust support vector regression for data streams. Ieee T Knowl Data En. 2020;34(1):150-163.

[14] Shamshirband S, Hashemi S, Salimi H, et al. Predicting standardized streamflow index for hydrological drought using machine learning models. Eng Appl Comp Fluid. 2020;14(1):339-350.

[15] Gomila R. Logistic or linear? Estimating causal effects of experimental treatments on binary outcomes using regression analysis. J Exp Psychol Gen. 2021;150(4):700-709.

[16] Brunton SL, Proctor JL, Kutz JN. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. P Natl Acad Sci Usa. 2016;113(15):3932-3937.

[17] Schaeffer H, McCalla SG. Sparse model selection via integral terms. Phys Rev E. 2017;96(2):023302.

[18] Reinbold PA, Gurevich DR, Grigoriev RO. Using noisy or incomplete data to discover models of spatiotemporal dynamics. Phys Rev E. 2020;101(1):010203.

[19] Berg J, Nyström K. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing. 2018;317:28-41.

[20] Gong S, Meng Q, Wang Y, et al. Incorporating NODE with pretrained neural differential operator for learning dynamics. Neurocomputing. 2023;528:48-58.

[21] Lu L, Jin P, Pang G, Zhang Z, Karniadakis GE. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nat Mach Intell. 2021;3(3):218-229.

[22] Finzi M, Stanton S, Izmailov P, Wilson AG. Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data. Paper presented at:37th International Conference on Machine Learning;July 13-18, 2020;Virtual. https://proceedings.mlr.press/v119/finzi20a.html

[23] Wang H, Liu Y, Wang S. Dense velocity reconstruction from particle image velocimetry/particle tracking velocimetry using a physicsinformed neural network. Phys Fluids. 2022;34(1):017116.

[24] Lin S, Chen Y. A two-stage physics-informed neural network method based on conserved quantities and applications in localized wave solutions. J Comput Phys. 2022;457:111053.

[25] Haghighat E, Raissi M, Moure A, Gomez H, Juanes R. A physicsinformed deep learning framework for inversion and surrogate modeling in solid mechanics. Comput Method Appl Mech Eng. 2021;379:113741.

[26] Goswami S, Anitescu C, Chakraborty S, Rabczuk T. Transfer learning enhanced physics informed neural network for phase-field modeling of fracture. Theor Appl Fract Mec. 2020;106:102447.

[27] Wang S, Wang H, Perdikaris P. Learning the solution operator of parametric partial differential equations with physics-informed deeponets. Sci Adv. 2021;7(40):eabi8605.

[28] Raissi M, Perdikaris P, Karniadakis GE. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys. 2019;378:686-707.

[29] Chen RT, Rubanova Y, Bettencourt J, Duvenaud DK. Neural ordinary differential equations. Proc Adv Neural Inf Process Syst. 2018;31:6571-6583.

[30] Cunha BZ, Droz C, Zine AM, Foulard S, Ichchou M. A review of machine learning methods applied to structural dynamics and vibroacoustic. Mech Syst Signal Pr. 2023;200:110535.

[31] Baddoo PJ, Herrmann B, McKeon BJ, Nathan Kutz J, Brunton SL. Physics-informed dynamic mode decomposition. P Roy Soc A-Math Phy. 2023;479(2271):20220576.

[32] Greydanus S, Dzamba M, Yosinski J. Hamiltonian neural networks. Proc Adv Neural Inf Process Syst. 2019;32:15379-15389.

[33] Cranmer M, Greydanus S, Hoyer S, Battaglia P, Spergel D, Ho S. Lagrangian neural networks. Paper presented at: Advances in Neural Information Processing Systems 33;March 26-27, 2020;The Cloud. https://openreview.net/forum?id=iE8tFa4Nq

[34] Finzi M, Wang KA, Wilson AG. Simplifying Hamiltonian and Lagrangian neural networks via explicit constraints. Proc Adv Neural Inf Process Syst. 2020;33:13880-13889.

[35] Lu Y, Lin S, Chen G, Pan J. Modlanets: Learning generalisable dynamics via modularity and physical inductive bias. Paper presented at:39th International Conference on Machine Learning;July 17-23, 2022;Baltimore, MD. https://proceedings.mlr.press/v162/lu22c.html

[36] Gruver N, Finzi MA, Stanton SD, Wilson AG. Deconstructing the inductive biases of Hamiltonian neural networks. Paper presented at:10th International Conference on Learning Representations;April 25-29, 2022;Virtual. https://openreview.net/forum?id=EDeVYpT42oS

[37] Karniadakis GE, Kevrekidis IG, Lu L, Perdikaris P, Wang S, Yang L. Physics-informed machine learning. Nat Rev Phys. 2021;3(6):422-440.

[38] Wang B. Multi-Scale Deep Neural Network (MscaleDNN) methods for oscillatory stokes flows in complex domains. Commun Comput Phys. 2020;28(5):2139-2157.

[39] Liu Z. Multi-Scale Deep Neural Network (MscaleDNN) for solving Poisson-Boltzmann equation in complex domains. Commun Comput Phys. 2020;28(5):1970-2001.

[40] Ding J, Liu J. Optimal control based on modified genetic algorithm for the deployment process of scissor-type deployable mast. Paper presented at:8th Asian Conference on Multibody Dynamics;August 7-10, 2016;Kyoto, KP. https://www.jstage.jst.go.jp/article/jsmeacmd/2016.8/0/2016.8_29_1289270/_article/-char/ja/

RIGHTS & PERMISSIONS

2024 The Authors. International Journal of Mechanical System Dynamics published by John Wiley & Sons Australia, Ltd on behalf of Nanjing University of Science and Technology.