Solving Time-Dependent Parametric PDEs by Multiclass Classification-Based Reduced Order Model

Abstract

In this paper, we propose a network model, the multiclass classificationbased reduced order model (MC-ROM), for solving time-dependent parametric partial differential equations (PPDEs). This work is inspired by the observation of applying the deep learning-based reduced order model (DL-ROM) [14] to solve diffusiondominant PPDEs. We find that the DL-ROM has a good approximation for some special model parameters, but it cannot approximate the drastic changes of the solution as time evolves. Based on this fact, we classify the dataset according to the magnitude of the solutions and construct corresponding subnets dependent on different types of data. Then we train a classifier to integrate different subnets together to obtain the MC-ROM. When subsets have the same architecture, we can use transfer learning techniques to accelerate offline training. Numerical experiments show that the MC-ROM improves the generalization ability of the DL-ROM both for diffusion- and convectiondominant problems, and maintains the DL-ROM's advantage of good approximation ability. We also compare the approximation accuracy and computational efficiency of the proper orthogonal decomposition (POD) which is not suitable for convectiondominant problems. For diffusion-dominant problems, the MC-ROM has better approximation accuracy than the POD in a small dimensionality reduction space, and its computational performance is more efficient than the POD's.

Keywords

Parametric partial differential equation / reduced order model / deep learning / generalization ability / classification / computational complexity

References

[1] S.E. Ahmed, O. San, A. Rasheed, and T. Iliescu, A long short-term memory embedding for hybrid uplifted reduced order models, Physica D, p. 132471, 2020.

[2] M. Alnæs, J. Blechta, J. Hake, A. Johansson, B. Kehlet, A. Logg, C. Richardson, J. Ring, M.E. Rognes, and G.N. Wells, The fenics project version 1.5, Archive of Numerical Software, 3, 2015.

[3] C. Bao, Q. Li, Z. Shen, C. Tai, L. Wu, and X. Xiang, Approximation analysis of convolutional neural networks, 65, 2014.

[4] P. Benner, S. Gugercin, and K. Willcox, A survey of projection-based model reduction methods for parametric dynamical systems, SIAM Rev., 57:483-531, 2015.

[5] G. Berkooz, P. Holmes, and J.L. Lumley, The proper orthogonal decomposition in the analysis of turbulent flows, Annu. Rev. Fluid Mech., 25:539-575, 1993.

[6] J. Berner, M. Dablander, and P. Grohs, Numerically solving parametric families of high-dimensional Kolmogorov partial differential equations via deep learning, Adv. Neural Inf. Process. Syst., 33, 2020.

[7] K. Bhattacharya, B. Hosseini, N.B. Kovachki, and A.M. Stuart, Model reduction and neural networks for parametric PDEs, arXiv:2005.03180, 2020.

[8] R. Caruana, Multitask learning, Mach. Learn., 28:41-75, 1997.

[9] R.T. Chen, Y. Rubanova, J. Bettencourt, and D.K. Duvenaud, Neural ordinary differential equations, Adv. Neural Inf. Process. Syst., 6571-6583, 2018.

[10] W. Chen, Q. Wang, J. Hesthaven, and C. Zhang, Physics-informed machine learning for reducedorder modeling of nonlinear problems, Preprint, 2020.

[11] D.-A. Clevert, T. Unterthiner, and S. Hochreiter, Fast and accurate deep network learning by exponential linear units (ELUs), arXiv:1511.07289, 2015.

[12] C. Cortes and V. Vapnik, Support-vector networks, Mach. Learn., 20:273-297, 1995.

[13] G. Cybenko, Approximation by superpositions of a sigmoidal function, Math. Control Signals Systems, 2:303-314, 1989.

[14] S. Fresca, L. Dede, and A. Manzoni, A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs, J. Sci. Comput., 87:1-36, 2021.

[15] K. Gallivan, A. Vandendorpe, and P. Van Dooren,Model reduction via tangential interpolation, in:MTNS 2002 (15th Symposium on the Mathematical Theory of Networks and Systems), p.6, 2002.

[16] F.J. Gonzalez and M. Balajewicz, Deep convolutional recurrent autoencoders for learning lowdimensional feature dynamics of fluid systems, arXiv:1808.01346, 2018.

[17] I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep Learning, Vol. 1, MIT Press Cambridge, 2016.

[18] J. Han, A. Jentzen, and E. Weinan,Solving high-dimensional partial differential equations using deep learning, Proc. Natl. Acad. Sci., 115:8505-8510, 2018.

[19] D. Hartman and L.K. Mestha, A deep learning framework for model reduction of dynamical systems, in: 2017 IEEE Conference on Control Technology and Applications (CCTA), IEEE, 1917-1922, 2017.

[20] J. He, L. Li, and J. Xu, Approximation properties of deep relu cnns, arXiv:2109.00190, 2021.

[21] K. He, X. Zhang, S. Ren, and J. Sun, Delving deep into rectifiers: Surpassing human-level performance on imagenet classification, in: Proceedings of the IEEE international conference on computer vision, 1026-1034, 2015.

[22] J.S. Hesthaven et al., Certified Reduced Basis Methods for Parametrized Partial Differential Equations, Vol. 590, Springer, 2016.

[23] J.S. Hesthaven, B. Stamm, and S. Zhang, Efficient greedy algorithms for high-dimensional parameter spaces with applications to empirical interpolation and reduced basis methods, ESAIM: Math. Model. Numer. Anal., 48:259-283, 2014.

[24] J.S. Hesthaven and S. Ubbiali, Non-intrusive reduced order modeling of nonlinear problems using neural networks, J. Comput. Phys., 363:55-78, 2018.

[25] D.P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv:1412.6980, 2014.

[26] G. Kutyniok, P. Petersen, M. Raslan, and R. Schneider, A theoretical analysis of deep neural networks and parametric PDEs, arXiv:1904.00377, 2019.

[27] I.E. Lagaris, A. Likas, and D.I. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE trans. neural netw., 9:987-1000, 1998.

[28] Y. LeCun, Y. Bengio, and G. Hinton, Deep learning, Nature, 521:436-444, 2015.

[29] K. Lee and K.T. Carlberg, Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders, J. Comput. Phys., 404:108973, 2020.

[30] Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, Fourier neural operator for parametric partial differential equations, arXiv:2010.08895, 2020.

[31] Y. Liang, H. Lee, S. Lim, W. Lin, K. Lee, and C. Wu, Proper orthogonal decomposition and its applications-Part i: Theory, J. Sound Vib., 252:527-544, 2002.

[32] L. Lu, P. Jin, and G.E. Karniadakis, Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators, arXiv:1910.03193, 2019.

[33] M. Ohlberger and S. Rave, Reduced basis methods: Success, limitations and future challenges, arXiv:1511.02021, 2015.

[34] T. O'Leary-Roseberry, U. Villa, P. Chen, and O. Ghattas, Derivative-informed projected neural networks for high-dimensional parametric maps governed by PDEs, arXiv:2011.15110, 2020.

[35] S.E. Otto and C.W. Rowley, Linearly recurrent autoencoder networks for learning dynamics, SIAM J. Appl. Dyn. Syst., 18:558-593, 2019.

[36] A. Paszke et al. Pytorch: An imperative style, high-performance deep learning library Adv., Neural Inf. Process. Syst., 8026-8037, 2019.

[37] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, Scikit-learn: Mach. Learn. in Python, J. Mach. Learn. Res., 12:2825-2830, 2011.

[38] P. Petersen and F. Voigtlaender,Equivalence of approximation by convolutional neural networks and fully-connected networks, Proc. Amer. Math. Soc., 148:1567-1581, 2020.

[39] T. Phillips, C.E. Heaney, P.N. Smith, and C.C. Pain, An autoencoder-based reduced-order model for eigenvalue problems with application to neutron diffusion, arXiv:2008.10532, 2020.

[40] A. Quarteroni, A. Manzoni, and F. Negri, Reduced Basis Methods for Partial Differential Equations: An Introduction, Vol. 92, 2015.

[41] M. Raissi, P. Perdikaris, and G.E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378:686-707, 2019.

[42] M. Stein, Large sample properties of simulations using latin hypercube sampling, Technometrics, 29:143-151, 1987.

[43] Q. Wang, J.S. Hesthaven, and D. Ray, Non-intrusive reduced order modeling of unsteady flows using artificial neural networks with application to a combustion problem, J. Comput. Phys., 384:289307, 2019.

[44] S. Wang, H. Wang, and P. Perdikaris, Learning the solution operator of parametric partial differential equations with physics-informed deeponets, arXiv:2103.10974, 2021.

[45] Y. Wang, H. Yao, and S. Zhao, Auto-encoder based dimensionality reduction, Neurocomputing, 184:232-242, 2016.

[46] E. Weinan and B. Yu, The deep ritz method: a deep learning-based numerical algorithm for solving variational problems, Commun. Math. Stat., 6:1-12, 2018.

[47] J. Weston and C. Watkins,Support vector machines for multi-class pattern recognition, ESANN' 1999 proceedings - European Symposium on Artificial Neural Networks, 219-224, 1999.

[48] D.-X. Zhou, Universality of deep convolutional neural networks, Appl. Comput. Harmon. Anal., 48:787-794, 2020.

[49] Y. Zhu and N. Zabaras, Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification, J. Comput. Phys., 366:415-447, 2018.

[50] Y. Zhu, N. Zabaras, P.S. Koutsourelakis, and P. Perdikaris, Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data, J. Comput. Phys., 394:56-81, 2019.