Machine Learning Approaches for the Solution of the Riemann Problem in Fluid Dynamics: a Case Study

Abstract

We present our results by using a machine learning (ML) approach for the solution of the Riemann problem for the Euler equations of fluid dynamics. The Riemann problem is an initial-value problem with piecewise-constant initial data and it represents a mathematical model of the shock tube. The solution of the Riemann problem is the building block for many numerical algorithms in computational fluid dynamics, such as finite-volume or discontinuous Galerkin methods. Therefore, a fast and accurate approximation of the solution of the Riemann problem and construction of the associated numerical fluxes is of crucial importance. The exact solution of the shock tube problem is fully described by the intermediate pressure and mathematically reduces to finding a solution of a nonlinear equation. Prior to delving into the complexities of ML for the Riemann problem, we consider a much simpler formulation, yet very informative, problem of learning roots of quadratic equations based on their coefficients. We compare two approaches: (i) Gaussian process (GP) regressions, and (ii) neural network (NN) approximations. Among these approaches, NNs prove to be more robust and efficient, although GP can be appreciably more accurate (about

30\%

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$30\%$$\end{document}

). We then use our experience with the quadratic equation to apply the GP and NN approaches to learn the exact solution of the Riemann problem from the initial data or coefficients of the gas equation of state (EOS). We compare GP and NN approximations in both regression and classification analysis and discuss the potential benefits and drawbacks of the ML approach.

Keywords

Machine learning (ML) / Neural network (NN) / Gaussian process (GP) / Riemann problem / Numerical fluxes / Finite-volume method

References

[1.] Machine Learning for Computational Fluid and Solid Dynamics. Santa Fe, NM, USA, 19–21 (2019)

[2.] Scientific Machine Learning. ICERM, Providence, RI, USA, January 28–30 (2019)

[3.] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). Software available from tensorflow.org

[4.] Bar-Sinai Y, Hoyer S, Hickey J, Brenner MP. Learning data-driven discretizations for partial differential equations. PNAS, 2019, 116: 15344-15349,

[5.] Beck, C., E, W.N., Jentzen, A.: Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations. Journal of Nonlinear Science 29, 1563–1619 (2019)

[6.] Berg J, Nystróm K. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing, 2018, 317: 28-41,

[7.] Brunton S, Proctor J, Kutz N. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl. Acad. Sci., 2016, 113(15): 3932-3937,

[8.] Cotter NE. The Stone-Weierstrass theorem and its application to neural networks. IEEE Trans. Neural Networks, 1990, 1: 290-295,

[9.] Csái, B.C.: Approximation with artificial neural networks. MSc Thesis, Eotvos Lorand University (ELTE), Budapest, Hungary (2001)

[10.] Du J, Xu Y. Hierarchical deep neural network for multivariate regression. Pattern Recogn., 2017, 63: 149-157,

[11.] François, F., et al.: Keras (2015) https://keras.io

[12.] Golak, S.: A MLP solver for first and second order partial differential equations. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D. (eds.) Artificial Neural Networks-ICANN 2007, pp. 789–797. Springer-Verlag, Berlin, Heidelberg (2007)

[13.] Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge, MA (2016)

[14.] Goulianas K, Margaris A, Refanidis I, Diamantaras K, Papadimitriou T. A back propagation-type neural network architecture for solving the complete n × \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\times$$\end{document} n nonlinear algebraic system of equations. Adv. Pure Math., 2016, 6: 455-480,

[15.] Gyrya, V., Shashkov, M., Skurikhin, A.: Exploration of machine learning for polynomial root finding. https://www.researchgate.netpublication/331101795_Exploration_of_Machine_learning_for_Polynomial_Root_Finding_Motivation

[16.] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 770–778. IEEE (2016)

[17.] Hornik K, Stinchcombe M, White H. Multilayer feedforward networks are universal approximators. Neural Netw., 1989, 2: 359-366,

[18.] Huang DS, Ip HHS, Chi Z. A neural root finder of polynomials based on root moments. Neural Comput., 2004, 16: 1721-1762,

[19.] Kumar M, Yadav N. Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations: a survey. Comput. Math. Appl., 2011, 62: 3796-3811,

[20.] Kurková V. Kolmogorov’s theorem and multilayer neural networks. Neural Netw., 1992, 5: 501-506,

[21.] Lagaris IE, Likas A, Fotiadis DI. Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans. Neural Networks, 1998, 9: 987-1000,

[22.] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521: 436-444,

[23.] Lin, H., Jegelka, S.: ResNet with one-neuron hidden layers is a universal approximator. In: Proc. of the Intern. Conf. on Neural Information Processing Systems, pp. 6172–6181 (2018)

[24.] Long, Z., Lu, Y., Ma, X., Dong, B.: PDE-Net: learning PDEs from data. In: Proc. the 35th Intern. Conf. on Machine Learning. PMLR (2018)

[25.] Lu, Z., Pu, H., Wang, F., Hu, Z., Wang, L.: The expressive power of neural networks: a view from width. In: Proc. of the Intern. Conf. on Neural Information Processing Systems, pp. 6232–6240 (2017)

[26.] Lye, K. O., Mishra, S., Ray, D.: Deep learning observables in computational fluid dynamics. J. Comput. Phys. 410, 109339 (2020)

[27.] Meade AJ, Fernandez AA. Solution of nonlinear ordinary differential equations by feedforward neural networks. Mathl. Comput. Modeling, 1994, 20: 19-44,

[28.] Mishra S. A machine learning framework for data driven acceleration of computations of differential equations. Mathematics in Engineering, 2018, 1: 118-146,

[29.] Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E. Scikit-learn: machine learning in Python. J. Mach. Learn. Res., 2011, 12: 2825-2830

[30.] Qin T, Wu K, Xiu D. Data driven governing equations approximation using deep neural networks. J. Comput. Phys., 2019, 395: 620-635,

[31.] Raissi, M., Babee, H., Karniadakis, G. E.: Parametric Gaussian process regression for big data. Comput. Mech. 64, 409–416 (2019)

[32.] Raissi M, Karniadakis GE. Hidden physics models: machine learning of nonlinear partial differential equations. J. Comput. Phys., 2018, 357: 125-141,

[33.] Raissi M, Perdikaris P, Karniadakis GE. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys., 2019, 378: 686-707,

[34.] Rasmussen CE, Williams CKI. . Gaussian Processes for Machine Learning, 2006 Cambridge, MA Adaptive Computation and Machine Learning. MIT Press

[35.] Ray D, Hesthaven J. An artificial neural network as a troubled-cell indicator. J. Comput. Phys., 2018, 367: 166-191,

[36.] Ray D, Hesthaven J. Detecting troubled-cells on two-dimensional unstructured grids using a neural network. J. Comput. Phys., 2018, 397,

[37.] Rumelhart, D. E., McClelland, J.L.: PDP Research Group. Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 1: Foundations. MIT Press, Cambridge, MA (1986)

[38.] Sirignano J, Spiliopoulos K. DGM: a deep learning algorithm for solving partial differential equations. J. Comput. Phys., 2018, 375: 1339-1364,

[39.] Specht DF. A general regression neural network. IEEE Trans. Neural Networks, 1991, 2: 568-576,

[40.] Tokareva, S., Shashkov, M., Skurikhin, A.: Machine learning approach for the solution of the Riemann problem in fluid dynamics. https://www.researchgate.net/publication/330798897_Machine_learning_approach_for_the_solution_of_the_Riemann_problem_in_fluid_dynamics

[41.] Tompson, J., Schlachter, K., Sprechmann, P., Perlin, K.: Accelerating Eulerian fluid simulation with convolutional networks. In: Proc. 34th Intern. Conf. on Machine Learning (2017)

[42.] Toro, E. F.: Riemann Solvers and Numerical Methods for Fluid Dynamics. Springer-Verlag, Berlin, Heidelberg (2009)

[43.] Veiga, H.M., Abgrall, R.: Towards a general stabilisation method for conservation laws using a multilayer perceptron neural network: 1D scalar and system of equations. In: ECCM-ECFD 2018 6th European Conference on Computational Mechanics (Solids, Structures and Coupled Problems) 7th European Conference on Computational Fluid Dynamics, Glasgow, United Kingdom (2018)

[44.] Winovich, N., Ramani, K., Lin, G.: ConvPDE-UQ: convolutional neural networks with quantified uncertainty for heterogeneous elliptic partial differential equations on varied domains. 394, 263–279 (2019)

[45.] Xie Y, Franz E, Chu M, Thuerey N. tempoGAN: A temporally coherent, volumetric GAN for super-resolution fluid flow. ACM Transactions on Graphics, 2018, 37(95): 1-15

Funding

U.S. Department of Energy