Neural Network-Based Variational Methods for Solving Quadratic Porous Medium Equations in High Dimensions

Abstract

In this paper, we propose and study neural network-based methods for solutions of high-dimensional quadratic porous medium equation (QPME). Three variational formulations of this nonlinear PDE are presented: a strong formulation and two weak formulations. For the strong formulation, the solution is directly parameterized with a neural network and optimized by minimizing the PDE residual. It can be proved that the convergence of the optimization problem guarantees the convergence of the approximate solution in the $L^1$ sense. The weak formulations are derived following (Brenier in Examples of hidden convexity in nonlinear PDEs, 2020) which characterizes the very weak solutions of QPME. Specifically speaking, the solutions are represented with intermediate functions who are parameterized with neural networks and are trained to optimize the weak formulations. Extensive numerical tests are further carried out to investigate the pros and cons of each formulation in low and high dimensions. This is an initial exploration made along the line of solving high-dimensional nonlinear PDEs with neural network-based methods, which we hope can provide some useful experience for future investigations.

Keywords

Quadratic porous medium equation / High-dimensional nonlinear PDE / Neural network / Variational formulation / Deep learning

References

[1] Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al.: $\{$TensorFlow$\}$: a system for $\{$Large-Scale$\}$ machine learning. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pp. 265–283 (2016)

[2] Barron AR. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inf. Theory. 1993, 39 3 930-945

[3] Bing, Yu., et al.: The deep ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun. Math. Stat. 6(1), 1–12 (2018)

[4] Boelens AM, Venturi D, Tartakovsky DM. Parallel tensor methods for high-dimensional linear PDEs. J. Comput. Phys.. 2018, 375 519-539

[5] Brenier, Y.: Examples of hidden convexity in nonlinear PDEs (2020)

[6] Dektor A, Venturi D. Dynamically orthogonal tensor methods for high-dimensional nonlinear PDEs. J. Comput. Phys.. 2020, 404

[7] Dektor A, Rodgers A, Venturi D. Rank-adaptive tensor methods for high-dimensional nonlinear PDEs. J. Sci. Comput.. 2021, 88 2 1-27

[8] Dolgov S, Kalise D, Kunisch KK. Tensor decomposition methods for high-dimensional Hamilton-Jacobi-Bellman equations. SIAM J. Sci. Comput.. 2021, 43 3 A1625-A1650

[9] Eigel M, Neumann J, Schneider R, Wolf S. Non-intrusive tensor reconstruction for high-dimensional random PDEs. Comput. Methods Appl. Math.. 2019, 19 1 39-53

[10] Han J, Jentzen A . Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Commun. Math. Stat.. 2017, 5 4 349-380

[11] Han J, Jentzen A, Weinan E. Solving high-dimensional partial differential equations using deep learning. Proc. Natl. Acad. Sci.. 2018, 115 34 8505-8510

[12] Henry-Labordere, P.: Counterparty risk valuation: a marked branching diffusion approach. Available at SSRN 1995503 (2012)

[13] Henry-Labordere, P., Oudjane, N., Tan, X., Touzi, N., Warin, X.: Branching diffusion representation of semilinear PDEs and Monte Carlo approximation. In: Annales de l’Institut Henri Poincaré, Probabilités et Statistiques, vol. 55, pp. 184–210. Institut Henri Poincaré (2019)

[14] Jentzen, A., Salimova, D., Welti, T.: A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients. arXiv preprint arXiv:1809.07321 (2018)

[15] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

[16] Lagaris IE, Likas A, Fotiadis DI. Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans. Neural Netw.. 1998, 9 5 987-1000

[17] Leibenzon, L.S.: The motion of a gas in a porous medium. Complete Works, Vol. 2, Acad. Sci. URSS Moscow (Russian) 63, 8–9 (1930)

[18] Lu, J., Lu, Y., Wang, M.: A priori generalization analysis of the deep ritz method for solving high dimensional elliptic partial differential equations. In: Conference on Learning Theory, pp. 3196–3241. PMLR (2021)

[19] Marsaglia G. Choosing a point from the surface of a sphere. Ann. Math. Stat.. 1972, 43 2 645-646

[20] Muskat M. The flow of homogeneous fluids through porous media. Soil Sci.. 1938, 46 2 169

[21] Ngo C, Huang W. A study on moving mesh finite element solution of the porous medium equation. J. Comput. Phys.. 2017, 331 357-380

[22] Raissi M, Perdikaris P, Karniadakis GE. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. phys.. 2019, 378 686-707

[23] Richter, L., Sallandt, L., Nüsken, N.: Solving high-dimensional parabolic PDEs using the tensor train format. In: International Conference on Machine Learning, pp. 8998–9009. PMLR (2021)

[24] Shmarev, S.: Interfaces in solutions of diffusion-absorption equations in arbitrary space dimension, pp. 257–273. Trends in Partial Differential Equations of Mathematical Physics (2005)

[25] Shukla MB, Kok R, Prasher SO, Clark G, Lacroix R. Use of artificial neural networks in transient drainage design. Trans. ASAE. 1996, 39 1 119-124

[26] Vázquez JL. The Porous Medium Equation: Mathematical Theory. 2007 Oxford: Oxford University Press

[27] Warin, X.: Variations on branching methods for non linear PDEs. arXiv preprint arXiv:1701.07660 (2017)

[28] Warin X. Nesting Monte Carlo for high-dimensional non-linear PDEs. Monte Carlo Methods Appl.. 2018, 24 4 225-247

[29] Wells, B.V.: A moving mesh finite element method for the numerical solution of partial differential equations and systems. PhD thesis, Citeseer (2004)

[30] Zang Y, Bao G, Ye X, Zhou H. Weak adversarial networks for high-dimensional partial differential equations. J. Comput. Phys.. 2020, 411

Funding

National Science Foundation(DMS-2012286)

U.S. Department of Energy(DE-SC0019449)