Adaptive Importance Sampling for Deep Ritz

Abstract

We introduce an adaptive sampling method for the Deep Ritz method aimed at solving partial differential equations (PDEs). Two deep neural networks are used. One network is employed to approximate the solution of PDEs, while the other one is a deep generative model used to generate new collocation points to refine the training set. The adaptive sampling procedure consists of two main steps. The first step is solving the PDEs using the Deep Ritz method by minimizing an associated variational loss discretized by the collocation points in the training set. The second step involves generating a new training set, which is then used in subsequent computations to further improve the accuracy of the current approximate solution. We treat the integrand in the variational loss as an unnormalized probability density function (PDF) and approximate it using a deep generative model called bounded KRnet. The new samples and their associated PDF values are obtained from the bounded KRnet. With these new samples and their associated PDF values, the variational loss can be approximated more accurately by importance sampling. Compared to the original Deep Ritz method, the proposed adaptive method improves the accuracy, especially for problems characterized by low regularity and high dimensionality. We demonstrate the effectiveness of our new method through a series of numerical experiments.

Keywords

Importance sampling / Deep Ritz method / Bounded KRnet

References

[1] CarlierG, GalichonA, SantambrogioF. From Knothe’s transport to Brenier’s map and a continuation method for optimal transport. SIAM J. Math. Anal., 2010, 4162554-2576.

[2] Chen, J., Du, R., Wu, K.: A comparison study of deep Galerkin method and deep Ritz method for elliptic problems with different boundary conditions. arXiv:2005.04554 (2020)

[3] EWN, YuB. The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun. Math. Stat., 2018, 611-12.

[4] Gao, Z., Tang, T., Yan, L., Zhou, T.: Failure-informed adaptive sampling for PINNs, part II: combining with re-sampling and subset simulation. Commun. Appl. Math. Comput., (2023). https://doi.org/10.1007/s42967-023-00312-7

[5] GaoZ, YanL, ZhouT. Failure-informed adaptive sampling for PINNs. SIAM J. Sci. Comput., 2023, 4541971-1994.

[6] Han, J., Jentzen, A., E, W.N.: Solving high-dimensional partial differential equations using deep learning. Proc. Natl. Acad. Sci. 115(34), 8505–8510 (2018)

[7] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

[8] LiuM, CaiZ, RamaniK. Deep Ritz method with adaptive quadrature for linear elasticity. Comput. Methods Appl. Mech. Eng., 2023, 415. 116229

[9] LuL, MengX, MaoZ, KarniadakisGE. DeepXDE: a deep learning library for solving differential equations. SIAM Rev., 2021, 631208-228.

[10] MorinP, NochettoRH, SiebertKG. Convergence of adaptive finite element methods. SIAM Rev., 2002, 444631-658.

[11] PapamakariosG, NalisnickE, RezendeDJ, MohamedS, LakshminarayananB. Normalizing flows for probabilistic modeling and inference. J. Mach. Learn. Res., 2021, 2212617-2680

[12] RaissiM, PerdikarisP, KarniadakisGE. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys., 2019, 378: 686-707.

[13] Santambrogio, F.: Optimal Transport for Applied Mathematicians. Birkäuser, Cham (2015)

[14] SirignanoJ, SpiliopoulosK. DGM: a deep learning algorithm for solving partial differential equations. J. Comput. Phys., 2018, 375: 1339-1364.

[15] StrangG, FixGJAn Analysis of the Finite Element Method, 1971Englewood CliffsPrentice-Hall

[16] TangK, WanX, YangC. DAS-PINNs: a deep adaptive sampling method for solving high-dimensional partial differential equations. J. Comput. Phys., 2023, 476. 111868

[17] WuC, ZhuM, TanQ, KarthaY, LuL. A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks. Comput. Methods Appl. Mech. Eng., 2023, 403. 115671

[18] ZangY, BaoG, YeX, ZhouH. Weak adversarial networks for high-dimensional partial differential equations. J. Comput. Phys., 2020, 411. 109409

[19] Zeng, L., Wan, X., Zhou, T.: Bounded krnet and its applications to density estimation and approximation. arXiv:2305.09063 (2023)

RIGHTS & PERMISSIONS

Shanghai University