Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations

Abstract

We study a new algorithm for solving parabolic partial differential equations (PDEs) and backward stochastic differential equations (BSDEs) in high dimension, which is based on an analogy between the BSDE and reinforcement learning with the gradient of the solution playing the role of the policy function, and the loss function given by the error between the prescribed terminal condition and the solution of the BSDE. The policy function is then approximated by a neural network, as is done in deep reinforcement learning. Numerical results using TensorFlow illustrate the efficiency and accuracy of the studied algorithm for several 100-dimensional nonlinear PDEs from physics and finance such as the Allen–Cahn equation, the Hamilton–Jacobi–Bellman equation, and a nonlinear pricing model for financial derivatives.

Keywords

PDEs / High dimension / Backward stochastic differential equations / Deep learning / Control / Feynman-Kac

References

[1] Bellman, R.: Dynamic programming. Princeton Landmarks in Mathematics. Princeton University Press, Princeton, NJ. Reprint of the 1957 edition, with a new introduction by Stuart Dreyfus (2010)

[2] Bender C, Denk R. A forward scheme for backward SDEs. Stoch. Process. Appl.. 2007, 117 12 1793-1812

[3] Bender, C., Schweizer, N., Zhuo, J.: A primal-dual algorithm for BSDEs. arXiv:1310.3694 (2014)

[4] Bergman YZ. Option pricing with differential interest rates. Rev. Financ. Stud.. 1995, 8 2 475-500

[5] Briand P, Labart C. Simulation of BSDEs by Wiener chaos expansion. Ann. Appl. Probab.. 2014, 24 3 1129-1171

[6] Chassagneux J-F. Linear multistep schemes for BSDEs. SIAM J. Numer. Anal.. 2014, 52 6 2815-2836

[7] Chassagneux J-F, Richou A. Numerical simulation of quadratic BSDEs. Ann. Appl. Probab.. 2016, 26 1 262-304

[8] Crisan D, Manolarakis K. Solving backward stochastic differential equations using the cubature method: application to nonlinear pricing. SIAM J. Financ. Math.. 2012, 3 1 534-571

[9] Darbon J, Osher S. Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere. Res. Math. Sci.. 2016, 3 19 26

[10] Debnath L. Nonlinear Partial Differential Equations for Scientists and Engineers. 2012 3 New York: Birkhäuser/Springer

[11] E, W., Han, J., Jentzen, A.: Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. arXiv:1706.04702 (2017)

[12] E, W., Hutzenthaler, M., Jentzen, A., Kruse, T.: Linear scaling algorithms for solving high-dimensional nonlinear parabolic differential equations. arXiv:1607.03295 (2017)

[13] E, W., Hutzenthaler, M., Jentzen, A., Kruse, T.: On multilevel Picard numerical approximations for high-dimensional nonlinear parabolic partial differential equations and high-dimensional nonlinear backward stochastic differential equations. arXiv:1708.03223 (2017)

[14] Gobet E, Lemor J-P, Warin X. A regression-based Monte Carlo method to solve backward stochastic differential equations. Ann. Appl. Probab.. 2005, 15 3 2172-2202

[15] Gobet E, Turkedjiev P. Linear regression MDP scheme for discrete backward stochastic differential equations under general conditions. Math. Comput.. 2016, 85 299 1359-1391

[16] Gobet E, Turkedjiev P. Adaptive importance sampling in least-squares Monte Carlo algorithms for backward stochastic differential equations. Stoch. Process. Appl.. 2017, 127 4 1171-1203

[17] Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press 2016. http://www.deeplearningbook.org

[18] Han, J., E, W.: Deep learning approximation for stochastic control problems. arXiv:1611.07422 (2016)

[19] Han, J., Jentzen, A., E, W.: Overcoming the curse of dimensionality: solving high-dimensional partial differential equations using deep learning. arXiv:1707.02568 (2017)

[20] Henry-Labordère, P.: Counterparty risk valuation: a marked branching diffusion approach. arXiv:1203.2369 (2012)

[21] Henry-Labordère, P., Oudjane, N., Tan, X., Touzi, N., Warin, X.: Branching diffusion representation of semilinear PDEs and Monte Carlo approximation. arXiv:1603.01727 (2016)

[22] Henry-Labordère P, Tan X, Touzi N. A numerical algorithm for a class of BSDEs via the branching process. Stoch. Process. Appl.. 2014, 124 2 1112-1140

[23] Hinton GE, Deng L, Yu D, Dahl G, Mohamed A, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath T, Kingsbury B. Deep neural networks for acoustic modeling in speech recognition. Sig. Process. Mag.. 2012, 29 82-97

[24] Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the International Conference on Machine Learning (ICML) (2015)

[25] Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the International Conference on Learning Representations (ICLR) (2015)

[26] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst.. 2012, 25 1097-1105

[27] LeCun Y, Bengio Y, Hinton GE. Deep learning. Nature. 2015, 521 436-444

[28] Pardoux, É., Peng, S.: Adapted solution of a backward stochastic differential equation. Syst. Control Lett. 14(1), 55–61 (1990)

[29] Pardoux, É., Peng, S.: Backward stochastic differential equations and quasilinear parabolic partial differential equations. In: Stochastic Partial Differential Equations and Their Applications (Charlotte, NC, 1991), vol. 176 of Lecture Notes in Control and Inform. Sci. Springer, Berlin, pp. 200–217 (1992)

[30] Pardoux É, Tang S. Forward-backward stochastic differential equations and quasilinear parabolic PDEs. Probab. Theory Relat. Fields. 1999, 114 2 123-150

[31] Peng S. Probabilistic interpretation for systems of quasilinear parabolic partial differential equations. Stoch. Stoch. Rep.. 1991, 37 1–2 61-74

Funding

U.S. Department of Energy(DOE DE-SC0009248)

U.S. Deparment of Energy(DOE DE-SC0009248)

Office of Naval Research Global(ONR N00014-13-1-0338)

National Natural Science Foundation of China(Major Program, Grant 91130005)