Efficient Legendre Polynomial Neural Networks Method for Time-Fractional Partial Differential Equations with Singularity

Abstract

Compared to multi-layer neural networks based on nonlinear activation functions, single-layer neural networks that use smooth linear functions as activation functions can quickly numerically solve time-fractional partial differential equations (TFPDEs) with time derivative of order $\alpha \in (0,1)$. However, due to the weak singularity of the solution which is not smooth near the initial time, resulting in significant numerical errors when using neural networks based on smooth functions as activation functions. In this paper, we develop a decomposition method to reduce the singularity of the initial value, and theoretically demonstrate that this method can reduce the singularity. Then, a new single-layer neural network and a fast training method are proposed based on the decomposition method and the smooth linear activation function. Numerical experiments have shown that this method reduces errors and improves computational speed.

Keywords

Singularity / Caputo fractional derivative / Caputo-Hadamard fractional derivative / Fitted method / Neural networks / 26A33 / 65D15 / 65F45 / 65M30

References

[1] BrunnerH, HanH, YinD. The maximum principle for time-fractional diffusion equations and its application. Numer. Funct. Anal. Optim., 2015, 36(10): 1307-1321.

[2] Bueno-OrovioA, KayD, BurrageK. Fourier spectral methods for fractional-in-space reaction-diffusion equations. BIT Numer. Math., 2014, 54(4): 937-954.

[3] BurrageK, HaleN, KayD. An efficient implicit FEM scheme for fractional-in-space reaction-diffusion equations. SIAM J. Sci. Comput., 2012, 34(4): A2145-A2172.

[4] CenD, VongS. The tracking of derivative discontinuities for delay fractional equations based on fitted L1 method. Comput. Meth. Appl. Mat., 2023, 23(3): 591-601.

[5] CourantR, HilbertDMethods of Mathematical Physics, 1953, New York. Interscience Publishers.

[6] DohaE, BhrawyA, Ezz-EldienS. A new Jacobi operational matrix: an application for solving fractional differential equations. Appl. Math. Model., 2012, 36(10): 4931-4943.

[7] DohaE, BhrawyA, HafezR. On shifted Jacobi spectral method for high-order multi-point boundary value problems. Commun. Nonlinear Sci. Numer. Simul., 2012, 17(10): 3802-3810.

[8] DuN, WangH. A fast finite element method for space-fractional dispersion equations on bounded domains in $\mathbb{R} ^2$. SIAM J. Sci. Comput., 2015, 37(3): A1614-A1635.

[9] FordNJ, XiaoJ, YanY. A finite element method for time fractional partial differential equations. Fract. Calc. Appl. Anal., 2011, 14: 454-474.

[10] GorenfloR, KilbasAA, MainardiF, RogosinSVMittag-Leffler Functions, 2014, Germany. Related Topics and Applications. Springer.

[11] GraciaJL, O’RiordanE, StynesM. A fitted scheme for a Caputo initial-boundary value problem. J. Sci. Comput., 2018, 76: 583-609.

[12] Guo, L., Wu, H., Yu, X., Zhou, T.: Monte Carlo PINNs: deep learning approach for forward and inverse problems involving high dimensional fractional partial differential equations. https://doi.org/10.48550/arXiv.2203.08501 (2022)

[13] HaoZ, MoongyuP, CaiZ. Neural network method for integral fractional Laplace equations. East Asian J. Appl. Math., 2023, 13: 95-118.

[14] HornikK. Approximation capabilities of multilayer feedforward networks. Neural Netw., 1991, 4(2): 251-257.

[15] IlicM, LiuF, TurnerI, AnhV. Numerical approximation of a fractional-in-space diffusion equation. I. Fract. Calc. Appl. Anal., 2005, 8(3): 323-341

[16] JaradF, AbdeljawadT, BaleanuD. Caputo-type modification of the Hadamard fractional derivatives. Adv. Differ. Equ., 2012, 142: 1-8

[17] JiangY, MaJ. High-order finite element methods for time-fractional partial differential equations. J. Comput. Appl. Math., 2011, 235(11): 3285-3290.

[18] KeR, NgM, SunH. A fast direct method for block triangular Toeplitz-like with tri-diagonal block systems from time-fractional partial differential equations. J. Comput. Phys., 2015, 303: 203-211.

[19] LiC, LiZ, WangZ. Mathematical analysis and the local discontinuous Galerkin method for Caputo-Hadamard fractional partial differential equation. J. Sci. Comput., 2020, 85: 1-27.

[20] LinX, NgM, SunH. A splitting preconditioner for Toeplitz-like linear systems arising from fractional diffusion equations. SIAM J. Matrix Anal. Appl., 2017, 38(4): 1580-1614.

[21] LiuF, ZhuangP, AnhV, TurnerI, BurrageK. Stability and convergence of the difference methods for the space-time fractional advection-diffusion equation. Appl. Math. Comput., 2007, 191(1): 12-20.

[22] Liu, M., Cai Z., Adaptive two-layer ReLU neural network: II. Ritz approximation to elliptic PDEs. Comput. Math. Appl. 113, 103–116 (2022)

[23] LuL, MengX, MaoZ, KarniadakisGE. DeepXDE: a deep learning library for solving differential equations. SIAM Rev., 2019, 63(1): 208-228.

[24] MaL, LiR, ZengF, KarniadakisGE. Bi-orthogonal fPINN: a physics-informed neural network method for solving time-dependent stochastic fractional PDEs. Commun. Comput. Phys., 2023, 34(4): 1133-1176.

[25] Ma, Z., Hou J., Zhu W., Peng Y., Li Y.: PMNN: physical model-driven neural network for solving time-fractional differential equations. Chaos Solitons Fractals 177, 114238 (2023)

[26] MeerschaertM, TadjeranC. Finite difference approximations for fractional advection-dispersion flow equations. J. Comput. Appl. Math., 2004, 172(1): 65-77.

[27] MeerschaertM, TadjeranC. Finite difference approximations for two-sided space-fractional partial differential equations. Appl. Numer. Math., 2006, 56(1): 80-90.

[28] OuC, CenD, VongS, WangZ. Mathematical analysis and numerical methods for Caputo-Hadamard fractional diffusion-wave equations. Appl. Numer. Math., 2022, 177: 34-57.

[29] OuC, CenD, WangZ, VongS. Fitted schemes for Caputo-Hadamard fractional differential equations. Numer. Algorithms, 2023, 97(1): 1-30

[30] PangG, LuL, KarniadakisGE. fPINNs: fractional physics-informed neural networks. SIAM J. Sci. Comput., 2019, 41(4): A2603-A2626.

[31] Podlubny, I.: Fractional Differential Equations: an Introduction to Fractional Derivatives, Fractional Differential Equations, to Methods of Their Solution and Some of Their Applications. Academic Press, San Diego (1999)

[32] Qu, H., Liu, X., Lu, X., Ur Rahman, M., She, Z.: Neural network method for solving nonlinear fractional advection-diffusion equation with spatiotemporal variable-order. Chaos Solitons Fractals 156, 111856 (2022)

[33] QuH, LiuX, SheZ. Neural network method for fractional-order partial differential equations. Neurocomputing, 2020, 414: 225-237.

[34] QuH, SheZ, LiuX. Neural network method for solving fractional diffusion equations. Appl. Math. Comput., 2021, 3911125635

[35] RaissiM, PerdikarisP, KarniadakisGE. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys., 2018, 378: 686-707.

[36] RoopJ, JohnP. Computational aspects of FEM approximation of fractional advection dispersion equations on bounded domains in $\mathbb{R} ^2$. J. Comput. Appl. Math., 2006, 193(1): 243-268.

[37] SakamotoK, YamamotoM. Initial value/boundary value problems for fractional diffusion-wave equations and applications to some inverse problems. J. Math. Anal. Appl., 2011, 382(1): 426-447.

[38] ShenJ, TangT, WangLSpectral Methods: Algorithms, 2011, Germany. Analysis and Applications. Springer. .

[39] Sivalingam, S.M., Govindaraj, V.: Physics informed neural network based scheme and its error analysis for $\psi $-Caputo type fractional differential equations. Phys. Scripta 99(9), 096002 (2024)

[40] StynesM, O’RiordanE, GraciaJL. Error analysis of a finite difference method on graded meshes for a time-fractional diffusion equation. SIAM J. Numer. Anal., 2017, 55(2): 1057-1079.

[41] SugandhaA, TrilokM, ShiviA, KamleshT, PhalguniG. Applications of fractional calculus in computer vision: a survey. Neurocomputing, 2022, 489: 407-428.

[42] TianW, ZhouH, DengW. A class of second order difference approximations for solving space fractional diffusion equations. Math. Comput., 2015, 84: 1703-1727.

[43] Wang, Q., Ma, J., Yu, S., Tan, L.: Noise detection and image denoising based on fractional calculus. Chaos Solitons Fractals 131, 109463 (2020)

[44] YadavP, JahanS, NisarKS. Solving fractional Bagley-Torvik equation by fractional order Fibonacci wavelet arising in fluid mechanics. Ain Shams Eng. J., 2024, 151. 102299

[45] YangQ, LiuF, TurnerI. Numerical methods for fractional partial differential equations with Riesz space fractional derivatives. Appl. Math. Model., 2010, 34(1): 200-218.

[46] YeY, LiY, FanH, LiuX, ZhangH. SLeNN-ELM: a shifted Legendre neural network method for fractional delay differential equations based on extreme learning machine. Netw. Heterog. Media, 2023, 18(1): 494-512.

[47] ZayernouriM, KarniadakisGE. Fractional spectral collocation method. SIAM J. Sci. Comput., 2014, 36(1): A40-A62.

[48] ZengF, LiuF, LiC. A Crank-Nicolson ADI spectral method for a two-dimensional Riesz space fractional nonlinear reaction-diffusion equation. SIAM J. Numer. Anal., 2014, 52(6): 2599-2622.

[49] ZhaoT, LiC, LiD. Efficient spectral collocation method for fractional differential equation with Caputo-Hadamard derivative. Fract. Calc. Appl. Anal., 2023, 26: 2903-2927.

Funding

Universidade de Macau(MYRG2022-00262-FST)

RIGHTS & PERMISSIONS

Shanghai University

Just Accepted

This article has successfully passed peer review and final editorial review, and will soon enter typesetting, proofreading and other publishing processes. The currently displayed version is the accepted final manuscript. The officially published version will be updated with format, DOI and citation information upon launch. We recommend that you pay attention to subsequent journal notifications and preferentially cite the officially published version. Thank you for your support and cooperation.