Understanding deep learning is increasingly emergent as it penetrates more and more into industry and science. In recent years, a research line from Fourier analysis sheds light on this magical “black box” by showing a Frequency principle (F-Principle or spectral bias) of the training behavior of deep neural networks (DNNs)—DNNs often fit functions from low to high frequencies during the training. The F-Principle is first demonstrated by one-dimensional (1D) synthetic data followed by the verification in high-dimensional real datasets. A series of works subsequently enhance the validity of the F-Principle. This low-frequency implicit bias reveals the strength of neural networks in learning low-frequency functions as well as its deficiency in learning high-frequency functions. Such understanding inspires the design of DNN-based algorithms in practical problems, explains experimental phenomena emerging in various scenarios, and further advances the study of deep learning from the frequency perspective. Although incomplete, we provide an overview of the F-Principle and propose some open problems for future research.
Thanks to the singularity of the solution of linear subdiffusion problems, most time-stepping methods on uniform meshes can result in accuracy where denotes the time step. The present work aims to discover the reason why some type of Crank-Nicolson schemes (the averaging Crank-Nicolson (ACN) scheme) for the subdiffusion can only yield accuracy, which is much lower than the desired. The existing well-developed error analysis for the subdiffusion, which has been successfully applied to many time-stepping methods such as the fractional BDF-, requires singular points to be out of the path of contour integrals involved. The ACN scheme in this work is quite natural but fails to meet this requirement. By resorting to the residue theorem, some novel sharp error analysis is developed in this study, upon which correction methods are further designed to obtain the optimal accuracy. All results are verified by numerical tests.
In this paper, we concentrate on high-order boundary treatments for finite volume methods solving hyperbolic conservation laws. The complex geometric physical domain is covered by a Cartesian mesh, resulting in the boundary intersecting the grids in various fashions. We propose two approaches to evaluate the cell averages on the ghost cells near the boundary. Both of them start from the inverse Lax-Wendroff (ILW) procedure, in which the normal spatial derivatives at inflow boundaries can be obtained by repeatedly using the governing equations and boundary conditions. After that, we can get an accurate evaluation of the ghost cell average by a Taylor expansion joined with high-order extrapolation, or by a Hermite extrapolation coupling with the cell averages on some “artificial” inner cells. The stability analysis is provided for both schemes, indicating that they can avoid the so-called “small-cell” problem. Moreover, the second method is more efficient under the premise of accuracy and stability. We perform numerical experiments on a collection of examples with the physical boundary not aligned with the grids and with various boundary conditions, indicating the high-order accuracy and efficiency of the proposed schemes.
A hybrid iterative method is proposed for numerically solving the elliptic variational inequality (EVI) of the second kind, through combining the regularized semi-smooth Newton method and the Int-Deep method. The convergence rate analysis and numerical examples on contact problems show this algorithm converges rapidly and is efficient for solving EVIs.
We introduce an adaptive sampling method for the Deep Ritz method aimed at solving partial differential equations (PDEs). Two deep neural networks are used. One network is employed to approximate the solution of PDEs, while the other one is a deep generative model used to generate new collocation points to refine the training set. The adaptive sampling procedure consists of two main steps. The first step is solving the PDEs using the Deep Ritz method by minimizing an associated variational loss discretized by the collocation points in the training set. The second step involves generating a new training set, which is then used in subsequent computations to further improve the accuracy of the current approximate solution. We treat the integrand in the variational loss as an unnormalized probability density function (PDF) and approximate it using a deep generative model called bounded KRnet. The new samples and their associated PDF values are obtained from the bounded KRnet. With these new samples and their associated PDF values, the variational loss can be approximated more accurately by importance sampling. Compared to the original Deep Ritz method, the proposed adaptive method improves the accuracy, especially for problems characterized by low regularity and high dimensionality. We demonstrate the effectiveness of our new method through a series of numerical experiments.
For solving large-scale nonlinear equations, a nonlinear fast deterministic block Kaczmarz method based on a greedy strategy is proposed. The method is adaptive and does not need to compute the pseudoinverses of submatrices. It is proved that the method will converge linearly to the nearest solution to the initial point under mild conditions. Numerical experiments are performed to illustrate that the method is efficient at least for the tested problems.
A class of geometric asynchronous parallel algorithms for solving large-scale discrete PDE eigenvalues has been studied by the author (Sun in Sci China Math 41(8): 701–725, 2011; Sun in Math Numer Sin 34(1): 1–24, 2012; Sun in J Numer Methods Comput Appl 42(2): 104–125, 2021; Sun in Math Numer Sin 44(4): 433–465, 2022; Sun in Sci China Math 53(6): 859–894, 2023; Sun et al. in Chin Ann Math Ser B 44(5): 735–752, 2023). Different from traditional preconditioning algorithm with the discrete matrix directly, our geometric pre-processing algorithm (GPA) algorithm is based on so-called intrinsic geometric invariance, i.e., commutativity between the stiff matrix A and the grid mesh matrix . Thus, the large-scale system solvers can be replaced with a much smaller block-solver as a pretreatment. In this paper, we study a sole PDE and assume G satisfies a periodic condition . Four special cases have been studied in this paper: two-point ODE eigen-problem, Laplace eigen-problems over L-shaped region, square ring, and 3D hexahedron. Two conclusions that “the parallelism of geometric mesh pre-transformation is mainly proportional to the number of faces of polyhedron” and “commutativity of grid mesh matrix and mass matrix is the essential condition for the GPA algorithm” have been obtained.
Based on the equivalence between the Sylvester tensor equation and the linear equation obtained by discretization of partial differential equations (PDEs), an overlapping Schwarz alternative method based on the tensor format and an overlapping parallel Schwarz method based on the tensor format for solving high-dimensional PDEs are proposed. The complexity of the new algorithms is discussed. Finally, the feasibility and effectiveness of the new methods are verified by some numerical examples.
An upwind weak Galerkin finite element scheme was devised and analyzed in this article for convection-dominated Oseen equations. The numerical algorithm was based on the weak Galerkin method enhanced by upwind stabilization. The resulting finite element scheme uses equal-order, say k, polynomial spaces on each element for the velocity and the pressure unknowns. With finite elements of order , the numerical solutions are proved to converge at the rate of in an energy-like norm for convection-dominated Oseen equations. Numerical results are presented to demonstrate the accuracy and effectiveness of the upwind weak Galerkin scheme.
In this paper, we investigate the instability of growing tumors by employing both analytical and numerical techniques to validate previous results and extend the analytical findings presented in a prior study by Feng et al. (Z Angew Math Phys 74:107, 2023). Building upon the insights derived from the analytical reconstruction of key results in the aforementioned work in one dimension and two dimensions, we extend our analysis to three dimensions. Specifically, we focus on the determination of boundary instability using perturbation and asymptotic analysis along with spherical harmonics. Additionally, we have validated our analytical results in a two-dimensional (2D) framework by implementing the Alternating Direction Implicit (ADI) method. Our primary focus has been on ensuring that the numerical simulation of the propagation speed aligns accurately with the analytical findings. Furthermore, we have matched the simulated boundary stability with the analytical predictions derived from the evolution function, which will be defined in subsequent sections of our paper. This alignment is essential for accurately determining the stability or instability of tumor boundaries.
Many three-dimensional physical applications can be better analyzed and solved using the cylindrical coordinates. In this paper, the immersed interface method (IIM) tailored for Navier-Stokes equations involving interfaces under the cylindrical coordinates has been developed. Note that, while the IIM has been developed for Stokes equations in the cylindrical coordinates assuming the axis-symmetry in the literature, there is a gap in dealing with Navier-Stokes equations, where the non-linear term includes an additional component involving the coordinate , even if the geometry and force term are axis-symmetric. Solving the Navier-Stokes equations in cylindrical coordinates becomes challenging when dealing with interfaces that feature a discontinuous pressure and a non-smooth velocity, in addition to the pole singularity at . In the newly developed algorithm, we have derived the jump conditions under the cylindrical coordinates. The numerical algorithm is based on a finite difference discretization on a uniform and staggered grid in the cylindrical coordinates. The finite difference scheme is standard away from the interface but is modified at grid points near and on the interface. As expected, the method is shown to be second-order accurate for the velocity. The developed new IIM is applied to the solution of some related fluid dynamic problems with interfaces.
In this paper, we propose mixed finite element methods for the Reissner-Mindlin Plate Problem by introducing the bending moment as an independent variable. We apply the finite element approximations of the stress field and the displacement field constructed for the elasticity problem by Hu (J Comp Math 33: 283–296, 2015), Hu and Zhang (arXiv:1406.7457, 2014) to solve the bending moment and the rotation for the Reissner-Mindlin Plate Problem. We propose two triples of finite element spaces to approximate the bending moment, the rotation, and the displacement. The feature of these methods is that they need neither reduction terms nor penalty terms. Then, we prove the well-posedness of the discrete problem and obtain the optimal estimates independent of the plate thickness. Finally, we present some numerical examples to demonstrate the theoretical results.
This paper investigates the scattering of biharmonic waves by a one-dimensional periodic array of cavities embedded in an infinite elastic thin plate. The transparent boundary conditions (TBCs) are introduced to formulate the problem from an unbounded domain to a bounded one. The well-posedness of the associated variational problem is demonstrated utilizing the Fredholm alternative theorem. The perfectly matched layer (PML) method is employed to reformulate the original scattering problem, transforming it from an unbounded domain to a bounded one. The TBCs for the PML problem are deduced, and the well-posedness of its variational problem is established. Moreover, the exponential convergence is achieved between the solution of the PML problem and that of the original scattering problem.