Learning Invariance Preserving Moment Closure Model for Boltzmann–BGK Equation

Abstract

As one of the main governing equations in kinetic theory, the Boltzmann equation is widely utilized in aerospace, microscopic flow, etc. Its high-resolution simulation is crucial in these related areas. However, due to the high dimensionality of the Boltzmann equation, high-resolution simulations are often difficult to achieve numerically. The moment method which was first proposed in Grad (Commun Pure Appl Math 2(4):331–407, 1949) is among the popular numerical methods to achieve efficient high-resolution simulations. We can derive the governing equations in the moment method by taking moments on both sides of the Boltzmann equation, which effectively reduces the dimensionality of the problem. However, one of the main challenges is that it leads to an unclosed moment system, and closure is needed to obtain a closed moment system. It is truly an art in designing closures for moment systems and has been a significant research field in kinetic theory. Other than the traditional human designs of closures, the machine learning-based approach has attracted much attention lately in Han et al. (Proc Natl Acad Sci USA 116(44):21983–21991, 2019) and Huang et al. (J Non-Equilib Thermodyn 46(4):355–370, 2021). In this work, we propose a machine learning-based method to derive a moment closure model for the Boltzmann–BGK equation. In particular, the closure relation is approximated by a carefully designed deep neural network that possesses desirable physical invariances, i.e., the Galilean invariance, reflecting invariance, and scaling invariance, inherited from the original Boltzmann–BGK equation and playing an important role in the correct simulation of the Boltzmann equation. Numerical simulations on the 1D–1D examples including the smooth and discontinuous initial condition problems, Sod shock tube problem, the shock structure problems, and the 1D–3D examples including the smooth and discontinuous problems demonstrate satisfactory numerical performances of the proposed invariance preserving neural closure method.

Keywords

Boltzmann equation / Moment closure / Machine learning / Neural networks / Invariance preserving

References

[1] Abramov R . The multidimensional maximum entropy moment problem: a review of numerical methods. Commun. Math. Sci.. 2010, 8 2 377-392

[2] Bhatnagar P, Gross E, Krook M. A model for collision processes in gases. I. Small amplitude processes in charged and neutral one-component systems. Phys. Rev.. 1954, 94 3 511-525

[3] Bird G. Molecular Gas Dynamics and the Direct Simulation of Gas Flows. 1994 Oxford: Clarendon Press

[4] Bois, L., Franck, E., Navoret, L., Vigon, V.: A neural network closure for the Euler–Poisson system based on kinetic simulations. arXiv:2011.06242 (2020)

[5] Cai Z, Li R. Numerical regularized moment method of arbitrary order for Boltzmann–BGK equation. SIAM J. Sci. Comput.. 2010, 32 5 2875-2907

[6] Cai Z, Fan Y, Li R. Globally hyperbolic regularization of Grad’s moment system. Commun. Pure Appl. Math.. 2014, 67 3 464-518

[7] Clevert, D., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (ELUs). In: Bengio, Y., LeCun, Y. (eds.) ICLR (2016)

[8] Fan, Y.: Development and application of moment method in gas kinetic theory (in Chinese). PhD thesis, Peking University (2016)

[9] Filbet F, Jin S. A class of asymptotic preserving schemes for kinetic equations and related problems with stiff sources. J. Comput. Phys.. 2010, 229 7625-7648

[10] Fox R. Higher-order quadrature-based moment methods for kinetic equations. J. Comput. Phys.. 2009, 228 7771-7791

[11] Gamba, I., Haack, J., Hu, J.: A fast conservative spectral solver for the nonlinear Boltzmann collision operator. In: Fan, J. (ed.) Proceedings of the 29th International Symposium on Rarefied Gas Dynamics, vol. 1628, pp. 1003–1008 (2014)

[12] Grad H. On the kinetic theory of rarefied gases. Commun. Pure Appl. Math.. 1949, 2 4 331-407

[13] Han J, Ma C, Ma Z, Weinan E. Uniformly accurate machine learning-based hydrodynamic models for kinetic equations. Proc. Natl. Acad. Sci.. 2019, 116 44 21983-21991

[14] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778. IEEE Computer Society (2016)

[15] Hsieh, J., Zhao, S., Eismann, S., Mirabella, L., Ermon, S.: Learning neural PDE solvers with convergence guarantees. In: ICLR (2019)

[16] Huang, J., Cheng, Y., Christlieb, A., Roberts, L.: Machine learning moment closure models for the radiative transfer equation I: directly learning a gradient based closure. arXiv:2105.05690 (2021)

[17] Huang, J., Cheng, Y., Christlieb, A., Roberts, L., Yong, W.: Machine learning moment closure models for the radiative transfer equation II: enforcing global hyperbolicity in gradient based closures. arXiv:2105.14410 (2021)

[18] Huang, J., Cheng, Y., Christlieb, A., Roberts, L.: Machine learning moment closure models for the radiative transfer equation III: enforcing hyperbolicity and physical characteristic speeds. arXiv:2109.00700 (2021)

[19] Huang, J., Ma, Z., Zhou, Y., Yong, W.: Learning thermodynamically stable and Galilean invariant partial differential equations for non-equilibrium flows. J. Non-Equilib. Thermodyn. (2021)

[20] Junk M. Domain of definition of Levermore’s five-moment system. J. Stat. Phys.. 1998, 93 5 1143-1167

[21] Koellermeier, J., Torrilhon, M.: Hyperbolic moment equations using quadrature-based projection methods. In: Proceedings of the 29th International Symposium on Rarefied Gas Dynamics. AIP Conf. Proc., vol. 1628, pp. 626–633 (2014)

[22] Koellermeier J, Schaerer R, Torrilhon M. A framework for hyperbolic approximation of kinetic equations using quadrature-based projection methods. Kinet. Relat. Mod.. 2014, 7 3 531-549

[23] Levermore C. Moment closure hierarchies for kinetic theories. J. Stat. Phys.. 1996, 83 5–6 1021-1065

[24] Levermore C. Moment closure hierarchies for the Boltzmann–Poisson equation. VLSI Design. 1998, 6 1–4 97-101

[25] Ling J, Jones R, Templeton J. Machine learning strategies for systems with invariance properties. J. Comput. Phys.. 2016, 318 22-35

[26] Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: ICLR (2019)

[27] Lou Q, Meng X, Karniadakis G. Physics-informed neural networks for solving forward and inverse flow problems via the Boltzmann–BGK formulation. J. Comput. Phys.. 2021, 447

[28] Mandli K, Ahmadia A, Berger M, Calhoun D, George D, Hadjimichael Y, Ketcheson D, Lemoine G, LeVeque R. Clawpack: building an open source ecosystem for solving hyperbolic PDEs. PeerJ Comput. Sci.. 2016, 2

[29] McDonald J, Groth C. Towards realizable hyperbolic moment closures for viscous heat-conducting gas flows based on a maximum-entropy distribution. Continuum Mech. Therm.. 2013, 25 5 573-603

[30] McDonald J, Torrilhon M. Affordable robust moment closures for CFD based on the maximum-entropy hierarchy. J. Comput. Phys.. 2013, 251 500-523

[31] McGraw R. Description of aerosol dynamics by the quadrature method of moments. Aerosol Sci. Technol.. 1997, 27 2 255-265

[32] Mouhot C, Pareschi L. Fast algorithms for computing the Boltzmann collision operator. Math. Comput.. 2006, 75 256 1833-1852

[33] Nair, V., Hinton, G.: Rectified linear units improve restricted Boltzmann machines. In: ICML (2010)

[34] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S.: Pytorch: an imperative style, high-performance deep learning library. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché Buc, F., Fox, E., Garnett, R. (eds.) Adv. Neural Inf. Process. Syst., vol. 32, pp. 8024–8035. Curran Associates, Inc. (2019)

[35] Patel R, Desjardins O, Fox R. Three-dimensional conditional hyperbolic quadrature method of moments. J. Comput. Phys. X. 2019, 1

[36] Porteous, W., Laiu, M., Hauck, C.: Data-driven, structure-preserving approximations to entropy-based moment closures for kinetic equations. arXiv:2106.08973 (2021)

[37] Qi, C., Su, H., Mo, K., Guibas, L.: Pointnet: deep learning on point sets for 3d classification and segmentation. In: CVPR, pp. 652–660 (2017)

[38] Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Med. Image Comput. Comput. Assist. Interv., pp. 234–241. Springer, Berlin (2015)

[39] Schaerer R, Bansal P, Torrilhon M. Efficient algorithms and implementations of entropy-based moment closures for rarefied gases. J. Comput. Phys.. 2017, 340 138-159

[40] Schotthöfer, S., Xiao, T., Frank, M., Hauck, C.: A structure-preserving surrogate model for the closure of the moment system of the Boltzmann equation using convex deep neural networks. arXiv:2106.09445 (2021)

[41] Struchtrup H. Derivation of 13 moment equations for rarefied gas flow to second order accuracy for arbitrary interaction potentials. SIAM Multiscale Model Sim.. 2005, 3 1 221-243

[42] Thuerey N, Weißenow K, Prantl L, Hu X. Deep learning methods for Reynolds-averaged Navier–Stokes simulations of airfoil flows. AIAA J.. 2020, 58 1 25-36

[43] Toro E. Riemann Solvers and Numerical Methods for Fluid Dynamics—A Practical Introduction. 2009 3 Berlin: Springer

[44] Um, K., Brand, R., Fei, Y., Holl, P., Thuerey, N.: Solver-in-the-loop: learning from differentiable physics to interact with iterative PDE-solvers. In: NeurIPS (2020)

[45] Xiao T, Frank M. Using neural networks to accelerate the solution of the Boltzmann equation. J. Comput. Phys.. 2021, 443

[46] Yuan C, Laurent F, Fox R. An extended quadrature method of moments for population balance equations. J. Aerosol Sci.. 2012, 51 1-23

[47] Zhang, J., Ma, W.: Data-driven discovery of governing equations for fluid dynamics based on molecular simulation. J. Fluid Mech. 892 (2020)

Funding

Natural Science Foundation of Beijing Municipality(180001)

National Natural Science Foundation of China(12090022)

National Natural Science Foundation of China(12171026)

NSAF Joint Fund(U1930402)

National Natural Science Foundation of China(12031013)