Stability Analysis and Performance Evaluation of Additive Mixed-Precision Runge-Kutta Methods

Abstract

Additive Runge-Kutta methods designed for preserving highly accurate solutions in mixed-precision computation were previously proposed and analyzed. These specially designed methods use reduced precision for the implicit computations and full precision for the explicit computations. In this work, we analyze the stability properties of these methods and their sensitivity to the low-precision rounding errors, and demonstrate their performance in terms of accuracy and efficiency. We develop codes in FORTRAN and Julia to solve nonlinear systems of ODEs and PDEs using the mixed-precision additive Runge-Kutta (MP-ARK) methods. The convergence, accuracy, and runtime of these methods are explored. We show that for a given level of accuracy, suitably chosen MP-ARK methods may provide significant reductions in runtime.

Keywords

Mixed precision / Runge-Kutta methods / Additive methods / Accuracy

References

[1.] Abdelfattah A, Anzt H, Boman E, Carson E, Cojean T, Dongarra J, Fox A, Gates M, Higham NJ, Li XS, Liu Y, Loe J, Luszczek P, Pranesh S, Rajamanickam S, Ribizel T, Smith B, Swirydowicz K, Thomas S, Tomov S, Tzai M, Yamazaki I, Yang UM. A survey of numerical linear algebra methods utilizing mixed-precision arithmetic. Int. J. High Performance Comput. Appl., 2021, 35(4): 344-369,

[2.] Abdelfattah, A., Tomov, S., Dongarra, J.J.: Fast batched matrix multiplication for small sizes using half-precision arithmetic on GPUs. In: 2019 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2019, Rio de Janeiro, Brazil, May 20–24, pp. 111–122. IEEE (2019)

[3.] Alexander R. Diagonally implicit Runge-Kutta methods for stiff ODEs. SIAM J. Numer. Anal., 1977, 14(6): 1006-1021,

[4.] Burnett, B., Gottlieb, S., Grant, Z.J., Heryudono, A.: Evaluation, performance, of mixed-precision Runge-Kutta methods. In: IEEE High Performance Extreme Computing Conference (HPEC). Waltham, MA, USA 2021, 1–6 (2021). https://doi.org/10.1109/HPEC49654.2021.9622803

[5.] Butcher JC. . Numerical Methods for Ordinary Differential Equations, 2016 Hoboken Wiley,

[6.] Butcher, J.C.: B-series: algebraic analysis of numerical methods. In: Springer Series in Computational Mathematics, 55. Springer, Cham, Switzerland (2021)

[7.] Croci M, Fasi M, Higham NJ, Mary T, Mikaitis M. Stochastic rounding: implementation, error analysis and applications. R. Soc. Open Sci., 2021, 9(3),

[8.] Croci M, Giles MB. Effects of round-to-nearest and stochastic rounding in the numerical solution of the heat equation in low precision. IMA J. Numer. Anal., 2023, 43(3): 1358-1390,

[9.] Croci M, Rosilho de Souza G. Mixed-precision explicit stabilized Runge-Kutta methods for single-and multi-scale differential equations. J. Comput. Phys., 2022, 464(1),

[10.] Grant ZJ. Perturbed Runge-Kutta methods for mixed precision applications. J. Sci. Comput., 2022, 92(1): 1-20,

[11.] Gupta, S., Agrawal, A., Gopalakrishnan, K., Narayanan, P.: Deep learning with limited numerical precision. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning, vol. 37, ICML’15, pp. 1737–1746. JMLR.org (2015)

[12.] Haidar, A., Tomov, S., Dongarra, J., Higham, N.J.: Harnessing GPU tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, SC’18, pp. 47:1–47:11, Piscataway, NJ, USA. IEEE Press (2018)

[13.] Higham, N.J.: Error analysis for standard and GMRES-based iterative refinement in two and three-precisions. MIMS EPrint 2019.19, Manchester Institute for Mathematical Sciences, The University of Manchester, November (2019)

[14.] Higham NJ, Mary T. A new approach to probabilistic rounding error analysis. SIAM J. Sci. Comput., 2019, 41(5): 2815-2835,

[15.] Higham NJ, Pranesh S. Simulating low precision floating-point arithmetic. SIAM J. Sci. Comput., 2019, 41(5): C585-C602,

[16.] Higham NJ, Pranesh S, Zounon M. Squeezing a matrix into half precision, with an application to solving linear systems. SIAM J. Sci. Comput., 2019, 41(4): A2536-A2551,

[17.] Higueras I, Ketcheson DI, Kocsis TA. Optimal monotonicity-preserving perturbations of a given Runge-Kutta method. J. Sci. Comput., 2018, 76(3): 1337-1369,

[18.] Kennedy CA, Carpenter MH. Additive Runge-Kutta schemes for convection-diffusion-reaction equations. Appl. Numer. Math., 2003, 44(1/2): 139-181,

[19.] Paxton EA, Chantry M, Klöwer M, Saffin L, Palmer T. Climate modeling in low precision: effects of both deterministic and stochastic rounding. J. Clim., 2022, 35(4): 1215-1229,

[20.] Petschow M, Quintana-Ort ES, Bientinesi P. Improved accuracy and parallelism for MRRR-based eigensolvers—a mixed precision approach. SIAM J. Sci. Comput., 2014, 36(2): C240-C263,

[21.] Richter C, Schops S, Clemens M. GPU-accelerated mixed-precision algebraic multigrid preconditioners for discrete elliptic field problems. IEEE Transact. Magnetics, 2014, 50(2): 83-90,

[22.] Sandu A, Gunther M. A generalized-structure approach to additive Runge-Kutta methods. SIAM J. Numer. Anal., 2015, 53(1): 17-42,

[23.] Zounon M, Higham NJ, Lucas C, Tisseur F. Performance impact of precision reduction in sparse linear systems solvers. J. Comput. Sci., 2022, 8

Funding

Air Force Office of Scientific Research(FA9550-23-1-0037); Michigan State University; US Department of Energy(DE-SC0023164)