Design and Comparison of Parallel Dynamic Matérn Kernel-Based Regression Models and Machine Learning Approaches: Application to Bias Correction in Numerical Weather Prediction

Abstract

Mathematical modelling is fundamental to understanding real-world phenomena. Despite the inherent complexity in designing such models, numerical approaches and, more recently, machine learning techniques, have emerged as powerful tools in this area. This work proposes integrating the finite element method (FEM) into forecasting and introduces parallel techniques for regression problems, with a specific focus on the use of Matérn kernels on local mesh support. This approach generalises the modelling based on radial basis function kernels and offers more flexibility to control the smoothness of the modelled functions. An exhaustive study explores the impact of diverse norms and Matérn kernel variations on the performance of models, and aims to improve the computational efficiency of the model fitting and prediction processes. Furthermore, a heuristic framework is introduced to derive optimal complexity parameters for each Matérn-based FEM kernel. The proposed parallel approaches use dynamic strategies, which significantly reduce the computational time of the algorithms compared to other methods and parallel computing techniques presented in recent years. The proposed methodology is assessed in the context of bias corrections for temperature forecasts made by the Local Data Assimilation and Prediction System (LDAPS) model. A comprehensive comparative analysis which includes machine learning algorithms provides significant insights into the training process, norm selection, and kernel choice, and shows that Matérn-based methods emerge as a choice to be considered for regression problems.

Keywords

Numerical modelling / Parallel programming / Matérn kernels / Machine learning / 65Y05 / 68W10 / 68T07

References

[1] Abramowitz, M., Stegun, I.A.: Handbook of Mathematical Functions: with Formulas, Graphs, and Mathematical Tables. Applied Mathematics Series. Dover Publications, Mineola (1972)

[2] Aggarwal CCNeural Networks and Deep Learning, 2018New YorkSpringer

[3] Apaydın, M., Yumuş, M., Değirmenci, A., Karal, Ö.: Evaluation of air temperature with machine learning regression methods using Seoul City meteorological data. Pamukkale Univ. J. Eng. Sci. 28(5), 737–747 (2022)

[4] Apsemidis A, Psarakis S, Moguerza JM. A review of machine learning kernel methods in statistical process monitoring. Comput. Ind. Eng., 2020, 142 Article ID: 106376

[5] Aragonés L, Pagán JI, López I, Navarro-González FJ, Villacampa Y. Galerkin’s formulation of the finite elements method to obtain the depth of closure. Sci. Total Environ., 2019, 660: 1256-1263

[6] Awad M, Khanna REfficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers, 2015BerkeleyApress Media

[7] Azadkia, M.: Optimal choice of k\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$k$$\end{document} for k\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$k$$\end{document}-nearest neighbor regression (2020). https://doi.org/10.48550/ARXIV.1909.05495

[8] Balázsová M, Feistauer M, Hadrava M, Kosík A. On the stability of the space-time discontinuous Galerkin method for the numerical solution of nonstationary nonlinear convection-diffusion problems. J. Numer. Math., 2015, 23(3): 211-233

[9] Baran, S., Horányi, A., Nemoda, D.: Comparison of BMA and EMOS statistical calibration methods for temperature and wind speed ensemble weather prediction (2013). https://doi.org/10.48550/arXiv.1312.3763

[10] Baran S, Lakatos M. Clustering-based spatial interpolation of parametric postprocessing models. Weather Forecast., 2024, 39(11): 1591-1604

[11] Barbour, S.L., Krahn, J.: Numerical modelling—prediction or process? In: Geotechnical News, pp. 44–52 (2004)

[12] Beisiegel N, Vater S, Behrens J, Dias F. An adaptive discontinuous Galerkin method for the simulation of hurricane storm surge. Ocean Dyn., 2020, 70: 641-666

[13] Berrar D Ranganathan S, Gribskov M, Nakai K, Schönbach C. Cross-validation. Encyclopedia of Bioinformatics and Computational Biology, 2019CambridgeAcademic Press542-545

[14] Bi K, Xie L, Zhang H, Chen X, Gu X, Tian Q. Accurate medium-range global weather forecasting with 3D neural networks. Nature, 2023, 619: 533-538

[15] Biau G, Scornet E. A random forest guided tour. TEST, 2016, 25: 197-227

[16] Biggs D, De Ville B, Suen E. A method of choosing multiway partitions for classification and decision trees. J. Appl. Stat., 1991, 18(1): 49-62

[17] Breiman L. Random forests. Mach. Learn., 2001, 45: 5-32

[18] Breiman L, Friedman JH, Olshen RA, Stone CJClassification and Regression Trees, 19841New YorkChapman and Hall/CRC

[19] Bühlmann PGentle J, Härdle W, Mori Y. Bagging, boosting and ensemble methods. Handbook of Computational Statistics. Springer Handbooks of Computational Statistics, 2012BerlinSpringer985-1022

[20] Chai T, Draxler RR. Root mean square error (RMSE) or mean absolute error (MAE)? Arguments against avoiding RMSE in the literature. Geosci. Model Dev., 2014, 7(3): 1247-1250

[21] Chan J. Weight-adjusted discontinuous Galerkin methods: matrix-valued weights and elastic wave propagation in heterogeneous media. Int. J. Numer. Methods Eng., 2018, 113(12): 1779-1809

[22] Chan J, Hewett RJ, Warburton T. Weight-adjusted discontinuous Galerkin methods: wave propagation in heterogeneous media. SIAM J. Sci. Comput., 2017, 36(6): 2935-2961

[23] Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794 (2016). https://doi.org/10.1145/2939672.2939785

[24] Cheng WYY, Steenburgh WJ. Strengths and weaknesses of MOS, running-mean bias removal, and Kalman filter techniques for improving model forecasts over the Western United States. Weather Forecast., 2007, 22(6): 1304-1318

[25] Cho D, Yoo C, Im J, Cha D-H. Comparative assessment of various machine learning-based bias correction methods for numerical weather prediction model forecasts of extreme air temperatures in urban areas. Earth Space Sci., 2020, 7(4): 2019–e2019EA000740

[26] Chorley MJ, Walker DW. Performance analysis of a hybrid MPI/OpenMP application on multi-core clusters. J. Comput. Sci., 2010, 1(3): 168-174

[27] Cifuentes J, Marulanda G, Bello A, Reneses J. Air temperature forecasting using machine learning techniques: a review. Energies, 2020, 13(16): 4215

[28] Constantin A, Johnson RS. On the modelling of large-scale atmospheric flow. J. Differ. Equ., 2021, 285: 751-798

[29] Dehghan M, Narimani N. An element-free Galerkin meshless method for simulating the behavior of cancer cell invasion of surrounding tissue. Appl. Math. Model., 2018, 59: 500-513

[30] Durai VR, Bhradwaj R. Evaluation of statistical bias correction methods for numerical weather prediction model forecasts of maximum and minimum temperatures. Nat. Hazards, 2014, 73: 1229-1254

[31] Freund Y, Schapire RE. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci., 1997, 55(1): 119-139

[32] Friedman JH. Greedy function approximation: a gradient boosting machine. Ann. Stat., 2001, 29(5): 1189-1232

[33] Friedman JH. Stochastic gradient boosting. Comput. Stat. Data Anal., 2002, 38(4): 367-378

[34] Galanis G, Anadranistakis M. A one-dimensional Kalman filter for the correction of near surface temperature forecasts. Meteorol. Appl., 2002, 9(4): 437-441

[35] Gallagher RHFinite Element Analysis: Fundamentals, 1975Englewood CliffsPrentice-Hall

[36] Gillijns, S., Mendoza, O.B., Chandrasekar, J., De Moor, B.L.R., Bernstein, D.S., Ridley, A.: What is the ensemble Kalman filter and how well does it work? In: 2006 American Control Conference, pp. 6 (2006). https://doi.org/10.1109/ACC.2006.1657419

[37] Gneiting T, Raftery AE, Westveld AH, Goldman T. Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Mon. Weather Rev., 2005, 133(5): 1098-1118

[38] Gu M, Wang X, Berger JO. Robust Gaussian stochastic process emulation. Ann. Stat., 2018, 46(6A): 3038-3066

[39] Hatami MWeighted Residual Methods: Principles, Modifications and Applications, 2017CambridgeAcademic Press

[40] Hsieh WWMachine Learning Methods in the Environmental Sciences: Neural Networks and Kernels, 2009VancouverCambridge University Press

[41] Jouan G, Cuzol A, Monbet V, Monnier G. Gaussian mixture models for clustering and calibration of ensemble weather forecasts. Discrete Contin. Dyn. Syst. S, 2023, 16(2): 309-328

[42] Kalman RE. A new approach to linear filtering and prediction problems. J. Basic Eng., 1960, 82(1): 35-45

[43] Karvonen, T.: Asymptotic bounds for smoothness parameter estimates in Gaussian process interpolation. SIAM-ASA J. Uncertain. Quantif. 11(4), 1225–1257 (2023). https://doi.org/10.1137/22M149288X

[44] Kass GV. An exploratory technique for investigating large quantities of categorical data. J. R. Stat. Soc. C Appl. Stat., 1980, 29(2): 119-127

[45] Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., Liu, T.-Y.: LightGBM: a highly efficient gradient boosting decision tree. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. NIPS’17, pp. 3149–3157. Curran Associates Inc., Red Hook (2017)

[46] Khodarahmi M, Maihami V. A review on Kalman filter models. Arch. Comput. Methods Eng., 2023, 30: 727-747

[47] Kim HM, Han SS. Seoul. Cities, 2012, 29(2): 142-154

[48] Kim Y-H, Baik J-J. Spatial and temporal structure of the urban heat island in Seoul. J. Appl. Meteorol. Climatol., 2005, 44(5): 591-605

[49] Klein, W.H., Glahn, H.R.: Forecasting local weather by means of model output statistics. Bull. Am. Meteorol. Soc. 55(10), 1217–1227 (1974)

[50] Kochkov D, Yuval J, Langmore I, Norgaard P, Smith J, Mooers G, Klöwer M, Lottes J, Rasp S, Düben P, Hatfield S, Battaglia P, Sanchez-Gonzalez A, Willson M, Brenner MP, Hoyer S. Neural general circulation models for weather and climate. Nature, 2024, 632: 1060-1066

[51] Konya A, Nematzadeh P. Recent applications of AI to environmental disciplines: a review. Sci. Total Environ., 2024, 906 Article ID: 167705

[52] Lam R, Sanchez-Gonzalez A, Willson M, Wirnsberger P, Fortunato M, Alet F, Ravuri S, Ewalds T, Eaton-Rosen Z, Hu W, Merose A, Hoyer S, Holland G, Vinyals O, Stott J, Pritzel A, Mohamed S, Battaglia P. Learning skillful medium-range global weather forecasting. Science, 2023, 382(6677): 1416-1421

[53] Le T-T, Le MV. Development of user-friendly kernel-based Gaussian process regression model for prediction of load-bearing capacity of square concrete-filled steel tubular members. Mater. Struct., 2021, 54: 59

[54] Lim Y-F, Ng CK, Vaitesswar US, Hippalgaonkar K. Extrapolative Bayesian optimization with Gaussian process and neural network ensemble surrogate models. Adv. Intell. Syst., 2021, 3(11): 2100101

[55] Liu X, Lu D, Zhang A, Liu Q, Jiang G. Data-driven machine learning in environmental pollution: gains and problems. Environ. Sci. Technol., 2022, 56(4): 2124-2133

[56] López I, Aragonés L, Villacampa Y, Navarro-González FJ. Gravel beaches nourishment: modelling the equilibrium beach profile. Sci. Total Environ., 2018, 619–620: 772-783

[57] Ma X, Liu H, Dong Q, Chen Q, Cai N. Statistical post-processing of multiple meteorological elements using the multimodel integration embedded method. Atmos. Res., 2024, 301 Article ID: 107269

[58] Makarova, E., Portabella, M., Stoffelen, A., Li, G., Lin, W.: Correction of NWP ocean surface wind biases with machine learning. In: IGARSS 2024—2024 IEEE International Geoscience and Remote Sensing Symposium, Athens, Greece, pp. 7560–7563 (2024). https://doi.org/10.1109/IGARSS53475.2024.10642503

[59] Marras S, Kelly JF, Moragues M, Müller A, Kopera MA, Vázquez M, Giraldo FX, Houzeaux G, Jorba O. A review of element-based Galerkin methods for numerical weather prediction: finite elements, spectral elements, and discontinuous Galerkin. Arch. Comput. Methods Eng., 2016, 23: 673-722

[60] Matérn BSpatial Variation. Lecture Notes in Statistics, 1986New YorkSpringer

[61] Mathault, J., Landari, H., Tessier, F., Fortier, P., Miled, A.: Biological modeling challenges in a multiphysics approach. In: 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS), pp. 88–91 (2017). https://doi.org/10.1109/MWSCAS.2017.8052867

[62] McDonald GC. Ridge regression. WIREs Comput. Stat., 2009, 1(1): 93-100

[63] Mesut B, Başkor A, Buket Aksu NPhilip A, Shahiwala A, Rashid M, Faiyazuddin M. Chapter 3—Role of artificial intelligence in quality profiling and optimization of drug products. A Handbook of Artificial Intelligence in Drug Delivery, 2023CambridgeAcademic Press35-54

[64] Michaelsen, J.: Cross-validation in statistical climate forecast models. J. Appl. Meteorol. Climatol. 26(11), 1589–1600 (1987)

[65] Migallón V, Navarro-González FJ, Penadés H, Penadés J, Villacampa Y. A parallel methodology using radial basis functions versus machine learning approaches applied to environmental modelling. J. Comput. Sci., 2022, 63 Article ID: 101817

[66] Migallón V, Navarro-González FJ, Penadés H, Penadés J, Villacampa Y. Designing a parallel nonlinear model for predicting nitrogen oxide emissions. AIP Conf. Proc., 2024, 3094(1): Article ID: 500001

[67] Migallón V, Navarro-González FJ, Penadés J, Villacampa Y. Parallel approach of a Galerkin-based methodology for predicting the compressive strength of the lightweight aggregate concrete. Constr. Build. Mater., 2019, 219: 56-68

[68] Min S-K, Simonis D, Hense A. Probabilistic climate change predictions applying Bayesian model averaging. Philos. Trans. R. Soc. A, 2007, 365: 2103-2116

[69] Navarro-González FJ, Villacampa Y. A new methodology for complex systems using n\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$n$$\end{document}-dimensional finite elements. Adv. Eng. Softw., 2012, 48: 52-57

[70] Navarro-González FJ, Villacampa Y. Generation of representation models for complex systems using Lagrangian functions. Adv. Eng. Softw., 2013, 64: 33-37

[71] Navarro-González FJ, Villacampa Y. A finite element numerical algorithm for modelling and data fitting in complex systems. Int. J. Comput. Methods Exp. Meas., 2016, 4(2): 100-113

[72] Navarro-González, F.J., Villacampa, Y., Cortés-Molina, M., Ivorra, S.: Numerical non-linear modelling algorithm using radial kernels on local mesh support. Mathematics 8(9), 1600 (2020). https://doi.org/10.3390/math8091600

[73] Nwankpa, C., Ijomah, W., Gachagan, A., Marshall, S.: Activation functions: comparison of trends in practice and research for deep learning (2018). https://arxiv.org/pdf/1811.03378

[74] Olson R, Fan Y, Evans JP. A simple method for Bayesian model averaging of regional climate model projections: application to southeast Australian temperatures. Geophys. Res. Lett., 2016, 43(14): 7661-7669

[75] OpenMP: OpenMP official site. Accessed in May, 2024 (2012). http://www.openmp.org

[76] Ouyang Z-L, Liu S-Y, Zou Z-J. Nonparametric modeling of ship maneuvering motion in waves based on Gaussian process regression. Ocean Eng., 2022, 264 Article ID: 112100

[77] Pal A, Agarwala A, Raha S, Bhattacharya B. Performance metrics in a hybrid MPI-OpenMP based molecular dynamics simulation with short-range interactions. J. Parallel Distrib. Comput., 2014, 74(3): 2203-2214

[78] Palazón, A., López, I., Aragonés, L., Villacampa, Y., Navarro-González, F.J.: Modelling of Escherichia coli concentrations in bathing water at microtidal coasts. Sci. Total Environ. 593/594, 173–181 (2017). https://doi.org/10.1016/j.scitotenv.2017.03.161

[79] Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E. Scikit-learn: machine learning in Python. J. Mach. Learn. Res., 2011, 12: 2825-2830

[80] Price I, Sanchez-Gonzalez A, Alet F, Andersson TR, El-Kadi A, Masters D, Ewalds T, Stott J, Mohamed S, Battaglia P, Lam R, Willson M. Probabilistic weather forecasting with machine learning. Nature, 2025, 637: 84-90

[81] Qin C, Tian J, Yang X, Liu K, Yan G, Feng J, Lv Y, Xu M. Galerkin-based meshless methods for photon transport in the biological tissue. Opt. Express, 2008, 16(25): 20317-20333

[82] Rabenseifner, R., Hager, G., Jost, G.: Hybrid MPI/OpenMP parallel programming on clusters of multi-core SMP nodes. In: 2009 17th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, pp. 427–436 (2009). https://doi.org/10.1109/PDP.2009.43

[83] Raftery AE, Gneiting T, Balabdaoui F, Polakowski M. Using Bayesian model averaging to calibrate forecast ensembles. Mon. Weather Rev., 2005, 133(5): 1155-1174

[84] Raissi M, Perdikaris P, Karniadakis GE. Machine learning of linear differential equations using Gaussian processes. J. Comput. Phys., 2017, 348: 683-693

[85] Rashmi, K.V., Gilad-Bachrach, R.: DART: dropouts meet multiple additive regression trees (2015). https://doi.org/10.48550/arXiv.1505.01866

[86] Ruocco G , Caccavale P, De Bonis MVKundu SC, Reis RL. Chapter 18—A predictive oncology framework-modeling tumor proliferation using a FEM platform. Biomaterials for 3D Tumor Modeling. Materials Today, 2020AmsterdamElsevier427-450

[87] Saminathan, S., Mitra, S.: Probabilistic post-processing of short to medium range temperature forecasts: implications for heatwave prediction in India. Environ. Monit. Assess. 196, 284 (2024). https://doi.org/10.1007/s10661-024-12418-3

[88] Shamshiri R, Ismail WIW. Implementation of Galerkin’s method and modal analysis for unforced vibration response of a tractor suspension model. Res. J. Appl. Sci. Eng. Technol., 2014, 7(1): 49-55

[89] Shi, J., Shirali, A., Jin, B., Zhou, S., Hu, W., Rangaraj, R., Wang, S., Han, J., Wang, Z., Lall, U., Wu, Y., Bobadilla, L., Narasimhan, G.: Deep learning and foundation models for weather prediction: a survey (2025). arXiv:2501.06907

[90] Smola AJ, Schölkopf B. A tutorial on support vector regression. Stat. Comput., 2004, 14(3): 199-222

[91] Snir, M., Otto, S., Huss-Lederman, S., Walker, D., Dongarra, J.: MPI: the Complete Reference, 2nd edn. MIT Press, Cambridge (1998)

[92] Stein MLInterpolation of Spatial Data: Some Theory for Kriging. Springer Series in Statistics, 1999New YorkSpringer

[93] Sutton CDRao CR, Wegman EJ, Solka JL. 11—Classification and regression trees, bagging, and boosting. Data Mining and Data Visualization. Handbook of Statistics, 2005AmsterdamElsevier303-329 24

[94] Tenza-Abril AJ, Villacampa Y, Solak AM, Baeza-Brotons F. Prediction and sensitivity analysis of compressive strength in segregated lightweight concrete based on artificial neural network using ultrasonic pulse velocity. Constr. Build. Mater., 2018, 189: 1173-1183

[95] The UCI Machine Learning Repository: Bias correction of numerical prediction model temperature forecast (2020). https://doi.org/10.24432/C59K76

[96] Thorarinsdottir TL, Johnson MS. Probabilistic wind gust forecasting using nonhomogeneous Gaussian regression. Mon. Weather Rev., 2012, 140(3): 889-897

[97] Tibshirani R. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B Methodol., 2018, 58(1): 267-288

[98] Wan, E.A., Van Der Merwe, R.: The unscented Kalman filter for nonlinear estimation. In: Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373), pp. 153–158 (2000). https://doi.org/10.1109/ASSPCC.2000.882463

[99] Willmott CJ, Matsuura K. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim. Res., 2005, 30: 79-82

[100] Yao N, Ye J, Wang S, Yang S, Lu Y, Zhang H, Yang X. Bias correction of the hourly satellite precipitation product using machine learning methods enhanced with high-resolution WRF meteorological simulations. Atmos. Res., 2024, 310 Article ID: 107637

[101] Zhang F, O’Donnell LJMechelli A, Vieira S. Chapter 7—Support vector regression. Machine Learning, 2020CambridgeAcademic Press123-140

[102] Zhang, G.: Machine learning for the bias correction of LDAPS air temperature prediction model. In: 2021 6th International Conference on Machine Learning Technologies. ICMLT ’21, pp. 1–6. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3468891.3468892

[103] Zhang, H., Chen, J., Wang, Y., Han, J., Xu, Y.: Improving 2 m temperature forecasts of numerical weather prediction through a machine learning-based Bayesian model. Meteorol. Atmos. Phys. 137, 9 (2025). https://doi.org/10.1007/s00703-024-01056-6

[104] Zhong S, Zhang K, Bagheri M, Burken JG, Gu A, Li B, Ma X, Marrone BL, Ren ZJ, Schrier J, Shi W, Tan H, Wang T, Wang X, Wong BM, Xiao X, Yu X, Zhu J-J, Zhang H. Machine learning: new ideas and tools in environmental science and engineering. Environ. Sci. Technol., 2021, 55(19): 12741-12754

[105] Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B Methodol. 67(2), 301–320 (2005). https://doi.org/10.1111/j.1467-9868.2005.00503.x

Funding

Ministerio de Ciencia e Innovación(PID2021-123627OB-C55)

ValgrAI - Valencian Graduate School and Research Network for Artificial Intelligence and Generalitat Valenciana

Universidad de Alicante

RIGHTS & PERMISSIONS

The Author(s)