Probabilistic versus deterministic machine learning for credit default prediction: A multidimensional benchmarking study under severe class imbalance

i

This article has been updated. A Correction to this article was published on .

i

The Original Article was published on

i

The Original Article was published on

i

This article has been updated

i

This article was retracted on

i

This article has been updated

Abstract

Deploying machine learning models for credit decisions under regulatory mandates for explainability and reliability requires evaluation frameworks beyond predictive accuracy. Yet, benchmarking studies in credit risk still focus narrowly on the area under the receiver operating characteristic curve (AUC), neglecting probability calibration, uncertainty decomposition, and interpretability. We address this gap by benchmarking ten models, spanning deterministic and probabilistic families, across four evaluation dimensions on a proprietary dataset of 50,819 customers from a Norwegian financial institution. Each customer is represented by 124 consecutive daily account balance observations and a binary 12-month default flag. Across 180 configurations combining five feature engineering modules with four resampling strategies, variational inference-based Bayesian models (Bayesian neural network, Bayesian logistic regression, convolutional-Bayesian neural network) exhibit posterior collapse on this imbalanced tabular dataset, producing near-uniform predicted probabilities and near-zero epistemic uncertainty, particularly under oversampling. Natural gradient boosting (NGB) emerges as the most effective probabilistic model, matching random forest and XGBoost in test AUC (0.80) while providing informative aleatoric uncertainty estimates and economically interpretable SHAP attributions. We further show that feature engineering and resampling effects are non-additive and model-specific, indicating both as first-order design variables in credit risk pipelines. These findings extend the literature to probabilistic evaluation dimensions and provide practitioners with guidance for model selection in regulatory-compliant credit decision systems under similar data conditions. Note that our performance figures represent exploratory upper-bound benchmarks rather than strictly unbiased out-of-sample estimates.

Keywords

probabilistic machine learning / credit default prediction / uncertainty quantification / class imbalance / SHAP interpretability

CHANGE HISTORY

A Correction to this paper has been published:
https://doi.org

The Retraction Note was updated to reflect reception of statements of disagreement from all authors.

This article has been retracted.Please see the Retraction Notice for more detail:
https://doi.org

References

[1] European Union, Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 on laying down harmonised rules on artificial intelligence and amending certain Union legislative acts (Artificial Intelligence Act), 2024. Available from: https://eur-lex.europa.eu/eli/reg/2024/1689/oj.

[2] B. Supervision, Basel committee on banking supervision, Principles for Sound Liquidity Risk Management and Supervision (September 2008) (2008). Available from: https://www.bis.org/publ/bcbs144.htm.

[3] N. Q. K. Le, Y. Y. Ou, Incorporating efficient radial basis function networks and significant amino acid pairs for predicting GTP binding sites in transport proteins, BMC Bioinf., 17(2016), 501. https://doi.org/10.1186/s12859-016-1369-y

[4] N. Q. K. Le, T. T. D. Nguyen, Y. Y. Ou, Identifying the molecular functions of electron transport proteins using radial basis function networks and biochemical properties, J. Mol. Graphics Modell., 73(2017), 166-178. https://doi.org/10.1016/j.jmgm.2017.01.003

[5] O. N. Nymo, M. Risstad, U. T. Sjøli, T. B. Sæther, Revisiting the spanning hypothesis with machine learning methods: A systematic literature review, J. Int. Money Financ., 167(2026), 103632. https://doi.org/10.1016/j.jimonfin.2026.103632

[6] S. Eggen, T. J. Espe, K. Grude, M. Risstad, R. Sandberg, Financial time series uncertainty: A review of probabilistic AI applications, J. Econ. Surv., (2025). https://doi.org/10.1111/joes.70018

[7] E. S. Gunnarsson, H. R. Isern, A. Kaloudis, M. Risstad, B. Vigdel, S. Westgaard, Prediction of realized volatility and implied volatility indices using AI and machine learning: A review, Int. Rev. Financ. Anal., 93(2024), 103221. https://doi.org/10.1016/j.irfa.2024.103221

[8] S. Lessmann, B. Baesens, H. V. Seow, L. C. Thomas, Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research, Eur. J. Oper. Res., 247(2015), 124-136. https://doi.org/10.1016/j.ejor.2015.05.030

[9] A. Engan, L. O. Hjelkrem, M. Risstad, A review of credit risk assessment in banking- from deterministic models to probabilistic AI, Finance Res. Open, 2(2026), 100125. https://doi.org/10.1016/j.finr.2026.100125

[10] Z. Ghahramani, Probabilistic machine learning and artificial intelligence, Nature, 521(2015), 452-459. https://doi.org/10.1038/nature14541

[11] Y. Gal, Z. Ghahramani, Dropout as a Bayesian approximation: Representing model uncertainty in deep learning, in International Conference on Machine Learning, PMLR, (2016), 1050-1059. Available from: https://proceedings.mlr.press/v48/gal16.html.

[12] Y. Wang, Y. Zhang, Y. Lu, X. Yu, A comparative assessment of credit risk model based on machine learning-A case study of bank loan data, Procedia Comput. Sci., 174(2020), 141-149. https://doi.org/10.1016/j.procs.2020.06.069

[13] Y. Ovadia, E. Fertig, J. Ren, Z. Nado, D. Sculley, S. Nowozin, et al., Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift, Adv. Neural Inf. Process. Syst., 32(2019). Available from: https://proceedings.neurips.cc/paper files/paper/2019/file/8558cb408c1d76621371888657d2eb1d-Paper.pdf.

[14] M. Abdar, F. Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, et al., A review of uncertainty quantification in deep learning: Techniques, applications and challenges, Inf. Fusion, 76(2021), 243-297. https://doi.org/10.1016/j.inffus.2021.05.008

[15] D. J. Hand, W. E. Henley, Statistical classification methods in consumer credit scoring: A review, J. R. Stat. Soc. Ser. A, 160(1997), 523-541. Available from: https://www.jstor.org/stable/2983268

[16] J. N. Crook, D. B. Edelman, L. C. Thomas, Recent developments in consumer credit risk assessment, Eur. J. Oper. Res., 183(2007), 1447-1465. https://doi.org/10.1016/j.ejor.2006.09.100

[17] I. R. Berrada, F. Barramou, O. B. Alami, Towards a machine learning-based model for corporate loan default prediction, Int. J. Adv. Comput. Sci. Appl., 15(2024). https://doi.org/10.14569/IJACSA.2024.0150357

[18] H. Kvamme, N. Sellereite, K. Aas, S. Sjursen, Predicting mortgage default using convolutional neural networks, Expert Syst. Appl., 102(2018), 207-217. https://doi.org/10.1016/j.eswa.2018.02.029

[19] L. O. Hjelkrem, P. E. De Lange, E. Nesset, An end-to-end deep learning approach to credit scoring using CNN+XGBoost on transaction data, J. Risk Model Valid., 16(2022), 1-25. https://doi.org/10.21314/JRMV.2022.018

[20] M. C. Aniceto, F. Barboza, H. Kimura, Machine learning predictivity applied to consumer creditworthiness, Future Bus. J., 6(2020), 37. https://doi.org/10.1186/s43093-020-00041-w

[21] L. Coenen, W. Verbeke, T. Guns, Machine learning methods for short-term probability of default: A comparison of classification, regression and ranking methods, J. Oper. Res. Soc., 73(2022), 191-206. https://doi.org/10.1080/01605682.2020.1865847

[22] P. E. De Lange, B. Melsom, C. B. Vennerød, S. Westgaard, Explainable AI for credit assessment in banks, J. Risk Financ. Manag., 15(2022), 556. https://doi.org/10.3390/jrfm15120556

[23] X. Zhu, Q. Chu, X. Song, P. Hu, L. Peng, Explainable prediction of loan default based on machine learning models, Data Sci. Manag., 6(2023), 123-133. https://doi.org/10.1016/j.dsm.2023.04.003

[24] N. G. B. Abrahamsen, E. Nylén-Forthun, M. Møller, P. E. de Lange, M. Risstad, Financial distress prediction in the nordics: Early warnings from machine learning models, J. Risk Financ. Manag., 17(2024), 432. https://www.mdpi.com/1911-8074/17/10/432

[25] A. Kendall, Y. Gal, What uncertainties do we need in Bayesian deep learning for computer vision, Adv. Neural Inf. Process. Syst., 30(2017), 5580-5590. Available from: https://proceedings.neurips.cc/paper files/paper/2017/file/2650d6089a6d640c5e85b2b88265dc2b-Paper.pdf.

[26] C. Blundell, J. Cornebise, K. Kavukcuoglu, D. Wierstra, Weight uncertainty in neural network, in International Conference on Machine Learning, PMLR, (2015), 1613-1622. Available from: http://proceedings.mlr.press/v37/blundell15.pdf.

[27] R. Mbuvha, I. Boulkaibet, T. Marwala, Automatic relevance determination Bayesian neural networks for credit card default modelling, preprint, arXiv:1906.06382, 2019. https://doi.org/10.48550/arXiv.1906.06382

[28] C. Louizos, M. Welling, Multiplicative normalizing flows for variational Bayesian neural networks, in International Conference on Machine Learning, PMLR, (2017), 2218-2227. Available from: http://proceedings.mlr.press/v70/louizos17a/louizos17a.pdf.

[29] A. Ashukha, A. Lyzhov, D. Molchanov, D. Vetrov, Pitfalls of in-domain uncertainty estimation and ensembling in deep learning, preprint, arXiv:2002.06470, 2021. https://doi.org/10.48550/arXiv.2002.06470

[30] J. Černevičienė, A. Kabašinskas, Explainable artificial intelligence (XAI) in finance: A systematic literature review, Artif. Intell. Rev., 57(2024), 216. https://doi.org/10.1007/s10462-024-10854-8

[31] C. Rudin, Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, Nat. Mach. Intell., 1(2019), 206-215. https://doi.org/10.1038/s42256-019-0048-x

[32] M. Nagl, M. Nagl, D. Rösch, Quantifying uncertainty of machine learning methods for loss given default, Front. Appl. Math. Stat., 8(2022), 1076083. https://doi.org/10.3389/fams.2022.1076083

[33] A. Niculescu-Mizil, R. Caruana, Predicting good probabilities with supervised learning, in Proceedings of the 22nd International Conference on Machine Learning, PMLR, (2005), 625-632. https://doi.org/10.1145/1102351.1102430

[34] C. Guo, G. Pleiss, Y. Sun, K. Q. Weinberger, On calibration of modern neural networks, in International Conference on Machine Learning, PMLR, (2017), 1321-1330. Available from: http://proceedings.mlr.press/v70/guo17a/guo17a.pdf.

[35] S. Lundberg, S. Lee, A unified approach to interpreting model predictions, in Advances in Neural Information Processing Systems 30, (2017), NIPS.

[36] Y. Chen, R. Calabrese, B. Martin-Barragan, Interpretable machine learning for imbalanced credit scoring datasets, Eur. J. Oper. Res., 312(2024), 357-372. https://doi.org/10.1016/j.ejor.2023.06.036

[37] T. Hastie, R. Tibshirani, J. H. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, 2009. Available from: https://link.springer.com/book/10.1007/978-0-387-84858-7.

[38] N. V. Chawla, K. W. Bowyer, L. O. Hall, W. P. Kegelmeyer, SMOTE: Synthetic minority over-sampling technique, J. Artif. Intell. Res., 16(2002), 321-357. https://doi.org/10.48550/arXiv.1106.1813

[39] H. He, E. A. Garcia, Learning from imbalanced data, IEEE Trans. Knowl. Data Eng., 21(2009), 1263-1284. https://doi.org/10.1109/TKDE.2008.239

[40] K. P. Murphy, Probabilistic Machine Learning: An Introduction, MIT Press, 2022. Available from: https://mitpress.mit.edu/9780262046824/probabilistic-machine-learning/.

[41] L. Breiman, Random forests, Mach. Learn., 45(2001), 5-32. https://doi.org/10.1023/A:1010933404324

[42] T. Chen, C. Guestrin, XGBoost: A scalable tree boosting system, in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, (2016), 785-794. https://doi.org/10.1145/2939672.2939785

[43] I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016. Available from: https://mitpress.mit.edu/9780262035613/deep-learning/.

[44] K. P. Murphy, Probabilistic Machine Learning: Advanced Topics, MIT Press, 2023. Available from: https://mitpress.mit.edu/9780262048439/probabilistic-machine-learning/.

[45] T. Duan, A. Anand, D. Y. Ding, K. K. Thai, S. Basu, A. Ng, et al., NGBoost: Natural gradient boosting for probabilistic prediction, in International Conference on Machine Learning, PMLR, (2020), 2690-2700. Available from: http://proceedings.mlr.press/v119/duan20a/duan20a.pdf.

[46] G. W. Brier, Verification of forecasts expressed in terms of probability, Mon. Weather Rev., 78(1950), 1-3. Available from: https://journals.ametsoc.org/view/journals/mwre/78/1/1520-0493_1950_078_0001_vofeit_2_0_co_2.xml.

[47] T. Saito, M. Rehmsmeier, The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets, PLoS One, 10(2015), 1-21. https://doi.org/10.1371/journal.pone.0118432

[48] B. Baesens, R. Setiono, C. Mues, J. Vanthienen, Using neural network rule extraction and decision tables for credit-risk evaluation, Manage. Sci., 49(2003), 312-329. Available from: https://www.jstor.org/stable/4133928.

[49] R. Shwartz-Ziv, A. Armon, Tabular data: Deep learning is not all you need, Inf. Fusion, 81(2022), 84-90. https://doi.org/10.1016/j.inffus.2021.11.011

[50] J. Van Amersfoort, L. Smith, Y. W. Teh, Y. Gal, Uncertainty estimation using a single deep deterministic neural network, preprint, arXiv:2003.02037, 2020. https://doi.org/10.48550/arXiv.2003.02037

[51] A. Alemi, B. Poole, I. Fischer, J. Dillon, R. A. Saurous, K. Murphy, Fixing a broken ELBO, preprint, arXiv:1711.00464, 2018. https://doi.org/10.48550/arXiv.1711.00464

[52] G. Incorvaia, D. Hond, H. Asgari, Uncertainty quantification for machine learning output assurance using anomaly-based dataset dissimilarity measures, in 2023 IEEE International Conference On Artificial Intelligence Testing (AITest), IEEE, (2023), 26-33. https://doi.org/10.1109/AITest58265.2023.00013

[53] E. I. Altman, Financial ratios, discriminant analysis and the prediction of corporate bankruptcy, J. Finance, 23(1968), 589-609. https://doi.org/10.2307/2978933

[54] L. Thomas, J. Crook, D. Edelman, Credit Scoring and Its Applications, SIAM, 2017. Available from: https://epubs.siam.org/doi/book/10.1137/1.9781611974560.

[55] M. Dusenberry, G. Jerfel, Y. Wen, Y. Ma, J. Snoek, K. Heller, et al., Efficient and scalable Bayesian neural nets with rank-1 factors, preprint, arXiv:2005.07186, 2020. https://doi.org/10.48550/arXiv.2005.07186

[56] P. Izmailov, S. Vikram, M. D. Hoffman, A. G. G. Wilson, What are Bayesian neural network posteriors really like, in International Conference on Machine Learning, PMLR, (2021), 4629-4640. Available from: http://proceedings.mlr.press/v139/izmailov21a/izmailov21a.pdf.

[57] A. G. Wilson, P. Izmailov, Bayesian deep learning and a probabilistic perspective of generalization, preprint, arXiv:2002.08791, 2022. https://doi.org/10.48550/arXiv.2002.08791

[58] B. Dai, D. Wipf, Diagnosing and enhancing VAE models, preprint, arXiv:1903.05789, 2019. https://doi.org/10.48550/arXiv.1903.05789

[59] J. He, D. Spokoyny, G. Neubig, T. Berg-Kirkpatrick, Lagging inference networks and posterior collapse in variational autoencoders, preprint, arXiv:1901.05534, 2019. https://doi.org/10.48550/arXiv.1901.05534

[60] E. R. DeLong, D. M. DeLong, D. L. Clarke-Pearson, Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach, Biometrics, 44(1988), 837-845. https://doi.org/10.2307/2531595

[61] R. J. Tibshirani, B. Efron, An introduction to the bootstrap, Monogr. Stat. Appl. Probab., 57(1993), 1-436. https://doi.org/10.1201/9780429246593