Combining classifiers for credit risk prediction

Abstract

Credit risk prediction models seek to predict quality factors such as whether an individual will default (bad applicant) on a loan or not (good applicant). This can be treated as a kind of machine learning (ML) problem. Recently, the use of ML algorithms has proven to be of great practical value in solving a variety of risk problems including credit risk prediction. One of the most active areas of recent research in ML has been the use of ensemble (combining) classifiers. Research indicates that ensemble individual classifiers lead to a significant improvement in classification performance by having them vote for the most popular class. This paper explores the predicted behaviour of five classifiers for different types of noise in terms of credit risk prediction accuracy, and how could such accuracy be improved by using pairs of classifier ensembles. Benchmarking results on five credit datasets and comparison with the performance of each individual classifier on predictive accuracy at various attribute noise levels are presented. The experimental evaluation shows that the ensemble of classifiers technique has the potential to improve prediction accuracy.

Keywords

Supervised learning / statistical pattern recognition / ensemble / credit risk / prediction

References

[1] Aha D.W., Kibbler D.W., Albert M.K.. Instance-based learning algorithms. Machine Learning, 1991, 6(37): 37-66.

[2] Aha D.W.. Tolerating noisy, irrelevant and novel attributes in instance-based learning algorithms. International Journal of Man Machine Studies, 1992, 36(2): 267-287.

[3] Albright, H.T. (1994). Construction of polynomial classifier for consumer loan applications using genetic algorithms. Department of Systems and Engineering, University of Virginia, Working Paper

[4] Altman E.I.. Financial ratios, discriminant analysis and prediction of corporate bankruptcy. Journal of Finance, 1968, 23(4): 589-609.

[5] Baesens B., Setiono R., Mues C.H., Vanthienen J.. Using neural network rule extraction and decision tables for credit risk evaluation. Management Science, 2003, 49: 312-329.

[6] Bauer E., Kohavi R.. An empirical comparison of voting classification algorithms: bagging, boosting and variants. Machine Learning, 1999, 36: 105-139.

[7] BIS. (2004). International convergence of capital measurement and capital standards: a revised framework. Basel Committee of Banking Supervision, Bank for International Settlements

[8] Blake, C., Keogh, E. & Merz, C.J. (1999). UCI respiratory of machine learning databases. University of California, Irvine, Department of Information and Computer Sciences

[9] Boyle M., Crook J.N., Hamilton R., Thomas L.C.. Thomas L.C., Crook J.N., Edelman D.B.. Methods for credit scoring applied to slow payers. Credit Scoring and Credit Control, 1992, Oxford: Clarendon 75-90.

[10] Breiman L.. Bagging predictors. Machine Learning, 1996, 26(2): 123-140.

[11] Breiman, L., Friedman, J., Olshen, R. & Stone, C. (1984). Classification and Regression Trees. Wadsworth

[12] Carter, C. & Catlett, J. (1987). Assessing credit card applications using machine learning. IEEE Expert, fall: 71–79

[13] Chatterjee S., Barcun S.. A nonparametric approach to credit screening. Journal of American Statistical Association, 1971, 65: 50-154.

[14] Coffman, J.Y. (1986). The proper role of tree analysis in forecasting the risk behaviour of borrowers. MDS reports 3, 4, 7 and 9. Management Decision Systems, Atlanta

[15] Davis R.H., Eedelman G.B., Gammerman A.J.. Machine learning algorithms for credit card applications. IMA Journal of Mathematics Applied Business and Industry, 1992, 4: 43-51.

[16] Desai Y.S., Crook J.N., Overstreet G.A.. A comparison of neural networks and linear scoring models in the credit environment. European Journal of Operations Research, 1996, 85: 24-37.

[17] Desai, V.S., Convay, J.N. & Overstreet, G.A. Credit scoring models in credit-union environment using neural networks and genetic algorithms. IMA Journal of Mathematics Applied in Business and Industry, 8: 323–346

[18] Dietterich T.. An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Machine Learning, 2000, 40(2): 139-158.

[19] Drucker H., Cortes C., Jackel L.D., Lecun Y., Vapkin V.. Boosting and other ensemble methods. Neural Computation, 1994, 6: 1289-1301.

[20] Feldman D., Gross S.. Mortgage default: classification tree analysis. Journal of Real Estate Finance and Economics, 2005, 30: 369-396.

[21] Forgaty T.C., Ireson N.S.. Evolving Bayesian classifier for credit control — a comparison with other machine learning methods. IMA Journal of Mathematics Applied in Business and Industry, 1993, 5: 63-76.

[22] Freund Y., Schapire R.. A decision theoretic generalization of on-line learning and an application to boosting. Journal of Computing and Systems, 1996, 55: 119-139.

[23] Frydman H.E., Altman E.E., Kao D.. Introducing recursive partitioning for financial classification: the case of financial distress. Journal of Finance, 1985, 40(1): 269-291.

[24] Henley W.E.. Statistical aspects of credit scoring. PhD Dissertation, 1995, Milton Keynes, United Kingdom: The Open University

[25] Hand, D.J. (2008). Private communication

[26] Hand D.J., Vinciotti V.. Choosing k for two-class nearest neighbour classifiers with unbalanced classes. Pattern Recognition Letters, 2003, 24: 1555-1562.

[27] Hand D.J., Henley W.E.. Statistical classification methods in consumer credit scoring: a review. Journal of the Royal Statistical Society, 1997, 160: 523-541.

[28] Henley W.E., Hand D.J.. A k-nearest neighbour classifier for assessing consumer risk. Statistician, 1996, 44: 77-95.

[29] Ho, T. K. (1995). Random decision forests. In: Holmes, C.C., Adams, N.M. (eds.), Proceedings of the 3rd International Conference on Document Analysis and Recognition, 278–282

[30] Holmes C.C., Adams N.M.. A probabilistic nearest neighbour method for statistical pattern recognition. Journal of Royal Statistical Society, Series B, 2002, 64: 295-306.

[31] Islam, M.J., Wu, Q.M.J., Ahmadi, M. & Sid-Ahmed, M.A. (2007). Investigating the performance of naïve Bayes classifiers and k-nearest neighbor classifiers. In: International Conference on Convergence Information Technology, 1541–1546, November, 21-23, 2007

[32] Jensen H.L.. Using neural networks for credit scoring. Managerial Finance, 1992, 18: 15-26.

[33] Khalik, A. & El-Sheshai, K.M. (1980). Information choice and utilization in an experiment of default prediction. Journal of Accounting Research, autumn: 325–342

[34] Kirk E.E.. Experimental Design, 1982, 2nd Ed. Brooks: Cole Publishing Company, Monterey, CA

[35] Kittler J., Hatef M., Duin R.P.W., Matas J.. On combining classifiers. IEEE Transaction on Pattern Analysis and Machine Intelligence, 1998, 20(3): 226-239.

[36] Kononenko, I. (1991). Semi-naïve Bayesian classifier. In: Proceedings of European conference on Artificial Intelligence, 206–219

[37] Lacher R.C., Oats P.K., Sharma S., Fant L.F.. A neural network for classifying the financial health of a firm. European Journal of Operational Research, 1995, 85: 53-65.

[38] Leonard K.J.. Empirical Bayes analysis of the commercial loans evaluation process. Statistics Probability Letters, 1993, 18: 289-296.

[39] Mahlhotra R., Malhotra D.L.. Evaluating consumer loans using neural networks. Omega: the International Journal of Management Science, 2003, 31(2): 83-96.

[40] Makowski P.. Credit scoring branches out. Credit World, 1985, 75: 30-37.

[41] MINITAB. Statistical software for windows 9.0, 2002, PA, USA: MINITAB, Inc.

[42] Neumann, D.E. (2002). An enhanced neural network technique for software risk analysis. IEEE Transactions on Software Engineering, 904–912

[43] Ong C.S., Huang J.J., Tzeng G.H.. Building credit scoring models using genetic programming. Expert Systems with Applications, 2005, 29: 41-47.

[44] Piramuthu S.. Financial credit risk evaluation with neural and neurofuzzy systems. European Journal of Operational Research, 1999, 112: 310-321.

[45] Quinlan J.R.. C.4.5: Programs for Machine Learning, 1993, Los Altos, California: Morgan Kauffman Publishers, Inc.

[46] Ripley B.D.. Pattern Recognition and Neural Networks, 1992, New York: John Wiley: Cambridge University Press

[47] Rosenberg E., Gleit A.. Quantitative methods in credit management: a survey. Operations Research, 1994, 42(4): 589-613.

[48] Salchenberger L.M., Cinar E.M., Lash N.A.. Neural networks: a new tool for predicting thrift failures. Decision Sciences, 1992, 23: 899-916.

[49] Schafer J.L.. Analysis of Incomplete Multivariate Data, 1997, London: Chapman and Hall.

[50] Tam K.Y., Kiang M.Y.. Managerial applications of neural networks: the case of bank failure predictions. Management Science, 1992, 38(7): 926-947.

[51] Twala B.. An empirical comparison of techniques for handling incomplete data using decision trees. Applied Artificial Intelligence, 2009, 23(5): 373-405.

[52] Twala B., Jones M.C., Hand D.J.. Good methods for coping with missing data in decision trees. Pattern Recognition Letters, 2008, 29: 950-956.

[53] West D.. Neural network credit scoring models. Computers & Operations Research, 2000, 27: 1131-1152.

[54] Wiginton J.C.. A note on the comparison of logit and discriminant models of consumer credit behavior. Journal of Financial and Quantitative Analysis, 1980, 15: 757-770.

[55] Wolpert D.. Stacked generalization. Neural Networks, 1992, 5(2): 241-259.

[56] Yobas, M.B., Crook, J.N. & Ross, P. (1997). Credit scoring using neural and evolutionary techniques. Credit Research Centre, University of Edinburgh, Working Paper 7/2