On the use of explainable AI for susceptibility modeling: Examining the spatial pattern of SHAP values

Abstract

Hydro-morphological processes (HMP, any natural phenomenon contained within the spectrum defined between debris flows and flash floods) are globally occurring natural hazards which pose great threats to our society, leading to fatalities and economical losses. For this reason, understanding the dynamics behind HMPs is needed to aid in hazard and risk assessment. In this work, we take advantage of an explainable deep learning model to extract global and local interpretations of the HMP occurrences across the whole Chinese territory. We use a deep neural network architecture and interpret the model results through the spatial pattern of SHAP values. In doing so, we can understand the model prediction on a hierarchical basis, looking at how the predictor set controls the overall susceptibility as well as doing the same at the level of the single mapping unit. Our model accurately predicts HMP occurrences with AUC values measured in a ten-fold cross-validation ranging between 0.83 and 0.86. This level of predictive performance attests for an excellent prediction skill. The main difference with respect to traditional statistical tools is that the latter usually lead to a clear interpretation at the expense of high performance, which is otherwise reached via machine/deep learning solutions, though at the expense of interpretation. The recent development of explainable AI is the key to combine both strengths. In this work, we explore this combination in the context of HMP susceptibility modeling. Specifically, we demonstrate the extent to which one can enter a new level of data-driven interpretation, supporting the decision-making process behind disaster risk mitigation and prevention actions.

Keywords

Hydro-morphological processes / SHAP maps / Explainable AI / China

References

M. Ahmed, H. Tanyas, R. Huser, A. Dahal, G. Titti, L. Borgatti, M. Francioni, L. Lombardo. Dynamic rainfall-induced landslide susceptibility: a step towards a unified forecasting system. Int. J. Appl. Earth Obs. Geoinf., 125 (2023), Article 103593

Albawi, S., Mohammed, T. A. and Al-Zawi, S. (2017) Understanding of a convolutional neural network. In 2017 International Conference on Engineering and Technology (ICET), pp. 1–6.

N.R. Aljohani, A. Fayoumi, S.-U. Hassan. A novel focal-loss and class-weight-aware convolutional neural network for the classification of in-text citations. J. Inf. Sci. (2021)

H.A. Al-Najjar, B. Pradhan, G. Beydoun, R. Sarkar, H.J. Park, A. Alamri. A novel method using explainable artificial intelligence (XAI)-based Shapley Additive Explanations for spatial landslide prediction using Time-Series SAR dataset. Gondw. Res., 123 (2023), pp. 107-124

M. Alvioli, I. Marchesini, B. Pokharel, K. Gnyawali, S. Lim. Geomorphological slope units of the Himalayas. J. Maps (2022), pp. 1-14

G. Amato, M. Fiorucci, S. Martino, L. Lombardo, L. Palombi. Earthquake-triggered landslide susceptibility in Italy by means of Artificial Neural Network. Bull. Eng. Geol. Environ., 82 (5) (2023), p. 160

S.S. Band, S. Janizadeh, S. Chandra Pal, A. Saha, R. Chakrabortty, A.M. Melesse, A. Mosavi. Flash flood susceptibility modeling using new approaches of hybrid and ensemble tree-based machine learning algorithms. Remote Sens. (Basel), 12 (21) (2020), p. 3568

M.L. Baptista, K. Goebel, E.M. Henriques. Relation between prognostics predictor evaluation metrics and local interpretability SHAP values. Artif. Intell., 306 (2022), Article 103667

A. Brenning. Spatial prediction models for landslide hazards: review, comparison and evaluation. Natural Hazards and Earth System Science, 5 (6) (2005), pp. 853-862

A. Brenning. Statistical geocomputing combining R and SAGA: The example of landslide susceptibility analysis with generalized additive models. Hamburger Beitr ̈agezur Physischen Geographie Und Landschafts ̈okologie, 19 (23–32) (2008), p. 410

E. Bryce, L. Lombardo, C. van Westen, H. Tanyas, D. Castro-Camilo. Unified landslide hazard assessment using hurdle models: a case study in the Island of Dominica. Stoch. Env. Res. Risk A, 36 (8) (2022), pp. 2071-2084

Bui, D. T., Hoang, N.-D., Mart ́ınez- ́Alvarez, F., Ngo, P.-T. T., Hoa, P. V., Pham, T. D., Samui, P. and Costache, R. (2020) A novel deep learning neural network approach for predicting flash flood susceptibility: A case study at a high frequency tropical storm area. Sci. Total Environ. 701, 134413.

D.C. Camilo, L. Lombardo, P.M. Mai, J. Dou, R. Huser. Handling high predictor dimensionality in slope-unit-based landslide susceptibility models through LASSO-penalized Generalized Linear Model. Environ. Model. Softw., 97 (2017), pp. 145-156

T. Can, H.A. Nefeslioglu, C. Gokceoglu, H. Sonmez, T.Y. Duman. Susceptibility assessments of shallow earthflows triggered by heavy rainfall at three catchments by logistic regression analyses. Geomorphology, 72 (1–4) (2005), pp. 250-271

A. Carrara, G. Crosta, P. Frattini. Comparing models of debris-flow susceptibility in the alpine environment. Geomorphology, 94 (3–4) (2008), pp. 353-378

Carrara, A., Cardinali, M., Guzzetti, F., Reichenbach, P., 1995. GIS Technology in Mapping Landslide Hazard. In: Carrara, A., Guzzetti, F. (Eds.), Geographical Information Systems in Assessing Natural Hazards. Advances in Natural and Technological Hazards Research, vol 5. Pp. 135–175. Springer, Dordrecht. https://doi.org/10.1007/978-94-015-8404-3_8.

L. Chang, G. Xing, H. Yin, L. Fan, R. Zhang, N. Zhao, F. Huang, J. Ma. Landslide susceptibility evaluation and interpretability analysis of typical loess areas based on deep learning. Natural Hazards Research, 3 (2) (2023), pp. 155-169

Cisneros, D., Richards, J., Dahal, A., Lombardo, L. and Huser, R., 2023. Deep graphical regression for jointly moderate and extreme Australian wildfires. arXiv preprint arXiv:2308.14547.

E. Collini, L.I. Palesi, P. Nesi, G. Pantaleo, N. Nocentini, A. Rosi. Predicting and understanding landslide events with explainable AI. IEEE Access, 10 (2022), pp. 31175-31189

A. Dahal, L. Lombardo. Explainable artificial intelligence in geoscience: A glimpse into the future of landslide susceptibility modeling. Comput. Geosci., 176 (2023), Article 105364

J. De Villiers, E. Barnard. Backpropagation neural nets with one and two hidden layers. IEEE Trans. Neural Netw., 4 (1) (1993), pp. 136-141

Di Napoli, M., Tanyas, H., Castro-Camilo, D., Calcaterra, D., Cevasco, A., Di Martire, D., Pepe, G., Brandolini, P. and Lombardo, L., 2023. On the estimation of landslide intensity, hazard and density via data-driven models. Natural Hazards, In press.

A. Dikshit, B. Pradhan. Interpretable and explainable AI (XAI) model for spatial drought prediction. Sci. Total Environ., 801 (2021), Article 149797

A. Domeneghetti, S. Vorogushyn, A. Castellarin, B. Merz, A. Brath. Probabilistic flood hazard mapping: effects of uncertain boundary conditions. Hydrol. Earth Syst. Sci., 17 (8) (2013), pp. 3127-3140

R. Fell, J. Corominas, C. Bonnard, L. Cascini, E. Leroi, W.Z. Savage, et al.. Guidelines for landslide susceptibility, hazard and risk zoning for land-use planning. Eng. Geol., 102 (3–4) (2008), pp. 99-111

R. Fluss, D. Faraggi, B. Reiser. Estimation of the Youden Index and its associated cutoff point. Biometrical Journal: Journal of Mathematical Methods in Biosciences, 47 (4) (2005), pp. 458-472

A.S. Fotheringham, C. Brunsdon, M. Charlton. Geographically weighted regression: the analysis of spatially varying relationships. John Wiley & Sons (2003)

P. Frattini, G. Crosta, A. Carrara. Techniques for evaluating the performance of landslide susceptibility models. Eng. Geol., 111 (1) (2010), pp. 62-72

S.L. Gariano, F. Guzzetti. Landslides in a changing climate. Earth Sci. Rev., 162 (2016), pp. 227-252

J. Goetz, A. Brenning, H. Petschko, P. Leopold. Evaluating machine learning and statistical prediction techniques for landslide susceptibility modeling. Comput. Geosci., 81 (2015), pp. 1-11

Gunning, D. (2017) Explainable Artificial Intelligence (XAI). Defense Advanced Research Projects Agency (DARPA), nd Web 2(2), 1.

F. Guzzetti, et al.. Landslide hazard evaluation: a review of current techniques and their application in a multi-scale study, Central Italy. Geomorphology, 31 (1-4) (1999), pp. 181-216

F. Guzzetti, P. Reichenbach, F. Ardizzone, M. Cardinali, M. Galli. Estimating the quality of landslide susceptibility models. Geomorphology, 81 (1–2) (2006), pp. 166-184

B. He, X. Huang, M. Ma, Q. Chang, Y. Tu, Q. Li, K. Zhang, Y. Hong. Analysis of flash flood disaster characteristics in China from 2011 to 2015. Nat. Hazards, 90 (2018), pp. 407-420

K. He, H. Tanyas, L. Chang, X. Hu, G. Luo, L. Lombardo. Modelling InSAR-derived hillslope velocities with multivariate statistics: A first attempt to generate interpretable predictions. Remote Sens. Environ., 289 (2023), Article 113518

R.E. Horton. Drainage-basin characteristics. Trans. Am. Geophys. Union, 13 (1) (1932), pp. 350-361

D.W. Hosmer Jr, S. Lemeshow, R.X. Sturdivant. . Applied Logistic Regression, Volume 398, John Wiley & Sons (2013)

F.S. Hosseini, B. Choubin, A. Mosavi, N. Nabipour, S. Shamshirband, H. Darabi, A.T. Haghighi. Flash-flood hazard assessment using ensembles and Bayesian-based machine learning models: Application of the simulated annealing feature selection method. Sci. Total Environ., 711 (2020), Article 135161

L. Jacobs, M. Kervyn, P. Reichenbach, M. Rossi, I. Marchesini, M. Alvioli, O. Dewitte. Regional susceptibility assessments with heterogeneous landslide information: Slope unit-vs. pixel-based approach. Geomorphology, 356 (2020), Article 107084

A.N. Kern, P. Addison, T. Oommen, S.E. Salazar, R.A. Coffman. Machine learning based predictive modeling of debris flow probability following wildfire in the intermountain Western United States. Math. Geosci., 49 (2017), pp. 717-735

M. Kobiyama, R.F. Goerl. Quantitative method to distinguish flood and flash flood as disasters. SUISUI Hydrological Research Letters, 1 (2007), pp. 11-14

E.E. Koks, J. Rozenberg, C. Zorn, M. Tariverdi, M. Vousdoukas, S.A. Fraser, J. Hall, S. Hallegatte. A global multi-hazard risk analysis of road and railway infrastructure assets. Nat. Commun., 10 (1) (2019), p. 2677

O. Korup, A. Stolle. Landslide prediction from machine learning. Geol. Today, 30 (1) (2014), pp. 26-33

Z. Li. Extracting spatial effects from machine learning model using local interpretation method: An example of SHAP and XGBoost. Comput. Environ. Urban Syst., 96 (2022), Article 101845

Li, Y., Yuan, Y., 2017. Convergence analysis of two-layer neural networks with ReLU activation. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 597–607.

P. Lima, S. Steger, T. Glade, F.G. Murillo-Garc ́ıa. Literature review and bibliometric analysis on data-driven assessment of landslide susceptibility. J. Mt. Sci., 19 (6) (2022), pp. 1670-1698

Q. Lin, P. Lima, S. Steger, T. Glade, T. Jiang, J. Zhang, T. Liu, Y. Wang. National-scale data-driven rainfall induced landslide susceptibility mapping for China by accounting for incomplete landslide data. Geosci. Front., 12 (6) (2021), Article 101248

Q. Lin, S. Steger, M. Pittore, J. Zhang, L. Wang, T. Jiang, Y. Wang. Evaluation of potential changes in landslide susceptibility and landslide occurrence frequency in China under climate change. Sci. Total Environ., 850 (2022), Article 158049

Y. Liu, Y. Huang, J. Wan, Z. Yang, X. Zhang. Analysis of human activity impact on flash floods in China from 1950 to 2015. Sustainability, 13 (1) (2021), p. 217

L. Lombardo, H. Bakka, H. Tanyas, C. van Westen, P.M. Mai, R. Huser. Geostatistical modeling to capture seismic-shaking patterns from earthquake-induced landslides. J. Geophys. Res. Earth, 124 (7) (2019), pp. 1958-1980

L. Lombardo, P.M. Mai. Presenting logistic regression-based landslide susceptibility results. Eng. Geol., 244 (2018), pp. 14-24

L. Lombardo, T. Opitz, F. Ardizzone, F. Guzzetti, R. Huser. Space-time landslide predictive modelling. Earth Sci. Rev. (2020), Article 103318

Lubo-Robles, D., Devegowda, D., Jayaram, V., Bedle, H., Marfurt, K. J., Pranter, M. J., 2020. Machine learning model interpretability using SHAP values: Application to a seismic facies classification task. In: SEG International Exposition and 90th Annual Meeting, pp. 1460–1464.

L. Marchi, M. Borga, E. Preciso, E. Gaume. Characterisation of selected extreme flash floods in Europe and implications for flood risk management. J. Hydrol., 394 (1–2) (2010), pp. 118-133

A. Merghadi, A.P. Yunus, J. Dou, J. Whiteley, B. ThaiPham, D.T. Bui, R. Avtar, B. Abderrahmane. Machine learning methods for landslide susceptibility studies: A comparative overview of algorithm performance. Earth Sci. Rev. (2020), Article 103225

B. Merz, F. Elmer, A. Thieken. Significance of “high probability/low damage” versus “low probability/high damage” flood events. Nat. Hazards Earth Syst. Sci., 9 (3) (2009), pp. 1033-1046

Molnar, C., 2020. Interpretable machine learning. Lulu.com, 318 pp.

Moreira, A., Krieger, G., Hajnsek, I., Hounam, D., Werner, M., Riegger, S. Settelmeyer, E., 2004. TanDEM-X: a TerraSAR-X add-on satellite for single-pass SAR interferometry. IEEE Transactions on Geoscience and Remote Sensing 45(11), 3317–3341.

Nicu, I. C., Elia, L., Rubensdotter, L., Tanya ¸s, H. and Lombardo, L. (2023) Multi-hazard susceptibility mapping of cryospheric hazards in a high-Arctic environment: Svalbard Archipelago. Earth System Science Data 15(1), 447–464.

T. Opitz, H. Bakka, R. Huser, L. Lombardo. High-resolution bayesian mapping of landslide hazard with unobserved trigger event. Ann. Appl. Stat., 16 (3) (2022), pp. 1653-1675

A. Ozdemir. Landslide susceptibility mapping of vicinity of Yaka Landslide (Gelendost, Turkey) using conditional probability approach in GIS. Environ. Geol., 57 (2009), pp. 1675-1686

M. Panahi, A. Jaafari, A. Shirzadi, H. Shahabi, O. Rahmati, E. Omidvar, S. Lee, D.T. Bui. Deep learning neural networks for spatially explicit prediction of flash flood probability. Geosci. Front., 12 (3) (2021), Article 101076

C. Panati, S. Wagner, S. Br ̈uggenwirth. Feature Relevance Evaluation using Grad-CAM, LIME and SHAP for Deep Learning SAR Data Classification. 2022 23rd International Radar Symposium (IRS) (2022), pp. 457-462

N.-W. Park. Using maximum entropy modeling for landslide susceptibility mapping with multiple geoenvironmental data sets. Environ. Earth Sci., 73 (2015), pp. 937-949

H. Petschko, A. Brenning, R. Bell, J. Goetz, T. Glade. Assessing the quality of landslide susceptibility maps—case study lower austria. Nat. Hazards Earth Syst. Sci., 14 (1) (2014), pp. 95-118

S. Ragettli, J. Zhou, H. Wang, C. Liu, L. Guo. Modeling flash floods in ungauged mountain catchments of China: A decision tree learning approach for parameter regionalization. J. Hydrol., 555 (2017), pp. 330-346

D. Ramyachitra, P. Manikandan. Imbalanced dataset classification and solutions: a review. International Journal of Computing and Business Research (IJCBR), 5 (4) (2014), pp. 1-29

P. Reichenbach, M. Rossi, B.D. Malamud, M. Mihir, F. Guzzetti. A review of statistically-based landslide susceptibility models. Earth Sci. Rev., 180 (2018), pp. 60-91

M.T. Ribeiro, S. Singh, C. Guestrin. “why should I trust you?” explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016), pp. 1135-1144

K. Roshan, A. Zafar. Using Kernel SHAP XAI Method to Optimize the Network Anomaly Detection Model. 2022 9th International Conference on Computing for Sustainable Global Development (INDIACom) (2022), pp. 74-80

Samek, W., Wiegand, T. and M ̈uller, K.-R. (2017) Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296.

N. Santangelo, A. Santo, G. Di Crescenzo, G. Foscari, V. Liuzza, S. Sciarrotta, V. Scorpio. Flood susceptibility assessment in a highly urbanized alluvial fan: the case study of Sala Consilina (southern Italy). Nat. Hazards Earth Syst. Sci., 11 (10) (2011), pp. 2765-2780

G. Schumann, R. Hostache, C. Puech, L. Hoffmann, P. Matgen, F. Pappenberger, L. Pfister. High-resolution 3-D flood information from radar imagery for flood hazard management. IEEE Trans. Geosci. Remote Sens., 45 (6) (2007), pp. 1715-1725

S.A. Schumm. Evolution of drainage systems and slopes in badlands at Perth Amboy, New Jersey. Geol. Soc. Am. Bull., 67 (5) (1956), pp. 597-646

A. Shirzadi, H. Shahabi, K. Chapi, D.T. Bui, B.T. Pham, K. Shahedi, B.B. Ahmad. A comparative study between popular statistical and machine learning methods for simulating volume of landslides. Catena, 157 (2017), pp. 213-226

A. Shrikumar, P. Greenside, A. Kundaje. Learning important features through propagating activation differences. In International Conference on Machine Learning (2017), pp. 3145-3153

A. Singh, S. Sengupta, V. Lakshminarayanan. Explainable deep learning models in medical image analysis. Journal of Imaging, 6 (6) (2020), p. 52

C. Song, X. Shi, J. Wang. Spatiotemporally varying coefficients (stvc) model: A bayesian local regression to detect spatial and temporal nonstationarity in variables relationships. Ann. GIS, 26 (3) (2020), pp. 277-291

S. Steger, M. Moreno, A. Crespi, P.J. Zellner, S.L. Gariano, M.T. Brunetti, M. Melillo, S. Peruccacci, F. Marra, R. Kohrs, J. Goetz, V. Mair, M. Pittore. Deciphering seasonal effects of triggering and preparatory precipitation for improved shallow landslide prediction using generalized additive mixed models. Natural Hazards and Earth System Sciences, 23 (2023), pp. 1483-1506

A.N. Strahler. Dynamic basis of geomorphology. Geol. Soc. Am. Bull., 63 (9) (1952), pp. 923-938

E. Strumbelj, I. Kononenko. Explaining prediction models and individual predictions with feature contributions. Knowledge and Information Systems, 41 (2014), pp. 647-665

D. Sun, D. Chen, J. Zhang, C. Mi, Q. Gu, H. Wen. Landslide Susceptibility Mapping Based on Interpretable Machine Learning from the Perspective of Geomorphological Differentiation. Land, 12 (5) (2023), p. 1018

F.S. Tehrani, M. Calvello, Z. Liu, L. Zhang, S. Lacasse. Machine learning and landslide studies: recent advances and applications. Nat. Hazards, 114 (2) (2022), pp. 1197-1245

G. Titti, G.N. Napoli, C. Conoscenti, L. Lombardo. Cloud-based interactive susceptibility modeling of gully erosion in Google Earth Engine. Int. J. Appl. Earth Obs. Geoinf., 115 (2022), Article 103089

J.T. Townsend. Theoretical analysis of an alphabetic confusion matrix. Percept. Psychophys., 9 (1971), pp. 40-50

I. Ullah, K. Liu, T. Yamamoto, M. Zahid, A. Jamal. Modeling of machine learning with shap approach for electric vehicle charging station choice behavior prediction. Travel Behav. Soc., 31 (2023), pp. 78-92

B. van den Bout, V.G. Jetten, C.J. van Westen, L. Lombardo. A breakthrough in fast flood simulation. Environ. Model. Softw., 168 (2023), Article 105787

N. Wang, W. Cheng, B. Wang, Q. Liu, C. Zhou. Geomorphological regionalization theory system and division methodology of China. J. Geog. Sci., 30 (2) (2020), pp. 212-232

N. Wang, W. Cheng, M. Marconcini, F. Bachofer, C. Liu, J. Xiong, L. Lombardo. Space-time susceptibility modeling of hydro-morphological processes at the Chinese national scale. Eng. Geol., 301 (2022), Article 106586

Wang, N., Cheng, W., Lombardo, L., Xiong, J., Guo, L., 2021. Statistical spatiotemporal analysis of hydro-morphological processes in China during 1950–2015. Stochastic Environmental Research and Risk Assessment 36, 2377–2397.

Wang, D., Thuńell, S., Lindberg, U., Jiang, L., Trygg, J. and Tysklind, M., 2022a. Towards better process management in wastewater treatment plants: Process analytics based on SHAP values for tree-based machine learning methods. Journal of Environmental Management 301, 113941.

J. Xiong, J. Li, W. Cheng, N. Wang, L. Guo. A GIS-based support vector machine model for flash flood vulnerability assessment and mapping in China. ISPRS Int. J. Geo Inf., 8 (7) (2019), p. 297

J. Xiong, Q. Pang, W. Cheng, N. Wang, Z. Yong. Reservoir risk modelling using a hybrid approach based on the feature selection technique and ensemble methods. Geocarto Int. (2020), pp. 1-22

Yadav, R., Huser, R., Opitz, T. and Lombardo, L., 2022. Joint modeling of landslide counts and sizes using spatial marked point processes with sub-asymptotic mark distributions. arXiv preprint arXiv:2205.09908.

Y.-K. Yeon, J.-G. Han, K.H. Ryu. Landslide susceptibility mapping in Injae, Korea, using a decision tree. Eng. Geol., 116 (3–4) (2010), pp. 274-283

I. Yilmaz. Landslide susceptibility mapping using frequency ratio, logistic regression, artificial neural networks and their comparison: a case study from Kat landslides (Tokat—Turkey). Comput. Geosci., 35 (6) (2009), pp. 1125-1138

L. Yu, A. Porwal, E.-J. Holden, M.C. Dentith. Towards automatic lithological classification from remote sensing data using support vector machines. Comput. Geosci., 45 (2012), pp. 229-239

J. Zhang, X. Ma, J. Zhang, D. Sun, X. Zhou, C. Mi, H. Wen. Insights into geospatial heterogeneity of landslide susceptibility based on the SHAP-XGBoost model. J. Environ. Manage., 332 (2023), Article 117357

G. Zhao, B. Pang, Z. Xu, J. Yue, T. Tu. Mapping flood susceptibility in mountainous areas on a national scale in China. Sci. Total Environ., 615 (2018), pp. 1133-1142

X. Zhou, H. Wen, Z. Li, H. Zhang, W. Zhang. An interpretable model for the susceptibility of rainfall-induced shallow landslides based on SHAP and XGBoost. Geocarto Int., 37 (26) (2022), pp. 13419-13450