Hybrid stacking ensemble algorithm and simulated annealing optimization for stability evaluation of underground entry-type excavations

Abstract

The stability of underground entry-type excavations (UETEs) is of paramount importance for ensuring the safety of mining operations. As more engineering cases are accumulated, machine learning (ML) has demonstrated great potential for the stability evaluation of UETEs. In this study, a hybrid stacking ensemble method aggregating support vector machine (SVM), k-nearest neighbor (KNN), decision tree (DT), random forest (RF), multilayer perceptron neural network (MLPNN) and extreme gradient boosting (XGBoost) algorithms was proposed to assess the stability of UETEs. Firstly, a total of 399 historical cases with two indicators were collected from seven mines. Subsequently, to pursue better evaluation performance, the hyperparameters of base learners (SVM, KNN, DT, RF, MLPNN and XGBoost) and meta learner (MLPNN) were tuned by combining a five-fold cross validation (CV) and simulated annealing (SA) approach. Based on the optimal hyperparameters configuration, the stacking ensemble models were constructed using the training set (75% of the data). Finally, the performance of the proposed approach was evaluated by two global metrics (accuracy and Cohen’s Kappa) and three within-class metrics (macro average of the precision, recall and F₁-score) on the test set (25% of the data). In addition, the evaluation results were compared with six base learners optimized by SA. The hybrid stacking ensemble algorithm achieved better comprehensive performance with the accuracy, Kappa coefficient, macro average of the precision, recall and F₁-score were 0.92, 0.851, 0.885, 0.88 and 0.883, respectively. The rock mass rating (RMR) had the most important influence on evaluation results. Moreover, the critical span graph (CSG) was updated based on the proposed model, representing a significant improvement compared with the previous studies. This study can provide valuable guidance for stability analysis and risk management of UETEs. However, it is necessary to consider more indicators and collect more extensive and balanced dataset to validate the model in future.

Keywords

Underground entry-type excavations (UETEs) / Hybrid stacking ensemble / Machine learning / Simulated annealing / Critical span graph / Base and meta learners

CRediT authorship contribution statement

Leilei Liu: Methodology, Software, Visualization, Writing - original draft. Guoyan Zhao: Resources, Formal analysis, Project administration. Weizhang Liang: Conceptualization, Investigation, Supervision, Funding acquisition, Writing - review & editing. Zheng Jian: Validation, Data curation.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

The authors would like to express their appreciation to Kumar (2003) for making his database available for this work. In addition, this work was supported by the National Natural Science Foundation of China (Grant No. 52204117), and the Natural Science Foundation of Hunan Province, China (Grant No. 2022JJ40601).

References

[1] Abordan, A., & Szabo, N. P. (2021). Machine learning based approach for the interpretation of engineering geophysical sounding logs. Acta Geodaetica et Geophysica, 56(4), 681-696.

[2] Akay, B., & Karaboga, D. (2012). A modified Artificial Bee Colony algorithm for real-parameter optimization. Information Sciences, 192 (1), 120-142.

[3] Alqahtani, A., Shah, M. I., Aldrees, A., & Javed, M. F. (2022). Comparative Assessment of Individual and Ensemble Machine Learning Models for Efficient Analysis of River Water Quality. Sustainability, 14(3), 1183.

[4] Altmann, A., Tolosi, L., Sander, O., & Lengauer, T. (2010). Permutation importance: A corrected feature importance measure. Bioinformatics, 26(10), 1340-1347.

[5] Amin, M. N., Khan, K., Aslam, F., Shah, M. I., Javed, M. F., Musarat, M. A., & Usanova, K. (2021). Multigene Expression Programming Based Forecasting the Hardened Properties of Sustainable Bagasse Ash Concrete. Materials, 14(19), 5659.

[6] Azam, M., Ahmed, T., Hashmi, M. U., Ahmad, R., Manan, A., Adrees, M., & Sabah, F. (2019). Improvement in Classification Algorithms through Model Stacking with the Consideration of their Correlation. International Journal of Advanced Computer Science and Applications, 10(3), 463-475.

[7] Bertoni, M. A., de Rosa, G. H., & Brega, J. R. F. (2022). Optimum-path forest stacking-based ensemble for intrusion detection. Evolutionary Intelligence, 15(3), 2037-2054.

[8] Bradley, A. P. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition, 30(7), 1145-1159.

[9] Breiman, L. (1996). Stacked regressions. Machine Learning, 24(1), 49-64.

[10] Breiman, L. (2001). Random Forests. Machine Learning, 45(1), 5-32.

[11] Cai, M. F., & Brown, E. T. (2017). Challenges in the Mining and Utilization of Deep Mineral Resources. Engineering, 3(4), 432-433.

[12] Chang, C. C., & Lin, C. J. (2011). LIBSVM: A Library for Support Vector Machines. ACM Transactions on Intelligent Systems and Technology, 2 (3), 1-27.

[13] Chen, T. Q., Guestrin, C., & Assoc Comp, M. (2016). XGBoost: A Scalable Tree Boosting System. In 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), San Francisco, CA (pp.785-794).

[14] Chen, Z. H., Xu, L. Y., Zhang, C. M., Huang, C. C., Wang, M. H., Feng, Z., & Xiong, Y. (2021). CT Radiomics Model for Discriminating the Risk Stratification of Gastrointestinal Stromal Tumors: A Multi-Class Classification and Multi-Center Study. Frontiers in Oncology, 11, 654114.

[15] Cortes, C., & Vapnik, V. (1995). Support-Vector Networks. Machine Learning, 20(3), 273-297.

[16] Dhawan, K. R., Singh, D. N., & Gupta, I. D. (2004). Three-Dimensional Finite Element Analysis of Underground Caverns. International Journal of Geomechanics, 4(3), 224-228.

[17] Ding, W. M., & Wu, S. L. (2019). ABC-based stacking method for multilabel classification. Turkish Journal of Electrical Engineering and Computer Sciences, 27(6), 4231-4245.

[18] Divina, F., Gilson, A., Gomez-Vela, F., Torres, M. G., & Torres, J. E. (2018). Stacking Ensemble Learning for Short-Term Electricity Consumption Forecasting. Energies, 11(4), 949.

[19] Feng, T., Wang, C., Zhang, J., Wang, B., & Jin, Y.-F. (2022). An improved artificial bee colony-random forest (IABC-RF) model for predicting the tunnel deformation due to an adjacent foundation pit excavation. Underground Space, 7(4), 514-527.

[20] Feng, X. T., Liu, J. P., Chen, B. R., Xiao, Y. X., Feng, G. L., & Zhang, F. P. (2017). Monitoring, Warning, and Control of Rockburst in Deep Metal Mines. Engineering, 3(4), 538-545.

[21] Fix, E., & Hodges, J. L. (1989). Discriminatory Analysis. Nonparametric Discrimination: Consistency Properties. International Statistical Review/ Revue Internationale de Statistique, 57(3), 238-247.

[22] Foong, L. K., & Moayedi, H. (2022). Slope stability evaluation using neural network optimized by equilibrium optimization and vortex search algorithm. Engineering with Computers, 38(SUPPL 2), 1269-1283.

[23] Garcia-Gonzalo, E., Fernandez-Muniz, Z., Garcia Nieto, P. J., Bernardo Sanchez, A., & Menendez Fernandez, M. (2016). Hard-Rock Stability Analysis for Span Design in Entry-Type Excavations with Learning Classifiers. Materials, 9(7), 531.

[24] Gislason, P. O., Benediktsson, J. A., & Sveinsson, J. R. (2006). Random Forests for land cover classification. Pattern Recognition Letters, 27(4), 294-300.

[25] Goh, A. T. C., Zhang, Y., Zhang, R., Zhang, W., & Xiao, Y. (2017). Evaluating stability of underground entry-type excavations using multivariate adaptive regression splines and logistic regression. Tunnelling and Underground Space Technology, 70, 148-154.

[26] He, X., Zhao, K. Y., & Chu, X. W. (2021). AutoML: A survey of the state-of-the-art. Knowledge-Based Systems, 212, 106622.

[27] Hu, Q. H., Yu, D. R., & Me, Z. (2008). Neighborhood classifiers. Expert Systems with Applications, 34(2), 866-876.

[28] Javan, S. L., Sepehri, M. M., Javan, M. L., & Khatibi, T. (2019). An intelligent warning model for early prediction of cardiac arrest in sepsis patients. Computer Methods and Programs in Biomedicine, 178, 47-58.

[29] Javed, M. F., Farooq, F., Memon, S. A., Akbar, A., Khan, M. A., Aslam, F., Alyousef, R., Alabduljabbar, H., & Rehman, S. K. U. (2020). New Prediction Model for the Ultimate Axial Capacity of Concrete-Filled Steel Tubes: An Evolutionary Approach. Crystals, 10(9), 741.

[30] Jiang, Y., Xiao, J., Tanabashi, Y., & Sasaki, I. (2003). Theoretical method for assessing stability of underground openings by considering the anisotropy of in-situ stresses. In 3rd International Symposium on Rock Stress, Kumamoto, Japan (pp.551-556).

[31] Kaboli, S. H. A., Selvaraj, J., & Rahim, N. A. (2017). Rain-fall optimization algorithm: A population based algorithm for solving constrained optimization problems. Journal of Computational Science, 19, 31-42.

[32] Kang, Z. Q., Yu, Y., & Wen, H. M. (2011). Damage Evolution of Rock and Acoustic Emission Study about Deep Diversion Tunnel in the Excavation Process. In International Conference on Material Engineering, Architectural Engineering and Informatization (MEAEI 2011), Wuhan, China (pp. 243-246).

[33] Kansara, D., Singh, R., Sanghvi, D., & Kanani, P. (2018). Improving Accuracy of Real Estate Valuation Using Stacked Regression. International Journal of Engineering Development and Research, 6(3), 2321-9939.

[34] Kardani, N., Zhou, A. N., Nazem, M., & Shen, S. L. (2021). Improved prediction of slope stability using a hybrid stacking ensemble method based on finite element analysis and field data. Journal of Rock Mechanics and Geotechnical Engineering, 13(1), 188-201.

[35] Kirkpatrick, S., Gelatt, C. D., Jr., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220(4598), 671-680.

[36] Kumar, P. (2003). Development of empirical and numerical design techniques in burst prone ground at the Red Lake Mine. Vancouver, BC, Canada: University of British Columbia.

[37] Lang, B. (1994). Span design for entry-type excavations. Vancouver, BC, Canada: University of British Columbia.

[38] Li, B., Ding, Q. F., Xu, N. W., Lei, Y. F., Xu, Y., Zhu, Z. P., & Liu, J. F. (2020). Mechanical response and stability analysis of rock mass in high geostress underground powerhouse caverns subjected to excavation. Journal of Central South University, 27(10), 2971-2984.

[39] Li, E. M., Yang, F. H., Ren, M. H., Zhang, X. L., Zhou, J., & Khandelwal, M. (2021). Prediction of blasting mean fragment size using support vector regression combined with five optimization algorithms. Journal of Rock Mechanics and Geotechnical Engineering, 13(6), 1380-1397.

[40] Liang, W., Luo, S., Zhao, G., & Wu, H. (2020a). Predicting Hard Rock Pillar Stability Using GBDT, XGBoost, and LightGBM Algorithms. Mathematics, 8(5), 765.

[41] Liang, W., Sari, A., Zhao, G., McKinnon, S. D., & Wu, H. (2020b). Short-term rockburst risk prediction using ensemble learning methods. Natural Hazards, 104(2), 1923-1946.

[42] Luo, L. K., Zhang, X. D., Peng, H., Lv, W. H., & Zhang, Y. (2013). A new pruning method for decision tree based on structural risk of leaf node. Neural Computing & Applications, 22, 17-26.

[43] Mahajan, A., Sharma, N., Aparicio-Obregon, S., Alyami, H., Alharbi, A., Anand, D., Sharma, M., & Goyal, N. (2022). A Novel Stacking-Based Deterministic Ensemble Model for Infectious Disease Prediction. Mathematics, 10(10), 3390.

[44] Malkowski, P., Niedbalski, Z., & Bednarek, L. (2021). Automatic Monitoring System Designed for Controlling the Stability of Underground Excavation. Inzynieria Mineralna-Journal of the Polish Mineral Engineering Society, 2(2), 15-29.

[45] Markoulidakis, I., Rallis, I., Georgoulas, I., Kopsiaftis, G., Doulamis, A., & Doulamis, N. (2021). Multiclass Confusion Matrix Reduction Method and Its Application on Net Promoter Score Classification Problem. Technologies, 9(4), 81.

[46] Mathews, K. E., Hoek, E., Wyllie, D. C., & Stewart, S. B. V. (1981). Prediction of Stable Excavation Spans for Mining at Depths Below 1,000 Meters in Hard Rock. Ottawa: Department Energy, Mines and Resources.

[47] McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276-282.

[48] Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953). Equation of State Calculations by Fast Computing Machines. Journal of Chemical Physics, 21(6), 1087-1092.

[49] Metropolis, N., & Ulam, S. (1949). The Monte Carlo Method. Journal of the American Statistical Association, 44(247), 335-341.

[50] Monjezi, M., Ahmadi, M., Sheikhan, M., Bahrami, A., & Salimi, A. R. (2010). Predicting blast-induced ground vibration using various types of neural networks. Soil Dynamics and Earthquake Engineering, 30(11), 1233-1236.

[51] Myles, A. J., Feudale, R. N., Liu, Y., Woody, N. A., & Brown, S. D. (2004). An introduction to decision tree modeling. Journal of Chemometrics, 18(6), 275-285.

[52] Nieto, P. J. G., Garcia-Gonzalo, E., Lasheras, F. S., Paredes-Sanchez, J. P., & Fernandez, P. R. (2019). Forecast of the higher heating value in biomass torrefaction by means of machine learning techniques. Journal of Computational and Applied Mathematics, 357, 284-301.

[53] Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, E. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12, 2825-2830.

[54] Potvin, Y. (1988). Empirical open stope design in Canada. Vancouver, Canada: University of British Columbia.

[55] Qi, C., & Tang, X. (2018). Slope stability prediction using integrated metaheuristic and machine learning approaches: A comparative study. Computers & Industrial Engineering, 118, 112-122.

[56] Qing, D. G., Zhu, Z. Q., Liu, Q. Y., & He, X. Q. (2009). Numerical simulation for the deformation and stability of large underground stope after excavation. In International Conference on Computer Modeling and Simulation, Macau, China (pp.235-238).

[57] Raffaldi, M. J., Seymour, J. B., Richardson, J., Zahl, E., & Board, M. (2019). Cemented Paste Backfill Geomechanics at a Narrow-Vein Underhand Cut-and-Fill Mine. Rock Mechanics and Rock Engineering, 52(12), 4925-4940.

[58] Rahman, M., Chen, N. S., Elbeltagi, A., Islam, M. M., Alam, M., Pourghasemi, H. R., Tao, W., Zhang, J., Tian, S. F., Faiz, H., Baig, M. A., & Dewan, A. (2021). Application of stacking hybrid machine learning algorithms in delineating multi-type flooding in Bangladesh. Journal of Environmental Management, 295, 113086.

[59] Sari, M. (2022). Two- and three-dimensional stability analysis of underground storage caverns in soft rock (Cappadocia, Turkey) by finite element method. Journal of Mountain Science, 19(4), 1182-1202.

[60] Shah, M. I., Javed, M. F., Alqahtani, A., & Aldrees, A. (2021). Environmental assessment based surface water quality prediction using hyper-parameter optimized machine learning models based on consistent big data. Process Safety and Environmental Protection, 151, 324-340.

[61] Smola, A. J., & Scholkopf, B. (2004). A tutorial on support vector regression. Statistics and Computing, 14(3), 199-222.

[62] Smyth, P., & Wolpert, D. H. (1997). Stacked Density Estimation. NIPS, 668-674.

[63] Teixeira, A. (2004). Classification and regression tree. Revue Des Maladies Respiratoires, 21(6), 1174-1176.

[64] Ting, K. M., & Witten, I. H. (1999). Issues in stacked generalization. Journal of Artificial Intelligence Research, 10, 271-289.

[65] Tsai, C. W., Hsia, C. H., Yang, S. J., Liu, S. J., & Fang, Z. Y. (2020). Optimizing hyperparameters of deep learning in predicting bus passengers based on simulated annealing. Applied Soft Computing, 88, 106068.

[66] Wang, J., Milne, D., & Pakalnis, R. (2002). Application of a neural network in the empirical design of underground excavation spans. Transactions of the Institution of Mining and Metallurgy Section a- Mining Technology, 111(1), 73-81.

[67] Wolpert, D. H. (1992). Stacked generalization. Neural Networks, 5(2), 241-259.

[68] Wu, A., Ding, X., Chen, S., & Shi, G. (2006). Researches on deformation and failure characteristics of an underground powerhouse with complicated geological conditions by DDA method. Chinese Journal of Rock Mechanics and Engineering, 25(1), 1-8 (in Chinese).

[69] Xu, N. W., Li, T. B., Dai, F., Li, B., Zhu, Y. G., & Yang, D. S. (2015). Microseismic monitoring and stability evaluation for the large scale underground caverns at the Houziyan hydropower station in Southwest China. Engineering Geology, 188, 48-67.

[70] Ye, J., Hou, L. D., & Destech Publicat, I. (2018). Improvement and Application of Decision Tree C4.5 Algorithm. In International Conference on Computer, Communication and Network Technology (CCNT), Wuzhen, China (pp.127-133).

[71] Zhang, S. C., Li, X. L., Zong, M., Zhu, X. F., & Wang, R. L. (2018). Efficient kNN Classification With Different Numbers of Nearest Neighbors. IEEE Transactions on Neural Networks and Learning Systems, 29(5), 1774-1785.

[72] Zhang, W. G., Li, H. R., Wu, C. Z., Li, Y. Q., Liu, Z. Q., & Liu, H. L. (2021). Soft computing approach for prediction of surface settlement induced by earth pressure balance shield tunneling. Underground Space, 6(4), 353-363.

[73] Zhao, G., Wang, M., & Liang, W. (2022). A Comparative Study of SSABPNN, SSA-ENN, and SSA-SVR Models for Predicting the Thickness of an Excavation Damaged Zone around the Roadway in Rock. Mathematics, 10(8), 1351.

[74] Zhao, Q., & Higuchi, T. (1996). Evolutionary learning of nearest-neighbor MLP. IEEE Transactions on Neural Networks, 7(3), 762-767.

[75] Zhao, Y., Zhao, G., Zhou, J., Ma, J., & Cai, X. (2021). Failure mechanism analysis of rock in particle discrete element method simulation based on moment tensors. Computers and Geotechnics, 136, 104215.

[76] Zhou, J., Huang, S., & Qiu, Y. (2022). Optimization of random forest through the use of MVO, GWO and MFO in evaluating the stability of underground entry-type excavations. Tunnelling and Underground Space Technology, 124, 104494.

[77] Zhou, J., Huang, S., Tao, M., Khandelwal, M., Dai, Y., & Zhao, M. (2023). Stability prediction of underground entry-type excavations based on particle swarm optimization and gradient boosting decision tree. Underground Space, 9, 234-249.

[78] Zian, S., Kareem, S. A., & Varathan, K. D. (2021). An Empirical Evaluation of Stacked Ensembles With Different Meta-Learners in Imbalanced Classification. IEEE Access, 9, 87434-87452.