Bayesian optimization-enhanced ensemble learning for the uniaxial compressive strength prediction of natural rock and its application

Abstract

Engineering disasters, such as rockburst and collapse, are closely related to structural instability caused by insufficient bearing capacity of geological materials. Uniaxial compressive strength (UCS) holds considerable significance in rock engineering projects. Consequently, this study endeavors to devise efficient models for the expeditious and economical estimation of UCS. Using a dataset of 729 samples, including the Schmidt hammer rebound number, P-wave velocity, and point load index data, we evaluated six algorithms, namely Adaptive Boosting (AdaBoost), Gradient Boosting Decision Tree (GBDT), Extreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM), Random Forest (RF), and Extra Trees (ET) and utilized Bayesian Optimization (BO) to optimize the aforementioned algorithms. Moreover, we applied model evaluation metrics such as Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Variance Accounted For (VAF), Nash-Sutcliffe Efficiency (NSE), Weighted Mean Absolute Percentage Error (WMAPE), Coefficient of Correlation (R), and Coefficient of Determination (R²). Among the six models, BO-ET emerged as the most optimal performer during training (RMSE = 4.5042, MAE = 3.2328, VAF = 0.9898, NSE = 0.9898, WMAPE = 0.0538, R = 0.9955, R² = 0.9898) and testing (RMSE = 4.8234, MAE = 3.9737, VAF = 0.9881, NSE = 0.9875, WMAPE = 0.2515, R = 0.9940, R² = 0.9875) phases. Additionally, we conducted a systematic comparison between ensemble and traditional single machine learning models such as decision tree, support vector machine, and K-Nearest Neighbors, thus highlighting the advantages of ensemble learning. Furthermore, the enhancement effect of BO on generalization performance was assessed. Finally, a BO-ET-based Graphical User Interface (GUI) system was developed and validated in a Tunnel Boring Machine-excavated tunnel.

Keywords

Rock mechanics / Uniaxial compressive strength / Prediction model / Ensemble learning / Bayesian optimization

Author contribution

Writing-original draft: Chukwuemeka Daniel, Jamiu Ajibola Busari, Amos Izuchukwu Daniel, and Xin Yin;

Writing-review & editing: Xin Yin, Chukwuemeka Daniel, and Xing Huang;

Conceptualization: Xin Yin and Chukwuemeka Daniel;

Methodology: Xin Yin and Chukwuemeka Daniel;

Data curation: Xin Yin;

Software: Xin Yin, Chukwuemeka Daniel, Honggan Yu, Yucong Pan, and Xing Huang;

Supervision: Xin Yin and Xing Huang;

Validation: Xin Yin and Xing Huang.

Declaration of competing interest

The authors declare that they have no known competing non-financial or financial interests that could have influenced the work reported in this paper.

Acknowledgement

This research is supported by the National Natural Science Foundation of China under Grant No. 42177140 and the Key Research and Development Project of Hubei Province of China under Grant No. 2021BCA133. These supports are gratefully acknowledged.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.ghm.2024.05.002.

References

[1] R. Asheghi, A.A. Shahri, M.K. Zak, Prediction of uniaxial compressive strength of different quarried rocks using metaheuristic algorithm, Arabian J. Sci. Eng. 44 (10) (2019) 8645-8659, https://doi.org/10.1007/s13369-019-04046-8.

[2] A.E. Aladejare, Evaluation of empirical estimation of uniaxial compressive strength of rock using measurements from index and physical tests, J. Rock Mech. Geotech. Eng. 12 (2) (2020) 256-268, https://doi.org/10.1016/j.jrmge.2019.08.001.

[3] M. Rezaei, M. Asadizadeh, Predicting unconfined compressive strength of intact rock using new hybrid intelligent models, Journal of Mining and Environment 11 (1) (2020) 231-246, https://doi.org/10.22044/jme.2019.8839.1774.

[4] X. Wei, N.M. Shahani, X. Zheng, Predictive modeling of the uniaxial compressive strength of rocks using an artificial neural network approach, Mathematics 11 (7) (2023), https://doi.org/10.3390/math11071650.

[5] Z. Liu, D.J. Armaghani, P. Fakharian, D. Li, D.V. Ulrikh, N.N. Orekhova, K.M. Khedher, Rock strength estimation using several tree-based ML techniques, CMES - Computer Modeling in Engineering and Sciences 133 (3) (2022) 799-824, https://doi.org/10.32604/cmes.2022.021165.

[6] A. Mahmoodzadeh, M. Mohammadi, H. Hashim Ibrahim, S. Nariman Abdulhamid, S. Ghafoor Salim, H. Farid Hama Ali, M. Kamal Majeed, Artificial intelligence forecasting models of uniaxial compressive strength, Transportation Geotechnics 27 (September 2020) (2021) 100499, https://doi.org/10.1016/j.trgeo.2020.100499.

[7] A.E. Aladejare, V.O. Akeju, Y. Wang, Probabilistic characterization of uniaxial compressive strength of rock using test results from multiple types of punch tests, Georisk 15 (3) (2020) 209-220, https://doi.org/10.1080/ 17499518.2020.1728559.

[8] A.E. Aladejare, E.D. Alofe, M. Onifade, A.I. Lawal, T.M. Ozoji, Z.X. Zhang, Empirical estimation of uniaxial compressive strength of rock: database of simple, multiple, and artificial intelligence-based regressions, Geotech. Geol. Eng. 39 (6) (2021) 4427-4455, https://doi.org/10.1007/s10706-021-01772-5.

[9] D. Jahed Armaghani, E. Tonnizam Mohamad, M. Hajihassani, S. Yagiz, H. Motaghedi, Application of several non-linear prediction tools for estimating uniaxial compressive strength of granitic rocks and comparison of their performances, Eng. Comput. 32 (2) (2016) 189-206, https://doi.org/10.1007/ s00366-015-0410-5.

[10] D. Sun, M. Lonbani, B. Askarian, D.J. Armaghani, R. Tarinejad, B.T. Pham, V. Van Huynh, Investigating the applications of machine learning techniques to predict the rock brittleness index, Appl. Sci. 10 (5) (2020) 1-17, https://doi.org/ 10.3390/app10051691.

[11] X. Yin, Q. Liu, Y. Pan, X. Huang, J. Wu, X. Wang, Strength of stacking technique of ensemble learning in rockburst prediction with imbalanced data: comparison of eight single and ensemble models, Natural Resources Research 30 (2) (2021) 1795-1815, https://doi.org/10.1007/s11053-020-09787-0.

[12] E.E. Meybodi, S.K. Hussain, M.F. Marji, V. Rasouli, Application of machine learning models for predicting rock fracture toughness mode-I and mode-II, Journal of Mining and Environment 13 (2) (2022) 467-482, https://doi.org/10.22044/ jme.2022.11596.2148.

[13] Y.T. Wang, X. Zhang, X.S. Liu, Machine learning approaches to rock fracture mechanics problems: mode-I fracture toughness determination, Eng. Fract. Mech. 253 (March) ( 2021) 107890, https://doi.org/10.1016/ j.engfracmech.2021.107890.

[14] S.A. Dantas Neto, B. Indraratna, D.A.F. Oliveira, A.P. de Assis, Modelling the shear behaviour of clean rock discontinuities using artificial neural networks, Rock Mech. Rock Eng. 50 (7) (2017) 1817-1831, https://doi.org/10.1007/s00603-017-1197-z.

[15] S. Zhou, Y. Lei, Z.X. Zhang, X. Luo, A. Aladejare, T. Ozoji, Estimating dynamic compressive strength of rock subjected to freeze-thaw weathering by data-driven models and non-destructive rock properties, Nondestr. Test. Eval. (2024) 1-24, https://doi.org/10.1080/10589759.2024.2313569.

[16] A.E. Aladejare, V.O. Akeju, Y. Wang, Data-driven characterization of the correlation between uniaxial compressive strength and Youngs' modulus of rock without regression models, Transportation Geotechnics 32 (May 2021) (2022) 100680, https://doi.org/10.1016/j.trgeo.2021.100680.

[17] V.C. Moussas, K. Diamantis, Predicting uniaxial compressive strength of serpentinites through physical, dynamic and mechanical properties using neural networks, J. Rock Mech. Geotech. Eng. 13 (1) (2021) 167-175, https://doi.org/ 10.1016/j.jrmge.2020.10.001.

[18] M.Y. Hassan, H. Arman, Several machine learning techniques comparison for the prediction of the uniaxial compressive strength of carbonate rocks, Sci. Rep. 12 (1) (2022) 1-17, https://doi.org/10.1038/s41598-022-25633-0.

[19] M. Salda-na, J. Gonz_alez I. P_erez-Rey, M. Jeldres, N. Toro, Applying statistical analysis and machine learning for modeling the UCS from P-wave velocity, density and porosity on dry travertine, Appl. Sci. 10 (13) (2020), https://doi.org/10.3390/ app10134565.

[20] M. Azarafza, M.H. Bonab, R. Derakhshani, A deep learning method for the prediction of the index mechanical properties and strength parameters of marlstone, Materials 15 (19) (2022), https://doi.org/10.3390/ma15196899.

[21] D.J. Armaghani, V. Safari, A. Fahimifar, M.F. Mohd Amin, M. Monjezi, M.A. Mohammadi, Uniaxial compressive strength prediction through a new technique based on gene expression programming, Neural Comput. Appl. 30 (11) (2017) 3523-3532, https://doi.org/10.1007/s00521-017-2939-2.

[22] _I. _Ince, A. Bozda_g, M. Fener, S. Kahraman, Estimation of uniaxial compressive strength of pyroclastic rocks (Cappadocia, Turkey) by gene expression programming, Arabian J. Geosci. 12 (24) (2019), https://doi.org/10.1007/s12517- 019-4953-4.

[23] L.O. Afolagboye, D.E. Ajayi, I.O. Afolabi, Machine learning models for predicting unconfined compressive strength: a case study for Precambrian basement complex rocks from Ado-Ekiti, Southwestern Nigeria, Scientific African 20 (2023), https:// doi.org/10.1016/j.sciaf.2023.e01715.

[24] A.F. Ibrahim, M. Hiba, S. Elkatatny, A. Ali, Estimation of tensile and uniaxial compressive strength of carbonate rocks from well-logging data: artificial intelligence approach, J. Pet. Explor. Prod. Technol. 14 (1) (2023) 317-329, https://doi.org/10.1007/s13202-023-01707-1.

[25] Y. Wang, M. Hasanipanah, A.S.A. Rashid, B.N. Le, D.V. Ulrikh, Advanced tree-based techniques for predicting unconfined compressive strength of rock material employing non-destructive and petrographic tests, Materials 16 (10) (2023), https://doi.org/10.3390/ma16103731.

[26] M. Wang, W. Wan, Y. Zhao, Prediction of the uniaxial compressive strength of rocks from simple index tests using a random forest predictive model, Compt. Rendus Mec. 348 (1) (2020) 3-32, https://doi.org/10.5802/CRMECA.3.

[27] H. Nasiri, A. Homafar, S.C. Chelgani, Prediction of uniaxial compressive strength and modulus of elasticity for Travertine samples using an explainable artificial intelligence, Results in Geophysical Sciences 8 (December) ( 2021) 100034, https:// doi.org/10.1016/j.ringps.2021.100034.

[28] N.M. Shahani, M. Kamran, X. Zheng, C. Liu, X. Guo, Application of gradient boosting machine learning algorithms to predict uniaxial compressive strength of soft sedimentary rocks at Thar coalfield, Adv. Civ. Eng. 2021 ( 2021), https:// doi.org/10.1155/2021/2565488.

[29] M. Abdelhedi, R. Jabbar, A.B. Said, N. Fetais, C. Abbes, Machine learning for prediction of the uniaxial compressive strength within carbonate rocks, Earth Science Informatics 16 (2) (2023) 1473-1487, https://doi.org/10.1007/s12145- 023-00979-9.

[30] N.M. Shahani, X. Zheng, Predictive modeling of uniaxial compressive strength of rocks for protecting environment using artificial neural network, PREPRINT (Version 1) Available at Research Square (2021) 1-20, https://doi.org/10.21203/ rs.3.rs-1078570/v1.

[31] B. Xu, Y. Tan, W. Sun, T. Ma, H. Liu, D. Wang, Study on the prediction of the uniaxial compressive strength of rock based on the SSA-xgboost Model, Sustainability 15 (6) (2023) 1-17, https://doi.org/10.3390/su15065201.

[32] J. Qiu, X. Yin, Y. Pan, X. Wang, M. Zhang, Prediction of uniaxial compressive strength in rocks based on extreme learning machine improved with metaheuristic algorithm, Mathematics 10 (19) (2022), https://doi.org/10.3390/math10193490.

[33] H. Fattahi, A new method for forecasting uniaxial compressive strength of weak rocks, Journal of Mining and Environment 11 (2) (2020) 505-515, https://doi.org/ 10.22044/jme.2020.9328.1835.

[34] J. Yin, N. Li, Ensemble learning models with a Bayesian optimization algorithm for mineral prospectivity mapping, Ore Geol. Rev. 145 (February) ( 2022) 104916, https://doi.org/10.1016/j.oregeorev.2022.104916.

[35] X. Yin, X. Huang, Y. Pan, Q. Liu, Point and interval estimation of rock mass boreability for tunnel boring machine using an improved attribute-weighted deep belief network, Acta Geotechnica 18 (4) (2023) 1769-1791, https://doi.org/ 10.1007/s11440-022-01651-0.

[36] D.J. Armaghani, E. Tonnizam Mohamad, E. Momeni, M. Monjezi, M. Sundaram Narayanasamy, Prediction of the strength and elasticity modulus of granite through an expert artificial neural network, Arabian J. Geosci. 9 (1) (2016) 1-16, https:// doi.org/10.1007/s12517-015-2057-3.

[37] E. Momeni, D. Jahed Armaghani, M. Hajihassani, M.F. Mohd Amin, Prediction of uniaxial compressive strength of rock samples using hybrid particle swarm optimization-based artificial neural networks, Measurement: Journal of the International Measurement Confederation 60 ( 2015) 50-63, https://doi.org/ 10.1016/j.measurement.2014.09.075.

[38] I.T. Ng, K.V. Yuen, C.H. Lau, Predictive model for uniaxial compressive strength for Grade III granitic rocks from Macao, Eng. Geol. 199 (2015) 28-37, https://doi.org/ 10.1016/j.enggeo.2015.10.008.

[39] D.A. Mishra, A. Basu, Estimation of uniaxial compressive strength of rock materials by index tests using regression analysis and fuzzy inference system, Eng. Geol. 160 (2013) 54-68, https://doi.org/10.1016/j.enggeo.2013.04.004.

[40] R.S. Tandon, V. Gupta, Estimation of strength characteristics of different Himalayan rocks from Schmidt hammer rebound, point load index, and compressional wave velocity, Bull. Eng. Geol. Environ. 74 (2) (2015) 521-533, https://doi.org/ 10.1007/s10064-014-0629-1.

[41] S. Dehghan, G. Sattari, C.S. Chehreh, M.A. Aliabadi, Prediction of uniaxial compressive strength and modulus of elasticity for Travertine samples using regression and artificial neural networks, Min. Sci. Technol. 20 (1) (2010) 41-46, https://doi.org/10.1016/S1674-5264(09)60158-7.

[42] M. Heidari, H. Mohseni, S.H. Jalali, Prediction of uniaxial compressive strength of some sedimentary rocks by fuzzy and regression models, Geotech. Geol. Eng. 36 (1) (2017) 401-412, https://doi.org/10.1007/s10706-017-0334-5.

[43] I. Dinçer, A. Acar, S. Ural, Estimation of strength and deformation properties of Quaternary caliche deposits, Bull. Eng. Geol. Environ. 67 (3) (2008) 353-366, https://doi.org/10.1007/s10064-008-0146-1.

[44] S. Kahraman, Evaluation of simple methods for assessing the uniaxial compressive strength of rock, Int. J. Rock Mech. Min. Sci. 38 (7) (2001) 981-994, https:// doi.org/10.1016/S1365-1609(01)00039-9.

[45] M. Karakus, B. Tutmez, Fuzzy and multiple regression modelling for evaluation of intact rock strength based on point load, schmidt hammer and sonic velocity, Rock Mech. Rock Eng. 39 (1) (2006) 45-57, https://doi.org/10.1007/s00603-005-0050- y.

[46] A. Kılıç, A. Teymen, Determination of mechanical properties of rocks using simple methods, Bull. Eng. Geol. Environ. 67 (2) (2008) 237-244, https://doi.org/ 10.1007/s10064-008-0128-3.

[47] A. Tuǧrul, I.H. Zarif, Correlation of mineralogical and textural characteristics with engineering properties of selected granitic rocks from Turkey, Eng. Geol. 51 (4) (1999) 303-317, https://doi.org/10.1016/S0013-7952(98)00071-4.

[48] M. Brydon, Correlation and scatterplots.

[49] Scikit-learn, Ensembles: gradient boosting, random forests, bagging, voting, stacking.

[50] R. Andreoni, Ensemble learning with scikit-learn: a friendly introduction. https ://towardsdatascience.com/ensemble-learning-with-scikit-learn-a-friendly-introd uction-5dd64650de6c, 2023.

[51] J. Brownlee, Ensemble machine learning algorithms in python with scikit-learn, Python Machine Learning (2020).

[52] S. Bhaskar, Machine learning-adaboost using scikit-learn.

[53] N. Arya,Implementing adaboost in scikit-learn, Mach. Learn. (2022). htt ps:// www.kdnuggets.com/2022/10/implementing-adaboost-scikitlearn.html.

[54] Scikit-learn, An Adaboost Regressor, 2007.

[55] T. Masui, All you need to know about gradient boosting algorithm _ part 1. regression, Data Sci. (2022).

[56] J. Brownlee, Gradient boosting with scikit-learn, xgboost, lightgbm, and catboost, Ensemble Learning (2021).

[57] C. Wade, Getting started with xgboost in scikit-learn, Data Sci. (2020).

[58] B. Wasike, Dive into xgboost and scikit-learn: machine learning with xgboost and scikit-learn.

[59] A. Mondal, Complete guide on how to use lightgbm in python, Data Science Blogathon (2023).

[60] Scikit-learn, A Random Forest Regressor, 2007.

[61] A. Dutta, Random forest regression in python.

[62] J. Brownlee, How to develop an extra trees ensemble with python, Ensemble Learning (2021).

[63] T. Huijskens, Bayesian optimization with scikit-learn.

[64] J. Czakon, Scikit optimize: Bayesian hyperparameter optimization in python, ML Tools (2023).

[65] I. Çobanoǧlu, S.B. Çelik, Estimation of uniaxial compressive strength from point load strength, Schmidt hardness and P-wave velocity, Bull. Eng. Geol. Environ. 67 (4) (2008) 491-498, https://doi.org/10.1007/s10064-008-0158-x.