Prediction of bearing capacity of pile foundation using deep learning approaches

Abstract

The accurate prediction of bearing capacity is crucial in ensuring the structural integrity and safety of pile foundations. This research compares the Deep Neural Networks (DNN), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), and Bidirectional LSTM (BiLSTM) algorithms utilizing a data set of 257 dynamic pile load tests for the first time. Also, this research illustrates the multicollinearity effect on DNN, CNN, RNN, LSTM, and BiLSTM models’ performance and accuracy for the first time. A comprehensive comparative analysis is conducted, employing various statistical performance parameters, rank analysis, and error matrix to evaluate the performance of these models. The performance is further validated using external validation, and visual interpretation is provided using the regression error characteristics (REC) curve and Taylor diagram. Results from the comparative analysis reveal that the DNN (Coefficient of determination (R²)_{training (TR)} = 0.97, root mean squared error (RMSE)_TR = 0.0413; R²_{testing (TS)} = 0.9, RMSE_TS = 0.08) followed by BiLSTM (R²_TR = 0.91, RMSE_TR = 0.782; R²_TS = 0.89, RMSE_TS = 0.0862) model demonstrates the highest performance accuracy. It is noted that the BiLSTM model is better than LSTM because the BiLSTM model, which increases the amount of information for the network, is a sequence processing model made up of two LSTMs, one of which takes the input in a forward manner, and the other in a backward direction. The prediction of pile-bearing capacity is strongly influenced by ram weight (having a considerable multicollinearity level), and the effect of the considerable multicollinearity level has been determined for the model based on the recurrent neural network approach. In this study, the recurrent neural network model has the least performance and accuracy in predicting the pile-bearing capacity.

Graphical abstract

Keywords

deep learning algorithms / high-strain dynamic pile test / bearing capacity of the pile

1 Introduction

Pile foundations are a prevalent choice in the construction industry due to their ability to support the load of the superstructure and transfer it to the underlying soil or bedrock. In recent decades, an abundance of experimental, computational, and analytical methodologies have been proposed for evaluating the bearing capacity of a pile. Since consumer confidence in these models is directly tied to their reliability, they must provide accurate value predictions. Determining the ultimate capacity (Q_ult) through applying the Meyerhof formula may exhibit variations compared to the results obtained from the field tests. The observed disparities may be attributed to the fundamental simplifying assumptions concerning the problem. The assessment of pile-bearing capacity is commonly conducted through two procedures: the high-strain dynamic load test (HSDT) and the static load test (SLT). The technological apparatus employed in HSDT is advanced, exhibiting superior speed and cost-effectiveness compared to the equipment utilized in SLT. The HSDT operates on the one-dimensional wave propagation theory principle and is executed using a pile drive analyzer (PDA). Currently, the HSDT testing protocol confirms the rigorous criteria of the American Standard Testing Procedures. The dynamic load test has emerged as a prevalent method for assessing the capacity and integrity of driven and cast-in situ piles. The dynamic pile capacity and the static load carrying capacity are often in satisfactory agreement [1]. This testing method’s primary benefits include its cost-effectiveness compared to traditional SLT and its relatively brief testing period.

Since conducting a PDA test entails significant expenses and consumes considerable time, minimizing the test administration frequency could reduce the project’s overall expenditure. Consequently, various design techniques have been proposed to approximate the load-bearing capacity of piles. Multiple approaches have been developed over the years to address this issue, ranging from basic empirical models [2–7] to intricate finite element analyses [6,8–10]. The empirical models, however, failed to capture the complex pile-soil interaction [11]. The past studies show that negative values of the coefficient of correlation and extremely high values of the root mean squared error (RMSE) occur when using empirical relations to make predictions due to poor data fitting and strong disagreement between input and predictor variables [12–14]. Further advanced methodologies are required to guarantee a suitable structural and serviceability efficacy due to the complex relationship of the associated factors, including but not limited to soil stratifications, soil-pile interaction, and allocation of soil resistance along the pile shaft. The utilization of computational models has the potential to facilitate the computation of a pile’s bearing capacity.

Over the past decade, much research has focused on using artificial intelligence (AI) to solve geotechnical engineering problems. Scholarly articles show significant advances in neural networks. Frank Rosenblatt's 1940s and 1950s perceptron model laid the groundwork for future advances. The backpropagation technique made multilayer perceptron networks with several hidden layers trainable in the 1980s, revolutionizing neural network research. Radial basis function networks were studied in the 1990s. In the 2000s, recurrent neural networks (RNNs) and long short-term memory (LSTM) revolutionized natural language processing, speech recognition, and other sequential data applications. In the 2010s, multilayer deep learning (DL) models dominated DL and convolutional neural networks (CNNs). Tran et al. [15], Dang et al. [16], Wang et al. [17], and Ho et al. [18] solved different engineering problems using advanced computational models. Samaniego et al. [19] solved the partial differential equation using computational mechanics and machine learning.

Recent investigations have used in situ tests and artificial neural networks (ANNs) to forecast driven pile capacity [20–22]. ANNs have a lot of advantages, but they also have several limitations and flaws that need to be carefully considered by academics and practitioners. The factors mentioned above are black-box nature, overfitting, computational complexity, the need for large training data sets, hyperparameter sensitivity, the lack of causation, and data needs. Black-box models can overfit due to their lack of transparency and interpretability. Their computational complexity is significant, and training requires big data sets, which can be time-consuming, expensive, or impracticable [23–25].

Additionally, adjusting their hyperparameters can be difficult, and they are not necessarily good at capturing causation. In this research paper, we investigate the applications and effectiveness of four key DL architectures: Deep Neural Networks (DNN), CNN, RNN, and LSTM, along with their variant, Bidirectional Long Short-Term Memory (BiLSTM); to predict the bearing capacity of the pile. DNNs are multiple layers of interconnected nodes that learn hierarchical representations of data. DNNs have remarkably succeeded in predicting pile capacity [26–28]. Guo et al. [29–32] employed the computational approaches (DNN and stochastic method) to analyze 1) bending analysis of Kirchhoff plate, 2) melting heat transfer, and 3) heterogeneous porous media. Zhuang et al. [33] used a deep autoencoder (transfer learning-based) to analyze Kirchhoff plates’ buckling, bending, and vibration. CNNs are a type of DL model consisting of convolutional, pooling, and fully connected layers. CNN has achieved significant success in predicting the lateral capacity of monopiles [34] and undrained bearing capacity of skirted foundation [35], it has minimal application in foundation engineering and, so far, has not been used for prediction of pile bearing capacity. RNNs are a type of neural network. Unlike feedforward neural networks, which process inputs independently, RNNs maintain an internal state that allows them to capture information from previous inputs in the sequence. RNNs have proven reliable for predicting pile capacity [36,37]. However, it has not been employed for pile load test data sets. LSTM is an extension of RNN architecture that has successfully mitigated the vanishing gradient problem. It introduces specialized memory cells and gating mechanisms that allow the network to control the flow of information, selectively remembering or forgetting information over long sequences. Though LSTM has not been used to simulate pile capacity field tests, it has been proven state-of-the-art in numerous civil engineering applications [38–40]. BiLSTM extends the capabilities of LSTM by simultaneously considering both past and future contexts. By processing sequences bidirectionally, BiLSTM captures dependencies in both directions, providing a more comprehensive understanding of the behavior of pile foundations. This architecture helps predict complex interactions between soil properties, loading conditions, and long-term foundation performance. BiLSTM has revolutionized the DL applications research domain [28,41,42]. However, it's worth noting that the method is yet to have application in foundation engineering to predict bearing capacity. Zhang and Yin [28] used the BiLSTM to model the behavior of the soil structure interface, while Tao et al. [41] utilized the model to simulate the mapping between soil parameters and deflection responses. The obtained model was robust for predicting shield thrust for tunnelling [43] and landslide susceptibility [44]. An overview of the literature survey reveals very scars but encouraging application of BiLSTM in civil engineering applications. It’s judicious to use the technique to model HSDT predictions. Overall, integrating DL architectures in the analysis of pile foundations holds great promise for enhancing our understanding of complex geotechnical phenomena and improving the design, assessment, and maintenance of these critical infrastructure components. By leveraging the power of machine learning, we can unlock new insights and improve the reliability and safety of pile foundations, ultimately contributing to advancing civil engineering practices. Tab.1 shows the details of published models.

1.1 Gap identification

The published works represent the capabilities of the computational models in assessing the pile-bearing capacity. Benbouras et al. [26] employed conventional, machine, and DL models to assess the bearing capacity of steel and concrete piles using 100 data sets. The researchers conducted the reported work using a static pile test database of steel and concrete piles. However, research on utilizing machine learning in simulating dynamic pile load testing is scarce. The existing literature on this research indicates that the published studies have either used the different limited databases, which raises concerns regarding the reliability of the predictions or have employed rudimentary machine learning models with not good performance and accuracy. Because of using different databases, the optimum performance soft computing model is still unidentified. Also, the multicollinearity effect has not been analyzed for the performance and accuracy of computational models.

1.2 Objectives of the current work

The literature gap depicts that many researchers used different computational models and databases to assess pile-bearing capacity. These researchers compared conventional, machine, and DL models in predicting pile-bearing capacity. Based on the literature analysis, the following objectives for this research have been mapped:

1) This work compares the deep and hybrid computational models using a dynamic pile load test database to introduce a robust computational model for predicting the pile-bearing capacity.

2) The current investigation employs and analyzes the DNN, CNN, RNN, LSTM, and BiLSTM models in predicting the pile-bearing capacity for the first time using a common database.

3) This work illustrates the multicollinearity effect on DNN, CNN, RNN, LSTM, and BiLSTM models’ performance and accuracy in predicting pile-bearing capacity.

1.3 Motivation of the work

DL techniques have emerged as powerful tools for analyzing and predicting complex geotechnical phenomena. They offer advantages over traditional methods like empirical approaches, in situ testing, numerical modeling, etc. DL models can learn from historical data and identify hidden patterns, leading to more reliable predictions. Most neural network models in the literature were either constructed with a small data set or provided poor values of performance parameters for modeling dynamic pile load tests. The literature reveals that DNN, CNN, RNN, LSTM, and BiLSTM have very limited applications in foundation engineering and have not been used so far for pile-bearing capacity prediction. This research investigates the applications and advantages of these DL architectures in analyzing and predicting pile foundation behavior. By harnessing the capabilities of DNN, CNN, RNN, LSTM, and BiLSTM, we aim to improve the accuracy and efficiency of predicting pile capacity, settlement, and performance and propose a state-of-the-art new methodology for predicting the bearing capacity of pile foundations. Additionally, using soft computing techniques, this research will assist geotechnical engineers and designers in determining the pile-bearing capacity.

2 Research methodology

The current work introduces an optimum-performance soft computing model to predict the pile-bearing capacity by comparing performance and accuracy. For that purpose, models based on DNN, CNN, RNN, LSTM, and BiLSTM soft computing approaches have been constructed and analyzed. A database has been compiled from 257 results of pile-bearing capacity. The multicollinearity and hypothesis analyses have been performed to check the quality of the database. Furthermore, the training (TR) and testing (TS) databases have been created by arbitrarily selecting 193 and 64 data points, respectively. Willmott’s index of agreement (WI), Performance index (PI), Variance accounted factor (VAF), Weighted mean absolute percentage error (WMAPE), RMSE, Mean absolute error (MAE), and Coefficient of determination (R²) performance metrics have been implemented to measure the TR and TS performance of the employed models. The rank analysis, error matrix, Taylor plot, regression error characteristics (REC) curve, and anderson-darling (AD) test have been performed to identify the optimum performance soft computing model to assess the pile-bearing capacity. In addition, external and literature validations have been carried out to ensure the robustness of the optimum performance model. Fig.1 illustrates the methodology of the present research.

3 Data analysis and computing models

3.1 Data analysis

The present research has been conducted to introduce a robust soft computing model for predicting pile capacity. For this purpose, a database has been compiled using 257 pile data sets. The database consists of diameter (D in m), length (L in m), pile set (PS), ram weight (W in kN), drop height (DH in m), and pile capacity (C in kN). The independent variables, D, L, PS, W, and DH, have been used as input parameters for predicting C. Fig.2 demonstrates the relationship between input and output variables.

The correlation coefficient values ± 1.0 to ± 0.81, ± 0.80 to ± 0.61, ± 0.60 to ± 0.41, ± 0.40 to ± 0.21, ± 0.20 to ± 0.0 show very strong, strong, moderate, weak, and no relationship between variables, respectively [52,53]. Fig.2 demonstrates that 1) L and PS have a weak relationship with depth; 2) rammer weight and C have a moderate relationship with depth; 3) the DH has no relationship with depth of pile; 4) L of the pile has no relationship with PS and C; 5) L of the pile has a weak relationship with rammer weight; 6) rammer weight has a strong relationship with DH and C of piles; and 7) DH has a moderate relationship with C of the pile. Furthermore, Fig.2 illustrates that the input variables have less multicollinearity because of the poor relationship between variables. Fig.3 reveals the strength of the input variable in assessing the C.

Fig.3 reveals that rammer weight has the highest strength with the C, followed by the pile depth, DH, L, and PSs. In addition, the nonlinear relationship between the variables has been determined using the distance correlation coefficient method. The distance correlation or covariance measures dependence between two paired random vectors of arbitrary, not necessarily equal, dimension. Fig.4 illustrates the distance correlation among the variables. Fig.4 illustrates that 1) D very strongly correlates with W and C than L, PS, and DH; 2) L is very strongly correlated with W and DH; 3) L is strongly correlated with PS and C; 4) PS strongly correlates with W, DH, and C; 5) W very strongly correlates with DH and C; and 6) DH very strongly correlates with C. It is noted that the D, W, and DH variables are important parameters for predicting the bearing capacity of piles.

Considering the strength of independent variables with dependent variables and nonlinear correlation, the frequency of the variables has been estimated and depicted in Fig.5. Also, descriptive statistics have been drawn for the overall database, as shown in Tab.2.

The 193 and 64 data points have been used as training and testing data points for employing, training, testing, and analyzing purposes. The hypothesis and multicollinearity analyses have been performed to check the quality of the overall database.

3.1.1 Hypothesis analysis

Considering the statements drawn from the correlation coefficient results, the following research hypothesis statements have been mapped: 1) the Ds of the pile and rammer weight are significant parameters for assessing C, 2) the D and L of the pile and PS have problematic multicollinearity. ANOVA and Z tests have been performed in this study for the hypothesis analysis. The ANOVA test results are given in Tab.3.

Tab.3 demonstrates that this research rejects the null hypothesis. In the case of D, L, W, PS, and DH, the F is higher than the F crit. Also, the p-value (calculated) is determined as less than the significance level, i.e., 0.05. Furthermore, the Z test results are reported in Tab.4.

The Z test results show that the current study rejects the null hypothesis since the Z critical two-tail value is greater than the Z critical one-tail value. Additionally, data points below the average are shown by the negative z value. i.e., −19.49 (for D), −24.97 (for L), −25.25 (for PS), −24.65 (for W), and −25.32 (for DH).

3.1.2 Multicollinearity analysis

It is a phenomenon that occurs while performing regression analysis. The database multicollinearity can be determined by performing Pearson product-moment correlation coefficient and variance inflation factor (VIF). Khatti and Grover [52,53] have proposed five multicollinearity levels using VIF values: 1) no multicollinearity (VIF = 0), 2) weak multicollinearity (0 < VIF≤ 2.5), 3) considerable multicollinearity (2.5 < VIF≤ 5), 4) moderate multicollinearity (5 < VIF≤ 10), and 5) problematic multicollinearity (10 < VIF). The multicollinearity results are summarized in Tab.5. It reveals that D, L, PS, and DH parameters have weak multicollinearity. Still, the W has considerable multicollinearity in predicting the C. Finally, the database used in the current work has weak to considerable multicollinearity, which doesn’t impact the performance and accuracy of models. Based on the multicollinearity results, it can be said the database used in the work is an excellent quality database.

3.2 Soft computing approaches

The models based on DNNs, CNNs, RNNs, LSTM, and BiLSTM have been developed to predict C in the present study. DNNs are general neural network architectures with multiple layers, CNNs are specialized for processing visual data, RNNs are designed for sequential data processing, LSTMs address long-term dependency issues in RNNs, and BiLSTMs combine information from past and future contexts. A brief description of adopted soft computing approaches is given.

3.2.1 Deep neural networks

It is an artificial neural network consisting of input, output, and hidden layers of interconnected neurons known as neurons. These networks are inspired by the human brain. The significant advantage of DNNs is their ability to automatically learn hierarchical representations of databases. DNNs are highly suited for tasks because of their ability to understand complex patterns and relationships in the data.

3.2.2 Conventional neural networks

CNN is a class of DNNs designed explicitly for analyzing visual data points. It is highly capable of automatically learning and extracting relevant features from a visual database through a process known as convolution. The CNN approach is being used to solve complex regression problems. A CNN network consists of a convolutional layer, pooling layer, activation function, and fully connected layers. However, the CNN regression model has been employed in this study to solve the regression problem based on C.

3.2.3 Recurrent neural networks

RNNs are designed to solve the problems associated with time series data or natural language. A recurrent neural network is based on feedback connections that allow it to maintain an internal memory or context, making it capable of capturing temporal dependencies and patterns in sequential data, and this is the significant advantage of RNN over feedforward neural networks. The components of recurrent neural networks are the recurrent neurons, time unfolding, long short-term memory and gated recurrent units (GRU), and bidirectional RNNs.

3.2.4 Long short-term memory

LSTM is a type of recurrent neural network class that addresses the vanishing gradient problem and is capable of capturing long-term dependencies in sequential data. It is specifically designed to overcome the limitations of traditional RNNs in preserving and utilizing information over long sequences. The LSTM network introduces a memory cell that retains and propagates information across different time steps. A typical LSTM network consists of input, forget, and output gates. Several published works using LSTM have revealed the superiority of long short-term memory models over other neural networks.

3.2.5 Bidirectional long short-term memory

BiLSTM is an advanced LSTM approach combining information from past and future time loops to make predictions or classifications. It consists of two separate LSTM networks: 1) processing the input sequences in the forward direction and 2) in a backward direction. These networks improve the understanding of sequential data. The forward LSTM, backward LSTM, and concatenation are the cells of a BiLSTM.

3.3 Performance evaluation

To measure the performance of the proposed model, various performance indices, namely, R², RMSE, WMAPE, VAF, PI, and WI, were utilized in the study. The performance parameters are expressed as follows:

R²

(1)R^2= \frac{\sum _{i=1}^n{\left(d_i-d_{\mathrm{a}\mathrm{v}\mathrm{g}}\right)}^2-\sum _{i=1}^n{\left(d_i-y_i\right)}^2}{\sum _{i=1}^n{\left(d_i-d_{\mathrm{a}\mathrm{v}\mathrm{g}}\right)}^2},

RMSE

(2)RMSE= \sqrt{\frac{1}{N}\sum _{i=1}^n{\left(d_i-y_i\right)}^2},

WMAPE

(3)WMAPE=\frac{\sum _{i=1}^n\left|{\displaystyle \frac{d_i-y_i}{d_i}}\right|\times d_i}{\sum _{i=1}^n{d}_i},

VAF

(4)VAF=\left(1-\frac{\mathrm{v}\mathrm{a}\mathrm{r}\left(d_i-y_i\right)}{\mathrm{v}\mathrm{a}\mathrm{r}\left(d_i\right)}\right)\times 100,

PI

(5)PI=adj.R^2+\left(0.01\times VAF\right)-RMSE,

WI

(6)WI=1- \left[\frac{\sum _{i=1}^n{\left(d_i-y_i\right)}^2}{\sum _{i=1}^n{\left\{\left|y_i-d_{\mathrm{a}\mathrm{v}\mathrm{g}}\right|+\left|d_i-d_{\mathrm{a}\mathrm{v}\mathrm{g}}\right|\right\}}^2}\right],

where, d_i is actual value, d_{\mathrm{a}\mathrm{v}\mathrm{g}} is average actual value, and y_i is a predicted value. The ideal R², RMSE, WMAPE, VAF, PI, and WI values are 1, 0, 0, 100, 2, and 1, respectively.

3.4 Sensitivity analysis

The sensitivity analysis (SA) is useful tool to find the most influencing input variables in the prediction or forecasting. Linear and nonlinear are the types of SA. This work uses global cosine amplitude sensitivity analysis to find the most influencing parameters in assessing pile-bearing capacity. The mathematical formulation of cosine amplitude sensitivity analysis is as follows:

(7)SS=\frac{{\sum }_{c=1}^n\left(X_{ic}\ast X_{jk}\right)}{\sqrt{{\sum }_{c=1}^n{X}_{ic}^2}{\sum }_{c=1}^n{X}_{jk}^2},

where X_{ic} is input parameters D, L, PS, W, DH, and X_{jk} is output parameter pile-bearing capacity. SS’s value near one indicates that the independent variable strongly influences the output. The results obtained using Eq. (7) are graphically illustrated in Fig.6. It illustrates that input variables D and W are highly sensitive in predicting pile-bearing capacity, followed by L, PS, and DH.

4 Results and discussion

4.1 Simulation of soft computing models

In the present study, different types of neural network models have been developed, learned, and analyzed in assessing pile-bearing capacity. The neural network is a network configured by the following hyperparameters: 1) layers (hidden, input, and output), 2) batch size, 3) activation functions, 4) epochs, 5 dropout rate, 6) learning rate, 7) loss function, and 8) optimizer. All developed models are coded in Python 3.8, realized based on Spider and executed on Intel (R) Core (TM) i7-4790 CPU @ 3.60 GHz and AMD RX580 8G GPU. Tab.6 lists the configurations of the soft computing models developed for this study.

Using the configurations mentioned in Tab.6, the DNN, CNN, RNN, LSTM, and BiLSTM soft computing models have been constructed, trained, and tested. The performance obtained from the training and testing phases has been summarized in Tab.7 and graphically presented in Fig.7. Tab.7 demonstrates that all models attain over 90% (R² = 0.9) accuracy in the TR phase. Comparing performance and accuracy, it is noted that model DNN has higher performance, i.e., R² = 0.9034, VAF = 89.7163%, PI = 1.7102, WI = 0.9697, WMAPE = 0.1971 kN, and RMSE = 0.0820 kN, in the TS phase. It can be seen that model DNN has predicted pile-bearing capacity with R² = 0.9753, VAF = 97.5269%, PI = 1.9086, WI = 0.9933, WMAPE = 0.0799 kN, and RMSE = 0.0413 kN, better than other soft computing models, in the TR phase. The DNN models’ performance is close to the ideal values, demonstrating the reliability of the DNN model in predicting the pile-bearing capacity. Hence, the model DNN is the best architectural model for assessing pile-load bearing.

The regression plot between the actual and predicted pile-bearing capacity using models is shown in Fig.7. Fig.7 (a) depicts that most predicted pile-bearing capacity values have been lying near the 1:1 line, presenting the better prediction of model DNN. A soft computing model is good and acceptable if it achieves a correlation coefficient of more than 0.8 [54,55]. Hence, model DNN is acceptable as the best architectural model in this study.

4.2 Visual interpretation of results

REC curve, Taylor plot, error matrix, and rank analysis have all been used to help interpret the data and choose the optimal architectural model. To further guarantee the capabilities of the used models, the AD test has been run on the entire actual and predicted database.

4.2.1 Rank analysis

The rank analysis is known as rank-based model selection used for selecting the robust soft computing model. It involves ranking the models based on their performance indices and selecting the model with the highest rank as the best choice. The model with the best performance indices gets the highest rank, while the model with the worst performance indices gets the lowest rank. The total rank obtained for any particular model is by adding the training and testing rank separately using the following mathematical Eq. (8).

(8)\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{s}\mathrm{c}\mathrm{o} \mathrm{r}\mathrm{e}={\sum }_{i=1}^n{R}_{i\mathrm{T}\mathrm{R}}+{\sum }_{i=1}^n{R}_{i\mathrm{T}\mathrm{S}},

where R_{i\mathrm{T}\mathrm{R}} and R_{i\mathrm{T}\mathrm{S}} represent the rank corresponding to individual performance indices for the TR and TS phase, respectively, and n represent the number of performance indices used in the rank analysis. The results of rank analysis for all employed models for both the TR and TS phases are presented in Tab.8. Analyze the rankings to identify the model with the highest rank that is considered the best architectural model according to the chosen performance indices. In this study, the DNN model achieved a total rank of 30 in both the training and testing phase, followed by BiLSTM (TR = 22, TS = 24), LSTM (TR = 11, TS = 17), RNN (TR = 18, TS = 9), and CNN (TR = 9, TS = 10). The grand rank total of DNN, CNN, RNN, LSTM, and BiLSTM models are graphically presented in Fig.8, and it demonstrates that model DNN has secured the first rank over other computational models in assessing the pile-bearing capacity.

4.2.2 Error matrix

A regression model’s performance evaluation can be conveniently summarized in a tabular form known as a confusion, error, or accuracy matrix. This matrix provides detailed information regarding the model’s correct and incorrect predictions for each model. By utilizing an accuracy matrix, one can gain a deeper understanding of the accuracy of a regression model, identify specific errors, and make more informed decisions. The matrix analysis is a valuable tool for assessing the effectiveness of soft computing models and fine-tuning them to achieve optimal performance. The TR and TS error matrix for each model employed in the present study is graphically presented in Fig.9.

Fig.9 reveals that the errors in the testing phase are lowest for DNN (10% for R² and 8% for RMSE) and highest for RNN (21% for R² and 12% for RMSE). The highest error received is 28% for the PI of the RNN model, followed by 26% for the PI and WMAPE of CNN. The LSTM has a 24% error for PI and WMAPE. Thus, DNN and BiLSTM are the models with robust accuracy, while others are lagging. The analysis also highlights the importance of examining the model through various performance parameters.

4.2.3 Taylor diagram

A Taylor diagram is a graphical representation of three statistical measures in a 2-dimensional graph to assess the skill and performance of proposed models compared to a reference point. Taylor plot is handy for comparing spatial patterns and variability of multiple model outputs against observations data. The diagram represents various statistical measures of model performance, such as correlation coefficient, standard deviation ratio, and centered root mean square difference (RMSD). The reference point located at the origin (0,0) represents the reference data set. The radial axis represents the standard deviation of the observed data or reference data set. It provides a measure of the observed variability. In this study, the Taylor diagrams are drawn for the TR and TS phases, as depicted in Fig.10.

Fig.10 illustrates that model DNN has predicted the pile-bearing capacity with a standard deviation of 0.252 (in TR) and 0.259 (in TS), close to the standard deviation of actual (reference) TR (= 0.255) and TS (= 0.257) databases. Model CNN is the one with the lowest rank, according to the rank analysis. Consequently, the standard deviation of the pile-bearing capacity predicted by the model CNN in the training and testing phases was compared, and it was discovered that the model CNN had pile-bearing capacity predicted by the training phase with a standard deviation of 0.279 and 0.277, respectively.

4.2.4 Regression error characteristics curve

REC curves are a variant of regression that are similar to the ROC curve in classification. In relation to the x-axis error tolerance, the percentage of predicted points within the y-axis tolerance is shown. The resulting curve is used to estimate the error’s cumulative distribution function. Either the squared error or the absolute deviation can be used to express the error on the x-axis. Similar to the ROC curve, the optimal model is located in the upper left corner. Consequently, the closer the curve gets to this point, the more accurate the model is. The overall inaccuracy is represented by the region above the curve, and it should ideally be as low as is practical. This work plots the REC curves for the training and testing phase of DNN, CNN, RNN, LSTM, and BiLSTM models, as shown in Fig.11.

Fig.11 depicts the REC curve for the TR and TS phases. The area over the curve (AOC) has been calculated and summarized in Tab.9. As can be seen in Tab.9, the predicted pile-bearing capacity of the model DNN is close to the actual TR and TS database’s AOC, or 0.00E−00, with AOCs in the TR and TS phases of 1.58E−03 and 6.46E−03, respectively. As a result, model DNN is acknowledged as the ideal architectural model.

4.2.5 AD test

The AD test is the nonparametric statistical test performed using the overall database to check the research hypothesis. Also, it confirms the distribution of actual and predicted databases. The AD test has been used in this study to determine the pile-bearing capacity, both actual and predicted (using DNN, CNN, RNN, LSTM, and BiLSTM models). Fig.12 shows the results of the AD test in a graphic format.

Fig.12 shows that model DNN has assessed the pile-bearing capacity with the AD value of 12.568, close to the actual pile-bearing capacity, i.e., 12.137. Also, Fig.12 depicts that model LSTM and BiLSTM have AD values of 21.122 and 21.216, respectively.

4.2.6 Accuracy metrix

The accuracy of each model has been computed in terms of performance metrics, i.e., R_{\mathrm{a}\mathrm{c}}^2= R^2\times 100, {VAF}_{\mathrm{a}\mathrm{c}}=VAF, {PI}_{\mathrm{a}\mathrm{c}}=\left(PI/2\right)\times 100, {WI}_{\mathrm{a}\mathrm{c}}=WI\times 100, {WMAPE}_{\mathrm{a}\mathrm{c}}=(1-WMA PE)\times 100, and {RMSE}_{\mathrm{a}\mathrm{c}}=\left(1-RMSE\right)\times 100, where R_{\mathrm{a}\mathrm{c}}^2, {VAF}_{\mathrm{a}\mathrm{c}}, {PI}_{\mathrm{a}\mathrm{c}}, {WI}_{\mathrm{a}\mathrm{c}}, {WMAPE}_{\mathrm{a}\mathrm{c}}, and {RMSE}_{\mathrm{a}\mathrm{c}} are accuracies of each model in terms of determination coefficient, VAF, PI, WI, WMAPE, and RMSE, respectively. The accuracy metric of the models used in each training and testing phase is shown in Fig.13(a) and Fig.13(b). It is evident from Fig.13(a) and Fig.13(b) that the model DNN has higher accuracy in estimating the pile-bearing capacity. In training and testing, model DNN obtained over 90% and 80% accuracy, respectively. Therefore, model DNN is the best computational model for determining the pile-bearing capacity.

4.3 Best architectural model validation

Based on the comparison of performance and accuracy, as well as the visual interpretation of the results, it can be seen that model DNN is the most accurate architectural model for predicting pile-bearing capacity. Furthermore, the best-performing soft computing model for estimating pile-bearing capacity has been identified through external and cross-validation.

4.3.1 External validation

To evaluate the model’s generalizability and make sure it isn’t just overfitting the training set, external validation is carried out. The results of the external validation aid in determining which model best predicts the C. The ability of the model to correctly classify patients as having or not having the outcome of interest is referred to as accuracy. Overfitting is discovered, and model reliability is confirmed through external validation. When a model is overfitted to the training set and struggles to generalize to new data, it is said to be overfit. By contrasting the model’s performance on TR and TS data, external validation can assist in detecting overfitting. In this study, an accurate model has been recognized by the proposed theory by Golbraikh and Tropsha [56]. The theory has several factors for mathematical expressions, as given in Tab.10.

where d_i denotes the experimental C and y_i denotes the predicted C, k and k^{\prime } represent the slopes of the predicted versus actual C and actual versus predicted C concerning the origin. R_0^2 and {R^{\prime }}_0^2 denotes the coefficients of determination of the predicted versus actual C and actual versus predicted C. m and n represent the factors for estimating the predictive power of the proposed models. Tab.11 displays the outcomes of external validation for every model that has been suggested, encompassing both the training and testing stages. Tab.11 shows that the DNN model is better than the CNN, RNN, LSTM, and BiLSTM models in terms of generalizability.

4.3.2 Cross-validation

The cross-validation of the best architectural model has been performed by comparing the test performance of soft computing models available in the literature (see Tab.1). The models developed by Kumar et al. [45], Momeni et al. [48], and Kumar and Samui [57] have performed well using very small databases with 39 and 50 data points, respectively. Other applications with significant data sets include GPR (R²_TS = 0.84), ANFIS (R²_TS = 0.67), GMDH (R²_TS = 0.63), ANFIS-GMDH-ICA (R²_TS = 0.88), ANFIS-PSO (R²_TS = 0.85) and ELM-PSO (R²_TS = 0.84). Compared to these ML models, DNN and BiLSTM are certainly way ahead. However, LSTM, CNN, and RNN are ahead of ANFIS and GMDH. However, it lags behind hybrid models, such as ANFIS-PSO, ELM-PSO, and ANFIS-GMDH-ICA. Finally, the model DNN developed in the present study has been recognized as an optimum performance model in predicting pile-bearing capacity.

4.4 Discussion on results

This work compares the prediction capabilities of DL-based computational models (Table 12). The analysis of results demonstrates that model DNN (R² = 0.9034) is better than models CNN (R² = 0.8237), RNN (R² = 0.7900), LSTM (R² = 0.8288), and BiLSTM (R² = 0.8908) in predicting the bearing capacity. Model DNN has attained 9.67%, 14.35%, 9.00%, and 1.41% higher accuracy than CNN, RNN, LSTM, and BiLSTM models in the TS phase. It is noted that model BiLSTM has performance and accuracy close to the DNN models. Therefore, it may be possible that model BiLSTM can achieve a higher performance than DNN in the case of reconfiguring the BiLSTM hyperparameters. The multicollinearity impact has been observed on RNN and LSTM models. Furthermore, the testing performance of the DNN model has been compared with the models available in the literature study, and it has been found that the DNN model has also outperformed the published models, i.e., ANFIS-PSO, ANFIS-GMDH-ICA, GPR, ANN, and GRNN.

5 Summary and conclusions

The current study presents a soft computing model with optimal performance for estimating pile-bearing capacity. To accomplish that, training and testing databases, as well as a database compiled from the literature, have been created. A mapping of developed models utilizing LSTM, CNN, RNN, DNN, and BiLSTM approaches has been done. Performance metrics, including RMSE, R², WMAPE, VAF, PI, and WI, have been used and compared. The following conclusions are reached from the comprehensive analysis of the data.

1) Deep and Hybrid Model Comparison: The performance comparison reveals that the DNN model performed better than the CNN, RNN, LSTM, and BiLSTM models. However, the BiLSTM model is a sequence processing model that consists of two LSTMs: one taking the input in a forward direction and the other in a backward direction, which increases the information for the network. Therefore, the BiLSTM model predicted bearing capacity better than the LSTM model.

2) Effect of Multicollinearity: The multicollinearity analysis reveals that W has a considerable multicollinearity level for pile-bearing capacity prediction. It can be seen that the D, L, PS, and DH variables have weak multicollinearity levels. However, model RNN has attained the least performance, and it can be stated that multicollinearity affects the performance of the RNN model. On the other side, it can be noted that the DNN model can predict pile-bearing capacity in considerable multicollinearity.

3) Sensitivity Analysis for Inputs Variables: The cosine amplitude sensitivity analysis shows that W highly affects the prediction of pile-bearing capacity, followed by D, L, PS, and DH input variables. Considering the results of the sensitivity analysis, it is proved that the multicollinearity of W has affected the performance of the RNN model in predicting pile-bearing capacity.

To sum up, the present research provides the most reliable and accurate soft computing tool for predicting pile-bearing capacity. Based on the performance and accuracy, model DNN may be implemented to solve the different geotechnical problems of soil related to the field and laboratory. This study maps a comparison among DL approaches for the first time in predicting the pile-bearing capacity, a significant advantage of this research. This work may be expanded by strengthening the data points, as it has been done with a restricted database. Additionally, the Runge Kutta Optimizer, Dwarf Mongoose Optimization, Artificial Gorilla Troops Optimizer, and Artificial Hummingbird algorithms may be used to optimize the soft computing approaches used in this study. The optimized models may then be utilized to forecast the pile-bearing capacity. Additionally, the current study may be expanded by comparing the DNN model with approaches such as extreme gradient boosting, light gradient boosting, and cat boosting.

References

[1] Rausche F, Goble G G, Likins G E. Dynamic determination of pile capacity. Journal of Geotechnical Engineering, 1985, 111(3): 367–383

[2] Heidarie Golafzani S, Jamshidi Chenari R, Eslami A. Reliability based assessment of axial pile bearing capacity: static analysis, SPT and CPT-based methods. Georisk. Assessment and Management of Risk for Engineered Systems and Geohazards, 2020, 14(3): 216–230

[3] Huynh V H, Nguyen T, Nguyen D P, Nguyen T S, Nguyen T C. A novel direct SPT method to accurately estimate ultimate axial bearing capacity of bored PHC nodular piles with 81 case studies in Vietnam. Soil and Foundation, 2022, 62(4): 101163

[4] Zein A K M, Ayoub E M. Evaluation of measured and interpreted failure loads of bored piles in alluvial soil deposits. GEOMATE Journal, 2016, 10(19): 1636–1643

[5] Krasiński A, Wiszniewski M. Static load test on instrumented pile–field data and numerical simulations. Studia Geotechnica et Mechanica, 2017, 39(3): 17–25

[6] Lastiasih Y, Sari P T K. Comparison of ultimate bearing capacity based on empirical method, interpretation of loading pile test and finite element. In: Proceedings of the IOP Conference Series: Materials Science and Engineering. IOP Publishing, 2020, 930(1): 012036

[7] WhittleA J. Assessment of an effective stress analysis for predicting the performance of driven piles in clays. In: SUT Offshore Site Investigation and Foundation Behaviour New Frontiers: Proceedings of an International Conference. London: Springer Netherlands, 1993: 607–643

[8] Bak E. Numerical modeling of pile load tests. Pollack Periodica, 2013, 8(2): 131–140

[9] He S, Lai J, Li Y, Wang K, Wang L, Zhang W. Pile group response induced by adjacent shield tunnelling in clay: Scale model test and numerical simulation. Tunnelling and Underground Space Technology, 2022, 120: 104039

[10] Józefiak K, Zbiciak A, Maślakowski M, Piotrowski T. Numerical modelling and bearing capacity analysis of pile foundation. Procedia Engineering, 2015, 111: 356–363

[11] Loganathan N, Poulos H G. Analytical prediction for tunneling-induced ground movements in clays. Journal of Geotechnical and Geoenvironmental Engineering, 1998, 124(9): 846–856

[12] Al-Atroush M E, Hefny A, Zaghloul Y, Sorour T. Behavior of a large diameter bored pile in drained and undrained conditions: comparative analysis. Geosciences, 2020, 10(7): 261–281

[13] Chaallal O, Arockiasamy M, Godat A. Field test performance of buried flexible pipes under live truck loads. Journal of Performance of Constructed Facilities, 2015, 29(5): 04014124

[14] Park D, Rilett L R. Forecasting freeway link travel times with a multilayer feedforward neural network. Computer-Aided Civil and Infrastructure Engineering, 1999, 14(5): 357–367

[15] Tran V T, Nguyen T K, Nguyen-Xuan H, Wahab M A. Vibration and buckling optimization of functionally graded porous microplates using BCMO-ANN algorithm. Thin-walled Structures, 2023, 182: 110267

[16] Dang B L, Nguyen-Xuan H, Wahab M A. An effective approach for VARANS-VOF modelling interactions of wave and perforated breakwater using gradient boosting decision tree algorithm. Ocean Engineering, 2023, 268: 113398

[17] Wang S, Wang H, Zhou Y, Liu J, Dai P, Du X, Wahab M A. Automatic laser profile recognition and fast tracking for structured light measurement using deep learning and template matching. Measurement, 2021, 169: 108362

[18] Ho L V, Trinh T T, De Roeck G, Bui-Tien T, Nguyen-Ngoc L, Wahab M A. An efficient stochastic-based coupled model for damage identification in plate structures. Engineering Failure Analysis, 2022, 131: 105866

[19] Samaniego E, Anitescu C, Goswami S, Nguyen-Thanh V M, Guo H, Hamdia K, Zhuang X, Rabczuk T. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Computer Methods in Applied Mechanics and Engineering, 2020, 362: 112790

[20] ZhaoJTu JShiY. An ANN Model for Predicting Level Ultimate Bearing Capacity of PHC Pipe Pile. Earth and Space 2010: Engineering, Science, Construction, and Operations in Challenging Environments, 2010, 3168–3176

[21] Kordnaeij A, Kalantary F, Kordtabar B, Mola-Abasi H. Prediction of recompression index using GMDH-type neural network based on geotechnical soil properties. Soil and Foundation, 2015, 55(6): 1335–1345

[22] Momeni E, Nazir R, Armaghani D J, Maizir H. Prediction of pile bearing capacity using a hybrid genetic algorithm-based ANN. Measurement, 2014, 57: 122–131

[23] Biswas R, Samui P, Rai B. Determination of compressive strength using relevance vector machine and emotional neural network. Asian Journal of Civil Engineering, 2019, 20(8): 1109–1118

[24] Biswas R, Rai B, Samui P, Roy S S. Estimating concrete compressive strength using MARS, LSSVM and GP. Engineering Journal, 2020, 24(2): 41–52

[25] Tu J V. Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. Journal of Clinical Epidemiology, 1996, 49(11): 1225–1231

[26] Benbouras M A, Petrişor A I, Zedira H, Ghelani L, Lefilef L. Forecasting the bearing capacity of the driven piles using advanced machine-learning techniques. Applied Sciences, 2021, 11(22): 10908

[27] PhamT ANguyen D HDuongH A T. Development of deep learning neural network for estimating pile bearing capacity. In: Proceedings of the 6th International Conference on Geotechnics, Civil Engineering and Structures. Singapore: Springer, 2022: 1815–1823

[28] Zhang P, Yin Z Y. A novel deep learning-based modelling strategy from image of particles to mechanical properties for granular materials with CNN and BiLSTM. Computer Methods in Applied Mechanics and Engineering, 2021, 382: 113858

[29] GuoHZhuangX FuXZhuY RabczukT. Physics-informed deep learning for three-dimensional transient heat transfer analysis of functionally graded materials. Computational Mechanics, 2023: 1–12

[30] Guo H, Zhuang X, Alajlan N, Rabczuk T. Physics-informed deep learning for melting heat transfer analysis with model-based transfer learning. Computers & Mathematics with Applications, 2023, 143: 303–317

[31] Guo H, Zhuang X, Chen P, Alajlan N, Rabczuk T. Stochastic deep collocation method based on neural architecture search and transfer learning for heterogeneous porous media. Engineering with Computers, 2022, 38(6): 5173–5198

[32] GuoHZhuangX RabczukT. A deep collocation method for the bending analysis of Kirchhoff plate. 2021, arXiv: 2102.02617

[33] Zhuang X, Guo H, Alajlan N, Zhu H, Rabczuk T. Deep autoencoder based energy method for the bending, vibration, and buckling analysis of Kirchhoff plates with transfer learning. European Journal of Mechanics A/Solids, 2021, 87: 104225

[34] TaherkhaniA HMeiQHanF. A deep learning model to predict the lateral capacity of monopiles. In: Proceeding of Geo-Congress 2023. Los Angel: ASCE, 220–227

[35] Cheng H, Zhang H, Liu Z, Wu Y. Prediction of undrained bearing capacity of skirted foundation in spatially variable soils based on convolutional neural network. Applied Sciences, 2023, 13(11): 6624

[36] Shahin M A. Load–settlement modeling of axially loaded steel driven piles using CPT-based recurrent neural networks. Soil and Foundation, 2014, 54(3): 515–522

[37] Shahin M A. State-of-the-art review of some artificial intelligence applications in pile foundations. Geoscience Frontiers, 2016, 7(1): 33–44

[38] Zhang W, Li H, Li Y, Liu H, Chen Y, Ding X. Application of deep learning algorithms in geotechnical engineering: a short critical review. Artificial Intelligence Review, 2021, 54(8): 1–41

[39] Gao B, Wang R, Lin C, Guo X, Liu B, Zhang W. TBM penetration rate prediction based on the long short-term memory neural network. Underground Space, 2021, 6(6): 718–731

[40] Uncuoglu E, Citakoglu H, Latifoglu L, Bayram S, Laman M, Ilkentapar M, Oner A A. Comparison of neural network, Gaussian regression, support vector machine, long short-term memory, multi-gene genetic programming, and M5 Trees methods for solving civil engineering problems. Applied Soft Computing, 2022, 129: 109623

[41] Tao Y, Sun H, Cai Y. Predictions of deep excavation responses considering model uncertainty: Integrating BiLSTM neural networks with Bayesian updating. International Journal of Geomechanics, 2022, 22(1): 04021250

[42] Zhang P, Yang Y, Yin Z Y. BiLSTM-based soil–structure interface modeling. International Journal of Geomechanics, 2021, 21(7): 04021096

[43] Chen C, Wu B, Jia P, Wang Z. A Novel Hybrid Deep Neural Network Prediction Model for Shield Tunneling Machine Thrust. IEEE Access: Practical Innovations, Open Solutions, 2022, 10: 123858–123873

[44] Wang H, Zhang L, Luo H, He J, Cheung R W M. AI-powered landslide susceptibility assessment in Hong Kong. Engineering Geology, 2021, 288: 106103

[45] Kumar M, Kumar V, Rajagopal B G, Samui P, Burman A. State of art soft computing-based simulation models for bearing capacity of pile foundation: a comparative study of hybrid ANNs and conventional models. Modeling Earth Systems and Environment, 2023, 9(2): 2533–2551

[46] Kumar M, Biswas R, Kumar D R, Pradeep T, Samui P. Metaheuristic models for the prediction of bearing capacity of pile foundation. Geomechanics and Engineering, 2022, 31(2): 129–147

[47] Armaghani D J, Harandizadeh H, Momeni E, Maizir H, Zhou J. An optimized system of GMDH-ANFIS predictive model by ICA for estimating pile bearing capacity. Artificial Intelligence Review, 2022, 55(3): 1–38

[48] Momeni E, Dowlatshahi M B, Omidinasab F, Maizir H, Armaghani D J. Gaussian process regression technique to estimate the pile bearing capacity. Arabian Journal for Science and Engineering, 2020, 45(10): 8255–8267

[49] Momeni E, Nazir R, Armaghani D J, Maizir H. Application of artificial neural network for predicting shaft and tip resistances of concrete piles. Earth Sciences Research Journal, 2015, 19(1): 85–93

[50] Shahin M A. Intelligent computing for modeling axial capacity of pile foundations. Canadian Geotechnical Journal, 2010, 47(2): 230–243

[51] Kiefa M A. General regression neural networks for driven piles in cohesionless soils. Journal of Geotechnical and Geoenvironmental Engineering, 1998, 124(12): 1177–1185

[52] Khatti J, Grover K S. Prediction of compaction parameters for fine-grained soil: Critical comparison of the deep learning and standalone models. Journal of Rock Mechanics and Geotechnical Engineering, 2023, 15(11): 3010–3038

[53] Khatti J, Grover K S. Prediction of UCS of fine-grained soil based on machine learning part 1: multivariable regression analysis, gaussian process regression, and gene expression programming. Multiscale and Multidisciplinary Modeling. Experiments and Design, 2023, 6(2): 199–222

[54] KhattiJSamadi HGroverK S. Estimation of Settlement of Pile Group in Clay Using Soft Computing Techniques. Geotechnical and Geological Engineering, 2023: 1−32

[55] SmithG N. Probability and Statistics in Civil Engineering. London: Collins, 1986, 1986

[56] GolbraikhATropsha A. Beware of q2! Journal of Molecular Graphics & Modelling, 2002, 20(4): 269–276

[57] KumarMSamui P. Reliability analysis of pile foundation using GMDH, GP and MARS. In: Proceedings of the 6th International Conference on Geotechnics, Civil Engineering and Structures. Singapore: Springer, 2022: 1151–1159

RIGHTS & PERMISSIONS

Higher Education Press