Intelligent feature extraction, data fusion and detection of concrete bridge cracks: current development and challenges

Abstract

As a common appearance defect of concrete bridges, cracks are important indices for bridge structure health assessment. Although there has been much research on crack identification, research on the evolution mechanism of bridge cracks is still far from practical applications. In this paper, the state-of-the-art research on intelligent theories and methodologies for intelligent feature extraction, data fusion and crack detection based on data-driven approaches is comprehensively reviewed. The research is discussed from three aspects: the feature extraction level of the multimodal parameters of bridge cracks, the description level and the diagnosis level of the bridge crack damage states. We focus on previous research concerning the quantitative characterization problems of multimodal parameters of bridge cracks and their implementation in crack identification, while highlighting some of their major drawbacks. In addition, the current challenges and potential future research directions are discussed.

Keywords

Intelligent detection / crack detection / deep learning / data fusion / feature extraction

References

[1] Sun Y,Sheng D,Dai B.Research on bridge structural damage identification.Sci Program2022;2022:1-14

[2] Lin W,Yang Q.Real-time comprehensive image processing system for detecting concrete bridges crack.Comput Concr2019;23:445-457

[3] Putra SA,Riyansyah M.Multiagent architecture for bridge capacity measurement system using wireless sensor network and weight in motion.IEEE Trans Instrum Meas2021;70:1-14

[4] Collings D.Modern concrete bridge deck analysis considering the effects of cracking.Proc Inst Civ Eng: Struct Build2021;174:595-605

[5] Mao J,Wang H,Lu H.Bayesian operational modal analysis with genetic optimization for structural health monitoring of the long-span bridge.Int J Str Stab Dyn2022;22:2250051

[6] Maes K,Reynders E.Validation of vibration-based structural health monitoring on retrofitted railway bridge KW51.Mech Syst Signal Process2022;165:108380

[7] Prasanna P,Gucunski N.Automated crack detection on concrete bridges.IEEE Trans Automat Sci Eng2016;13:591-9

[8] Zhang L,Han Y,Wu Y.Application of internet of things technology and convolutional neural network model in bridge crack detection.IEEE Access2018;6:39442-51

[9] Li H,Tian X.Bridge crack detection based on ssenets.Appl Sci2020;10:4230 PMCID:PMC9531840

[10] Jiang W,Peng Y,Wang Y.HDCB-net: a neural network with the hybrid dilated convolution for pixel-level crack detection on concrete bridges.IEEE Trans Ind Inf2021;17:5485-94

[11] Yamaguchi T,Tarumi M.Sensitive damage detection of reinforced concrete bridge slab by “time-variant deconvolution” of SHF-band radar signal.IEEE Trans Geosci Remote Sensing2019;57:1478-88

[12] Xu H,Wang Y,Cui K.Automatic bridge crack detection using a convolutional neural network.Applied Sciences2019;9:2867

[13] Gao R.Seismic performance assessment of concrete bridges with traffic-induced fatigue damage.Eng Fail Anal2022;134:106042

[14] Lon Wah W, Xia Y. Elimination of outlier measurements for damage detection of structures under changing environmental conditions.Struct Health Monit2022;21:320-38

[15] Guo L,Jiang B.A cascade broad neural network for concrete structural crack damage automated classification.IEEE Trans Industr Inform2021;17:2737-42

[16] Okazaki Y,Asamoto S.Applicability of machine learning to a crack model in concrete bridges.COMPUT-AIDED CIV INF2020;35:775-92

[17] Lu Q,Zhang W.Quantification of fatigue damage for structural details in slender coastal bridges using machine learning-based methods.J Bridge Eng2020;25:04020033

[18] Li G,Zhao S,Ren X.Automatic crack recognition for concrete bridges using a fully convolutional neural network and naive Bayes data fusion based on a visual detection system.Meas Sci Technol2020;31:075403

[19] Kumar P,Kota SR.Automatic multiclass instance segmentation of concrete damage using deep learning model.IEEE Access2021;9:90330-45

[20] Pathak N.Bridge health monitoring using CNN. 2020 International Conference on Convergence to Digital World - Quo Vadis (ICCDW); 2020 Feb. 18-20; Mumbai, India: IEEE; 2020. p. 1-4.

[21] Attard L,Valentino G,Masi A.Automatic crack detection using mask R-CNN. 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA); 2019 Sept. 23-25 Dubrovnik, Croatia: IEEE; 2019. p. 152-157.

[22] Kamada S,Iwasaki T.An adaptive structural learning of deep belief network for image-based crack detection in concrete structures using SDNET2018. 2020 International Conference on Image Processing and Robotics (ICIP); 2020 Mar. 06-08 Negombo, Sri Lanka: IEEE; 2020. p. 1-6.

[23] Liu T,Zhou G,Cai C.BC-DUnet-based segmentation of fine cracks in bridges under a complex background.PLoS One2022;17:e0265258 PMCID:PMC8923471

[24] Simonetti F,Brath AJ.Cryo-ultrasonic NDE: ice-cold ultrasonic waves for the detection of damage in complex-shaped engineering components.IEEE Trans Ultrason Ferroelectr Freq Control2018;65:638-47

[25] Zhou LQ,Myronov M.Ultrasonic inspection and self-healing of ge and 3C-SiC semiconductor membranes.J Microelectromech Syst2020;29:370-7

[26] Lee FW,Lim KS.Concrete sub-surface crack characterization by means of surface rayleigh wave method.ACI Materials Journal2019;116

[27] Wadas SH,Polom U.Ground instability of sinkhole areas indicated by elastic moduli and seismic attributes.Geophys J Int2020;222:289-304

[28] Ghasemi MF.Application of rock physics modelling to investigate the differences between static and dynamic elastic moduli of carbonates.Geophys J Int2020;222:1992-2023

[29] Luan X,Sedghi S.Probabilistic PCR based near-infrared modeling with temperature compensation.ISA Trans2018;81:46-51

[30] Zhu J,Song Z.Distributed parallel PCA for modeling and monitoring of large-scale plant-wide processes with big data.IEEE Trans Ind Inf2017;13:1877-85

[31] Yu Y,Samali B,Wang W.Multi-image-feature-based hierarchical concrete crack identification framework using optimized SVM multi-classifiers and D-S fusion algorithm for bridge structures.Remote Sens2021;13:240

[32] Mei Q,Boay M.Indirect health monitoring of bridges using Mel-frequency cepstral coefficients and principal component analysis.Mech Syst Signal Process2019;119:523-46

[33] Dong Z,Xu F.A low-rank and sparse decomposition-based method of improving the accuracy of sub-pixel grayscale centroid extraction for spot images.IEEE Sensors J2020;20:5845-54

[34] Li L,Sun R.Study on bridge floor crack classification method based on sparse coding.J Light Technol2018;33:66-74

[35] Wang B,Zhao W.Fast concrete crack detection method via L2 sparse representation.Electron lett2018;54:752-4

[36] Naveed K.Wavelet based multivariate signal denoising using mahalanobis distance and EDF statistics.IEEE Trans Signal Process2020;68:5997-6010

[37] Nguyen TQ,Le CM,Nguyen-Xuan H.A data-driven approach based on wavelet analysis and deep learning for identification of multiple-cracked beam structures under moving load.Meas Sci Technol2020;162:1-21

[38] Nigam R.Crack detection in a beam using wavelet transform and photographic measurements.Structures2020;25:436-47

[39] Wang S,Jiang Y.Input-output method to fault detection for discrete-time fuzzy networked systems with time-varying delay and multiple packet losses.Int J Syst Sci2016;47:1495-513

[40] Liu L,Zhang H.Adaptive NN fault-tolerant control for discrete-time systems in triangular forms with actuator fault.Neurocomputing2015;152:209-221

[41] Mao W-L.Modeless magnetic bearing system tracking using an adaptive fuzzy hermite neural network method.IEEE Sens J2019;19:5904-15

[42] Na J,Wu X,Li G.Adaptive finite-time fuzzy control of nonlinear active suspension systems with input delay.IEEE Trans Cybern2020;50:2639-50

[43] Bilir T,Topcu IB.Prediction of restrained shrinkage crack widths of slag mortar composites by Takagi and Sugeno ANFIS models.Neural Comput Appl2016;27:2523-36

[44] Sharma M,Chaiyasarn K.Concrete crack detection using the integration of convolutional neural network and support vector machine.Sci Technol Asia2018;23:19-28

[45] Wang D,Pan Y.Machine vision-based monitoring methodology for the fatigue cracks in U-Rib-to-Deck weld seams.IEEE Access2020;8:94204-19

[46] Zheng M,Zhang K.Intelligent detection of building cracks based on deep learning.Image Vis Comput2020;103:1-10

[47] Xu H,Wang Y,Cui K.Automatic bridge crack detection using a convolutional neural network.Appl Sci2019;9:2867

[48] Teng S,Chen G.Concrete crack detection based on well-known feature extractor model and the YOLO_v2 network.Appl Sci2021;11:1-13

[49] Pan Y,Zhang L.A spatial-channel hierarchical deep learning network for pixel-level automated crack detection.Autom Constr2020;119:103357

[50] Zhao J,Qiao W,Xu Y.A modified U-net for crack segmentation by self-attention-self-adaption neuron and random elastic deformation.Smart Struct Syst2022;29:1-16

[51] Yang Y,Xue J.Correction to: estimating evapotranspiration by coupling bayesian model averaging methods with machine learning algorithms.Environ Monit Assess2021;193:207

[52] Zhao X,Gu H,Mo YL.Innovative data fusion enabled structural health monitoring approach.Math Probl Eng2014;2014:1-10

[53] Guo T.Data fusion of multi-scale representations for structural damage detection.Mech Syst Signal Process2018;98:1020-33

[54] Zhang Y,Yang Y.Data‐driven design of two‐degree‐of‐freedom controllers using reinforcement learning techniques.IIET Control Theory Appl2015;9:1011-21

[55] Palanisamy RP,Kim H.Experimental validation of Kalman filter-based strain estimation in structures subjected to non-zero mean input.Smart Struct Syst2015;15:489-503

[56] Xu Y,Chen J,Li H.Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumer-grade camera images.Struct Health Monit2019;18:653-74

[57] Li G,Zhou J,Ren W.Pixel-level bridge crack detection using a deep fusion about recurrent residual convolution and context encoder network.Measurement2021;176:109171

[58] Chen F.NB-CNN: Deep learning-based crack detection using convolutional neural network and naïve bayes data fusion.IEEE Trans Ind Electron2018;65:4392-400

[59] Yang J,Zou J,Li R.Concrete crack segmentation based on UAV-enabled edge computing.Neurocomputing2022;485:233-41

[60] Li X,Pedrycz W.Linking granular computing, big data and decision making: a case study in urban path planning.Soft Comput2020;24:7435-50

[61] Wang B,Gao P,Wang Z.Crack damage detection method via multiple visual features and efficient multi-task learning model.Sensors (Basel)2018;18:1796 PMCID:PMC6021921

[62] Yan B,Zhang L.Beam structure damage identification based on BP neural network and support vector machine.Math Probl Eng2014;2014:1-8

[63] Liu M,Zhao X.The crack injury detection based CMR and Natural frequency with BP neutral network. 2nd International Conference on Future Networks (ICFN 2010); 2010 Jan. 22-24; Sanya, Chin: IEEE; 2010. p. 314-17.

[64] Chen CLP.Broad learning system: a new learning paradigm and system without going deep. 32nd Youth Academic Annual Conference of Chinese Association of Automation (YAC); 2017 May 19-21; Hefei, China: IEEE; 2017. p. 17009788.

[65] Chen CLP.Broad learning system: an effective and efficient incremental learning system without the need for deep architecture.IEEE Trans Neural Netw Learn Syst2018;29:10-24

[66] Guo L,Jiang B.Automatic crack distress classification from concrete surface images using a novel deep-width network architecture.Neurocomputing2020;397:383-92

[67] Chen CLP,Feng S.Universal approximation capability of broad learning system and its structural variations.IEEE Trans Neural Netw Learn Syst2019;30:1191-204

[68] Xu M,Chen CLP.Recurrent broad learning systems for time series prediction.IEEE Trans Cybern2020;50:1405-17

[69] Li C,Li Y.Research on bridge crack detection algorithm based on deep learning.J Lab Autom2019;45:16

[70] Islam MMM.Vision-based autonomous crack detection of concrete structures using a fully convolutional encoder-decoder network.Sensors (Basel)2019;19:4251 PMCID:PMC6806320

[71] Liang D,Wang S.Research on concrete cracks recognition based on dual convolutional neural network.KSCE J Civ Eng2019;23:3066-74

[72] Li L,Zhang L.FS-LSTM-based sensor fault and structural damage isolation in SHM.IEEE Sensors J2021;21:3250-9

[73] Zhao Z.3D digital analysis of cracking behaviors of rocks through 3D reconstruction model under triaxial compression.J Eng Mech2020;146:04020084

[74] Yang L,Li W,Jiang B.Concrete defects inspection and 3D mapping using CityFlyer quadrotor robot.IEEE/CAA J Autom Sinica2020;7:991-1002