Computer-vision-guided semi-autonomous concrete crack repair for infrastructure maintenance using a robotic arm

Abstract

Engineering inspection and maintenance technologies play an important role in safety, operation, maintenance and management of buildings. In project construction control, supervision of engineering quality is a difficult task. To address such inspection and maintenance issues, this study presents a computer-vision-guided semi-autonomous robotic system for identification and repair of concrete cracks, and humans can make repair plans for this system. Concrete cracks are characterized through computer vision, and a crack feature database is established. Furthermore, a trajectory generation and coordinate transformation method is designed to determine the robotic execution coordinates. In addition, a knowledge base repair method is examined to make appropriate decisions on repair technology for concrete cracks, and a robotic arm is designed for crack repair. Finally, simulations and experiments are conducted, proving the feasibility of the repair method proposed. The result of this study can potentially improve the performance of on-site automatic concrete crack repair, while addressing such issues as high accident rate, low efficiency, and big loss of skilled workers.

Keywords

Computer vision / Concrete crack repair / Robotic construction / Semi-autonomous / Knowledge base system / Human decision making

References

[1] AhmadS, ElahiA, BarbhuiyaS, FarooqiY. Repair of cracks in simply supported beams using epoxy injection technique. Materials and Structures., 2013, 46(9):1547-1559

[2] AlgaifiHA, BakarSA, SamARM, AbidinARZ, ShahirS, AL-TowaytiWAH. Numerical modeling for crack self-healing concrete by microbial calcium carbonate. Construction and Building Materials., 2018, 189: 816-824

[3] AttardL, DebonoCJ, ValentinoG, Di CastroM. Vision-based change detection for inspection of tunnel liners. Automation in Construction, 2018, 91: 142-154

[4] BadugeSK, ThilakarathnaS, PereraJS. Artificial intelligence and smart vision for building and construction 4.0: Machine and deep learning methods and applications. Automation in Construction, 2022, 141: 104440

[5] BoumaazaM, BezaziA, BouchelaghemH, BenzennacheN, AmzianeS, ScarpaF. Behavior of pre-cracked deep beams with composite materials repairs. Structural Engineering and Mechanics., 2017, 63(5):575-583

[6] BykovA, MatveenkoV, ShardakovI, ShestakovA. Shock wave method for monitoring crack repair processes in reinforced concrete structures. Mechanics of Solids., 2017, 52(4):378-383

[7] ChaYJ, ChoiW, BüyüköztürkO. Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering., 2017, 32(5):361-378

[8] ChaYJ, ChoiW, SuhG, MahmoudkhaniS, BüyüköztürkO. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Computer-Aided Civil and Infrastructure Engineering., 2018, 33(9):731-747

[9] ChhengC, LikitlersuangS. Underground excavation behaviour in Bangkok using three-dimensional finite element method. Computers and Geotechnics., 2018, 95: 68-81

[10] ChoiS-G, WangK, WenZ, ChuJ. Mortar crack repair using microbial induced calcite precipitation method. Cement and Concrete Composites., 2017, 83: 209-221

[11] Dan, D. H, & Dan, Q. (2021). Automatic recognition of surface cracks in bridges based on 2D-APES and mobile machine vision. Measurement. 2021.

[12] DanaD, KarlD, AgatheR. Effect of casting and curing temperature on the interfacial bond strength of epoxy bonded concretes. Construction and Building Materials, 2021, 307

[13] Harsh, R, Chris, B, & Ali, M. (2020). Defects detection and quantification in dam spillways using robotics and computer vision. 573–582.

[14] HuangH-W, LiQ-T, ZhangD-M. Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunnelling and Underground Space Technology., 2018, 77: 166-176

[15] JafarniaMS, SaryazdiMK, MoshtaghiounSM. Use of bacteria for repairing cracks and improving properties of concrete containing limestone powder and natural zeolite. Construction and Building Materials, 2020, 242: 118059

[16] KimH, LiuXM, AhnE, ShinM, ShinSW, SimS. Performance assessment method for crack repair in concrete using PZT-based electromechanical impedance technique. NDT and E International., 2019, 104: 90-97

[17] KimH, SimS-H, SpencerBF. Automated concrete crack evaluation using stereo vision with two different focal lengths. Automation in Construction, 2022, 135: 104136

[18] KovačevićMS, BačićM, VukomanovićM, CerićA. A framework for automatic calculation of life-cycle remediation costs of secondary lining cracks. Automation in Construction, 2021, 129: 103714

[19] LiuL, YanR-J, MaruvancheryV, KayacanE, ChenI-M, TiongLK. Transfer learning on convolutional activation feature as applied to a building quality assessment robot. International Journal of Advanced Robotic Systems., 2017, 14(3):1729881417712620

[20] LiuYQ, YeohJ. Robust pixel-wise concrete crack segmentation and properties retrieval using image patches. Automation in Construction., 2021, 123: 103535

[21] LiuZQ, CaoYW, WangYZ, WangW. Computer vision-based concrete crack detection using U-net fully convolutional networks. Automation in Construction., 2019, 104: 129-139

[22] MarazaniT, MadyiraDM, AkinlabiET. Repair of cracks in metals: A review. Procedia Manufacturing., 2017, 8: 673-679

[23] MardasiAG, WuN, WuC. Experimental study on the crack detection with optimized spatial wavelet analysis and windowing. Mechanical Systems and Signal Processing, 2018, 104: 619-630

[24] MenendezE, VictoresJG, MonteroR, MartínezS, BalaguerC. Tunnel structural inspection and assessment using an autonomous robotic system. Automation in Construction, 2018, 87: 117-126

[25] MohanA, PoobalS. Crack detection using image processing: A critical review and analysis. Alexandria Engineering Journal., 2018, 57(2):787-798

[26] OsmanBH, WuE, BohaiJ, AbdallahM. Repair technique of pre-cracked reinforced concrete (RC) beams with transverse openings strengthened with steel plate under sustained load. Journal of Adhesion Science and Technology., 2017, 31(21):2360-2379

[27] Pantoja-RoseroBG, OnerD, KozinskiM. TOPO-Loss for continuity-preserving crack detection using deep learning. Construction and Building Materials, 2022, 344(15):128264

[28] RameshG, SrinathD, RamyaD, KrishnaBV. Repair, rehabilitation and retrofitting of reinforced concrete structures by using non-destructive testing methods. Materials Today: Proceedings., 2021

[29] RiccardoM, LorenzoB, BrunellaB, CristinaT, JohnS, IuliaM. A crack closure system for cementitious composite materials using knotted shape memory polymer (k-SMP) fibres. Cement and Concrete Composites, 2020, 114: 103757

[30] ShamsabadiEA, XuC, RaoAS. Vision transformer-based autonomous crack detection on asphalt and concrete surfaces. Automation in Construction, 2022, 140: 104316

[31] SutterB, LelevéA, PhamMT, GouinO, JupilleN, KuhnM, LuléP, MichaudP, RémyP. A semi-autonomous mobile robot for bridge inspection. Automation in Construction., 2018, 91: 111-119

[32] TedeschiA, BenedettoF. A real-time automatic pavement crack and pothole recognition system for mobile Android-based devices. Advanced Engineering Informatics., 2017, 32: 11-25

[33] TsiatasG, RobinsonJ. Durability evaluation of concrete crack repair systems. Transportation Research Record Journal of the Transportation Research Board., 2002, 1795: 82-87

[34] ValençaJ, Dias-da-CostaD, JúlioE. Characterisation of concrete cracking during laboratorial tests using image processing. Construction and Building Materials., 2012, 28(1):607-615

[35] Wang, P., & Huang, H. (2010). Comparison analysis on present image-based crack detection methods in concrete structures. In Proc., Image and Signal Processing (CISP), 2010 3rd International Congress on, IEEE. 2530–2533. https://doi.org/10.1109/CISP.2010.5647496

[36] WangSHM, WilliamsP, ShiJ, YangH. From green to sustainability—trends in the assessment methods of green buildings. Frontiers of Engineering Management., 2015, 2(2):114-121

[37] XueY, LiY. A fast detection method via region-based fully convolutional neural networks for shield tunnel lining defects. Computer-Aided Civil and Infrastructure Engineering., 2018, 33(8):638-654

[38] Zhang, Z. (2000). A flexible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence. 22. https://dwz.cn/m0obPtcb

[39] ZhangA, WangKC, FeiY, LiuY, ChenC, YangG, LiJQ, YangE, QiuS. Automated pixel-level pavement crack detection on 3D Asphalt surfaces with a recurrent neural network. Computer-Aided Civil and Infrastructure Engineering., 2017

[40] ZhangA, WangKC, LiB, YangE, DaiX, PengY, FeiY, LiuY, LiJQ, ChenC. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Computer-Aided Civil and Infrastructure Engineering., 2017, 32(10):805-819

[41] ZhangQY, BarriK, BabanajadS, AlaviA. Real-time detection of cracks on concrete bridge decks using deep learning in the frequency domain. Journal Pre-Proofs, 2020

[42] ZhouC, DingL, ZhouY, SkibniewskiMJ. Visibility graph analysis on time series of shield tunneling parameters based on complex network theory. Tunnelling and Underground Space Technology., 2019, 89: 10-24

[43] ZhuJ-W, ZhouL-N, YuM-Y, ZhaiZ. The development and comparative analysis of engineering project management modes. Frontiers of Engineering Management., 2015, 2(4):351-361

Funding

National Natural Science Foundation of China(71821001); Hubei Province, China: Key Technologies and Applications of Intelligent Construction(2020ACA006)