Optimal CNN-based semantic segmentation model of cutting slope images

Abstract

This paper utilizes three popular semantic segmentation networks, specifically DeepLab v3+, fully convolutional network (FCN), and U-Net to qualitively analyze and identify the key components of cutting slope images in complex scenes and achieve rapid image-based slope detection. The elements of cutting slope images are divided into 7 categories. In order to determine the best algorithm for pixel level classification of cutting slope images, the networks are compared from three aspects: a) different neural networks, b) different feature extractors, and c) 2 different optimization algorithms. It is found that DeepLab v3+ with Resnet18 and Sgdm performs best, FCN 32s with Sgdm takes the second, and U-Net with Adam ranks third. This paper also analyzes the segmentation strategies of the three networks in terms of feature map visualization. Results show that the contour generated by DeepLab v3+ (combined with Resnet18 and Sgdm) is closest to the ground truth, while the resulting contour of U-Net (combined with Adam) is closest to the input images.

Graphical abstract

Keywords

slope damage / image recognition / semantic segmentation / feature map / visualizations

References

[1] ChenZ. Soil Slope Stability Analysis––Principle, Methods and Programs. Beijing: China Water & Power Press, 2003 (in Chinese)

[2] Wu C I, Kung H Y, Chen C H, Kuo L C. An intelligent slope disaster prediction and monitoring system based on WSN and ANP. Expert Systems with Applications, 2014, 41( 10): 4554–4562

[3] ShuJZhangJ WuJ. Research on highway slope disaster identification based on deep convolution neural network. Highway Traffic Technology, 2017, 13(10): 70–74 (in Chinese)

[4] WuJ. Feature learning of highway image and detection of slope failure. Thesis for the Master’s Degree. Beijing: Beijing University of Posts and Telecommunications, 2018 (in Chinese)

[5] Xu J, Gui C, Han Q. Recognition of rust grade and rust ratio of steel structures based on ensembled convolutional neural network. Computer-Aided Civil and Infrastructure Engineering, 2020, 35( 10): 1160–1174

[6] Guo H, Zhuang X, Rabczuk T. A deep collocation method for the bending analysis of Kirchhoff plate. Computers, Materials & Continua, 2019, 59( 2): 433–456

[7] Anitescu C, Atroshchenko E, Alajlan N, Rabczuk T. Artificial neural network methods for the solution of second order boundary value problems. Computers, Materials & Continua, 2019, 59( 1): 345–359

[8] Samaniego E, Anitescu C, Goswami S, Nguyen-Thanh V M, Guo H, Hamdia K, Zhuang X, Rabczuk T. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Computer Methods in Applied Mechanics and Engineering, 2020, 362 : 112790

[9] ZhouHChen YTianR. Distance prediction of slope-foot landslide in southwest of China based on GA-BP neural network. In: 2019 the 6th Annual International Conference on Material Engineering and Application. Guangzhou: IOP Publishing, 2020

[10] XingYWang JLiXLiuRGaoJ. Slope stability prediction model based on GA-SVM. In: 2010 International Conference on Educational and Information Technology. Chongqing: IEEE, 2010

[11] Lin H M, Chang S K, Wu J H, Juang C H. Neural network-based model for assessing failure potential of highway slopes in the Alishan, Taiwan Area (China): Pre- and post-earthquake investigation. Engineering Geology, 2009, 104( 3-4): 280–289

[12] Xia Y, Chen B, Weng S, Ni Y Q, Xu Y L. Temperature effect on vibration properties of civil structures: A literature review and case studies. Journal of Civil Structural Health Monitoring, 2012, 2( 1): 29–46

[13] Yao X. Evolutionary artificial neural networks. International Journal of Neural Systems, 1993, 4( 3): 203–222

[14] Lin Y, Nie Z, Ma H. Structural damage detection with automatic feature-extraction through deep learning. Computer-Aided Civil and Infrastructure Engineering, 2017, 32( 12): 1025–1046

[15] Zhong K, Teng S, Liu G, Chen G, Cui F. Structural damage features extracted by convolutional neural networks from mode shapes. Applied Sciences (Basel, Switzerland), 2020, 10( 12): 4247–4262

[16] Teng S, Liu Z, Chen G, Cheng L. Concrete crack detection based on well-known feature extractor model and the YOLO_v2 network. Applied Sciences (Basel, Switzerland), 2021, 11( 2): 813–825

[17] Ghorbanzadeh O, Meena S R, Blaschke T, Aryal J. UAV-based slope failure detection using deep-learning convolutional neural networks. Remote Sensing, 2019, 11( 17): 2046–2069

[18] Badrinarayanan V, Kendall A, Cipolla R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39( 12): 2481–2495

[19] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). Munich: Springer, 2015, 234–241

[20] Chen L C, Zhu Y, Papandreou G, Schroff F, Adam H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: European Conference on Computer Vision (ECCV). Munich: Springer, 2018, 833–851

[21] Shelhamer E, Long J, Darrell T. Fully Convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39( 4): 640–651

[22] Narazaki Y, Hoskere V, Hoang T A, Fujino Y, Sakurai A, Spencer B F Jr. Vision ‐ based automated bridge component recognition with high ‐ level scene consistency. Computer-Aided Civil and Infrastructure Engineering, 2020, 35( 5): 465–482

[23] Liu J, Yang X, Lau S, Wang X, Luo S, Lee V C S, Ding L. Automated pavement crack detection and segmentation based on two-step convolutional neural network. Computer-Aided Civil and Infrastructure Engineering, 2020, 35( 11): 1291–1305

[24] Dung C V, Anh L D. Autonomous concrete crack detection using deep fully convolutional neural network. Automation in Construction, 2019, 99 : 52–58

[25] Teng S, Chen G, Gong P, Liu G, Cui F. Structural damage detection using convolutional neural networks combining strain energy and dynamic response. Meccanica, 2020, 55( 4): 945–959

[26] RojahnCBonneville D RQuadriN DPhippsM TRanousR A RussellJ EStaehlin W ETurnerZ. Postearthquake Safety Evaluation of Buildings. Redwood City, CA: Applied Technology Council, 2005

[27] Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation. In: 2015 IEEE International Conference on Computer Vision (ICCV). Las Condes: IEEE, 2015, 1520–1528

[28] Dong C, Loy C C, Tang X. Accelerating the super-resolution convolutional neural network. In: European Conference on Computer Vision (ECCV). Amsterdam: Springer, 2016, 391–407

[29] Nguyen-Thanh V M, Anitescu C, Alajlan N, Rabczuk T, Zhuang X. Parametric deep energy approach for elasticity accounting for strain gradient effects. Computer Methods in Applied Mechanics and Engineering, 2021, 386 : 114096

[30] Zhuang X, Guo H, Alajlan N, Zhu H, Rabczuk T. Deep autoencoder based energy method for the bending, vibration, and buckling analysis of Kirchhoff plates with transfer learning. European Journal of Mechanics. A, Solids, 2021, 87 : 104225

[31] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV: IEEE, 2016, 770–778

[32] Chen L C, Papandreou G, Kokkinos I, Murphy K, Yuille A L. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40( 4): 834–848

[33] Cha Y J, Choi W, Büyüköztürk O. Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering, 2017, 32( 5): 361–378

[34] He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: 2015 IEEE International Conference on Computer Vision (ICCV). Las Condes: IEEE, 2015, 1026–1034

[35] IoffeSSzegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. 2015, arXiv:1502.03167

[36] Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 2014, 15( 1): 1929–1958

[37] CsurkaGLarlus DPerronninF. What is a good evaluation measure for semantic segmentation? In: Proceedings of the British Machine Vision Conference. Bristol: BMVA, 2013

[38] Randall Wilson D, Martinez T R. The need for small learning rates on large problems. In: International Joint Conference on Neural Networks. Washington, D.C.: IEEE, 2001, 115–119

[39] KroghAHertz J A. A Simple Weight Decay Can Improve Generalization. In: Proceedings of the 4th International Conference on Neural Information Processing Systems (NIPS). Denver: MIT Press, 1991

[40] David EigenR F. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In: IEEE International Conference on Computer Vision (ICCV). Las Condes: IEEE, 2015,

[41] Zhang Y, Yang Y. Cross-validation for selecting a model selection procedure. Journal of Econometrics, 2015, 187( 1): 95–112

Appendix

See Figs. 20 and 21.

Convergence graphs (loss/accuracy vs number of epoch) on training and validation dataset for the CNN model. TD = training dataset. VD = validation dataset.

Confusion matrix for the classification metric with presented Pixel level classification network models. (a) DeepLab v3+ (Resnet18 Sgdm); (b) DeepLab v3+ (Resnet18 Adam); (c) DeepLab v3+ (Resnet50 Sdgm); (d) DeepLab v3+ (Resnet50 Adam); (e) FCN 32s (Sgdm); (f) FCN 32s (Adam); (g) U-Net (Sdgm); (h) U-Net (Adam).

Acknowledgements

The authors would like to express their sincere gratitude to Yang HE, Wei DENG, Ronghao ZHANG, from the Guangdong University of Technology, for labeling the image data.

RIGHTS & PERMISSIONS

2022 Higher Education Press 2022