Automatic road extraction framework based on codec network

Abstract

Road extraction based on deep learning is one of hot spots of semantic segmentation in the past decade. In this work, we proposed a framework based on codec network for automatic road extraction from remote sensing images. Firstly, a pre-trained ResNet34 was migrated to U-Net and its encoding structure was replaced to deepen the number of network layers, which reduces the error rate of road segmentation and the loss of details. Secondly, dilated convolution was used to connect the encoder and the decoder of network to expand the receptive field and retain more low-dimensional information of the image. Afterwards, the channel attention mechanism was used to select the information of the feature image obtained by up-sampling of the encoder, the weights of target features were optimized to enhance the features of target region and suppress the features of background and noise regions, and thus the feature extraction effect of the remote sensing image with complex background was optimized. Finally, an adaptive sigmoid loss function was proposed, which optimizes the imbalance between the road and the background, and makes the model reach the optimal solution. Experimental results show that compared with several semantic segmentation networks, the proposed method can greatly reduce the error rate of road segmentation and effectively improve the accuracy of road extraction from remote sensing images.

Keywords

remote sensing image / road extraction / ResNet34 / U-Net / channel attention mechanism / sigmoid loss function

References

[1] LU X Y, ZHONG Y F, ZHENG Z, et al. Multi-scale and multi-task deep learning framework for automatic road extraction. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11): 9362-9377.

[2] HAN J W, ZHANG D W, CHENG G, et al. Object detection in optical remote sensing images based on weakly supervised learning and high-level feature learning. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(6): 3325-3337.

[3] ROMERO A, GATTA C, CAMPS-VALLS G. Unsupervised deep feature extraction for remote sensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(3): 1349-1362.

[4] GUO Z L, SHAO X W, XU Y W, et al. Identification of village building via google earth images and supervised machine learning methods. Remote Sensing, 2016, 8(4): 271.

[5] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation//The IEEE Conference on Computer Vision and Pattern Recognition, June 7-12, 2015, Boston, USA. New York: 2015: 3431-3440.

[6] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.

[7] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.

[8] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848.

[9] PAPANDREOU G, CHEN L C, MURPHY K P, et al. Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation//2015 IEEE International Conference on Computer Vision, December 7-13, 2015, Santiago, Chile. New York: IEEE, 2015: 1742-1750.

[10] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation //Lecture Notes in Computer Science. Cham: Springer International Publishing, 2015: 234-241.

[11] GAO X R, CAI Y H, QIU C Y, et al. Retinal blood vessel segmentation based on the Gaussian matched filter and U-Net.//2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics. Shanghai, China. New York: IEEE, 2017: 1-5.

[12] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition, June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE, 2016: 770-778.

[13] HE K M, ZHANG X Y, REN S Q, et al. Identity mappings in deep residual networks//European Conference on Computer Vision, October 11–14, 2016, Amsterdam, The Netherlands. Cham: Springer, 2016: 630-645.

[14] LIU Z, ZHANG X L, SONG Y Q, et al. Liver segmentation with improved U-Net and Morphsnakes algorithm. Journal of Image and Graphics, 2018, 23(8): 1254-1262.

[15] JIN F, WANG L F, LIU Z, et al. Double U-Net remote sensing image road extraction method. Journal of Geomatics Science and Technology, 2019, 36(4): 377-381.

[16] SHANKARANARAYANA S M, RAM K, MITRA K, et al. Joint optic disc and cup segmentation using fully convolutional and adversarial networks//International Workshop on Ophthalmic Medical Image Analysis, International Workshop on Fetal and Infant Image Analysis. September 14, 2017, Québec City, QC, Canada. Cham: Springer, 2017: 168-176.

[17] ALOM M Z, YAKOPCIC C, TAHA T M, et al. Nuclei segmentation with recurrent residual convolutional neural networks based U-Net (R2U-Net)// IEEE National Aerospace and Electronics Conference,July 23-26, 2018, Dayton, OH, USA. New York: IEEE, 2018: 228-233.

[18] OKTAY O, SCHLEMPER J, LE FOLGOC L, et al. Attention U-Net: learning where to look for the pancreas. 2018: 1804.03999.

[19] ZHANG Z X, LIU Q J, WANG Y H. Road extraction by deep residual U-Net. IEEE Geoscience and Remote Sensing Letters, 2018, 15(5): 749-753.

[20] LIU Z H, WU J Z, FU L S, et al. Improved kiwifruit detection using pre-trained VGG16 with RGB and NIR information fusion. IEEE Access, 2019, 8: 2327-2336.

[21] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition. ArXiv e-Prints, 2014: arXiv: 1409.1556.

[22] HU J, SHEN L, SUN G. Squeeze-and-excitation networks//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA. New York: IEEE, 2018: 7132-7141.

[23] CLEVERT D A, UNTERTHINER T, HOCHREITER S. Fast and accurate deep network learning by exponential linear units (ELUs). 2015: 1511.07289.

[24] DE BOER P T, KROESE D P, MANNOR S, et al. A tutorial on the cross-entropy method. Annals of Operations Research, 2005, 134(1): 19-67.

[25] CRESWELL A, ARULKUMARAN K, BHARATH A A. On denoising autoencoders trained to minimise binary cross-entropy. 2017: 1708.08487.

[26] DEMIR I, KOPERSKI K, LINDENBAUM D, et al. DeepGlobe 2018: a challenge to parse the earth through satellite images. 2018: 1805.06561.

[27] HE K M, ZHANG X Y, REN S Q, et al. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification//2015 IEEE International Conference on Computer Vision (ICCV), December 7-13, 2015, Santiago, Chile. New York: IEEE, 2015: 1026-1034.,

[28] MILLETARI F, NAVAB N, AHMADI S A. V-net: fully convolutional neural networks for volumetric medical image segmentation. 2016 Fourth International Conference on 3D Vision,October 25-28, 2016, Stanford, CA, USA. New York: IEEE, 2016: 565-571.