Improved SE-UNet network-based semantic segmentation and extraction of hidden geological significance in geological maps

Abstract

Automatic segmentation and recognition of content and element information in color geological map are of great significance for researchers to analyze the distribution of mineral resources and predict disaster information. This article focuses on color planar raster geological map (geological maps include planar geological maps, columnar maps, and profiles). While existing deep learning approaches are often used to segment general images, their performance is limited due to complex elements, diverse regional features, and complicated backgrounds for color geological map in the domain of geoscience. To address the issue, a color geological map segmentation model is proposed that combines the Felz clustering algorithm and an improved SE-UNet deep learning network (named GeoMSeg). Firstly, a symmetrical encoder-decoder structure backbone network based on UNet is constructed, and the channel attention mechanism SENet has been incorporated to augment the network's capacity for feature representation, enabling the model to purposefully extract map information. The SE-UNet network is employed for feature extraction from the geological map and obtain coarse segmentation results. Secondly, the Felz clustering algorithm is used for super pixel pre-segmentation of geological maps. The coarse segmentation results are refined and modified based on the super pixel pre-segmentation results to obtain the final segmentation results. This study applies GeoMSeg to the constructed dataset, and the experimental results show that the algorithm proposed in this paper has superior performance compared to other mainstream map segmentation models, with an accuracy of 91.89% and a MIoU of 71.91%.

Keywords

Geological map / UNet model / Image segmentation / Semantic segmentation / Pixel pre-segmentation / Clustering algorithm / Attention mechanismDeep learningArtificial intelligence / Geological survey engineering

CRediT authorship contribution statement

Experiment proposal: Kai Ma and Qin-jun Qiu; funding acquisition: Zhong Xie, Ma Kai and Qin-jun Qiu; preliminary research: Jun-jie Liu and Si-qi Lu; data collection: Jun-jie Liu, Miao Tian, Jun-yuan Deng and Ze-hua Huang; experimental design and analysis: Jun-jie Liu and Qin-jun Qiu; writing the original manuscript: Kai Ma, Jun-jie Liu, Qin-jun Qiu; writing-review and editing: Jun-jie Liu, Kai Ma, Qin-jun Qiu. All authors have read and agreed to the published version of the manuscript.

Declaration of competing interest

The authors declare no conflicts of interest.

Acknowledgments

The authors are very grateful to National Geological Archives for providing data at no cost. This study was financially supported by the Natural Science Foundation of China (42301492), the Open Fund of Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering (2022SDSJ04, 2024SDSJ03), and the Opening Fund of Key Laboratory of Geological Survey and Evaluation of Ministry of Education (GLAB 2023ZR01, GLAB2024ZR08) and the Fundamental Research Funds for the Central Universities.

References

[1] Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S. 2012. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11), 2274-2282. doi: 10.1109/TPAMI.2012.120.

[2] Adams R, Bischof L. 1994. Seeded region growing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(6), 641-647. doi: 10.1109/34.295913.

[3] Angulo J, Serra J. 2007. Modelling and segmentation of colour images in polar representations. Image and Vision Computing, 25(4), 475-495. doi: 10.1016/j.imavis.2006.07.018.

[4] Badrinarayanan V, Kendall A, Cipolla R. 2017. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481-2495. doi: 10.1109/TPAMI.2016.2644615.

[5] Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL. 2018. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 834-848. doi: 10.1109/TPAMI.2017.2699184.

[6] Chen WK, Zhou WF, Zhu L, Cao Y, Gu HM, Yu B. 2022. MTDCNet: A 3D multi-threading dilated convolutional network for brain tumor automatic segmentation. Journal of Biomedical Informatics, 133, 104173. doi: 10.1016/j.jbi.2022.104173.

[7] Cheng YZ. 1995. Mean shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(8), 790-799. doi: 10.1109/34.400568.

[8] Chockler H, Farchi E, Godlin B, Novikov S. 2007. Cross-entropy based testing. Formal Methods in Computer Aided Design (FMCAD'07). November 11-14, 2007, Austin, TX, USA. IEEE, 101-108. doi: 10.1109/FAMCAD.2007.19.

[9] Ding L, Tang H, Bruzzone L. 2021. LANet: Local attention embedding to improve the semantic segmentation of remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 59(1), 426-435. doi: 10.1109/TGRS.2020.2994150.

[10] do Valle RF Jr, Siqueira HE, Valera CA, Oliveira CF, Sanches Fernandes LF, Moura JP, Pacheco FAL. 2019. Diagnosis of degraded pastures using an improved NDVI-based remote sensing approach: An application to the Environmental Protection Area of Uberaba River Basin (Minas Gerais, Brazil). Remote Sensing Applications: Society and Environment, 14, 20-33. doi: 10.1016/j.rsase.2019.02.001.

[11] Felzenszwalb PF, Huttenlocher DP. 2004. Efficient graph-based image segmentation. International Journal of Computer Vision, 59(2), 167-181. doi: 10.1023/B:VISI.0000022288.19776.77.

[12] Glorot X, Bordes A, Bengio Y. 2011. Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, 315-323.

[13] Gonzalez RC, Woods RE. 2002. Digital Image Processing (Second Edition). Beijing, Publishing House of Electronics Industry, 455.

[14] Hinton GE, Osindero S, Teh YW. 2006. A fast learning algorithm for deep belief nets. Neural Computation, 18(7), 1527-1554. doi: 10.1162/neco.2006.18.7.1527.

[15] Hosseini-Fard E, Roshandel-Kahoo A, Soleimani-Monfared M, Khayer K, Ahmadi-Fard AR. 2022. Automatic seismic image segmentation by introducing a novel strategy in histogram of oriented gradients. Journal of Petroleum Science and Engineering, 209, 109971. doi: 10.1016/j.petrol.2021.109971.

[16] Hu J, Shen L, Sun G. 2018. Squeeze-and-excitation networks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 7132-7141. doi: 10.1109/CVPR.2018.00745.

[17] Hu JL, Deng JB, Zou SS. 2010. A novel algorithm for color space conversion model from CMYK to LAB. Journal of Multimedia, 5(2),159. doi: 10.4304/jmm.5.2.159-166.

[18] Huang CL, Chen JJ, Chen CJ, Wu YG. 2016. Geological segmentation on UAV aerial image using shape-based LSM with dominant color. 2016 30th International Conference on Advanced Information Networking and Applications Workshops (WAINA). March 23-25, 2016, Crans-Montana, Switzerland. IEEE, 928-933. doi: 10.1109/WAINA.2016.82.

[19] Huang MX, Yu WJ, Zhu DH. 2012. An improved image segmentation algorithm based on the otsu method. 2012 13th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing. August 8-10, 2012, Kyoto, Japan. IEEE, 135-139. doi: 10.1109/SNPD.2012.26.

[20] Ji J, Lu XC, Luo M, Yin MH, Miao QG, Liu XZ. 2020. Parallel fully convolutional network for semantic segmentation. IEEE Access, 9, 673-682. doi: 10.1109/ACCESS.2020.3042254.

[21] Ji XQ, Li Y, Cheng JZ, Yu YH, Wang MJ. 2015. Cell image segmentation based on an improved watershed algorithm. 2015 8th International Congress on Image and Signal Processing (CISP). October 14-16, 2015, Shenyang, China. IEEE, 433-437. doi: 10.1109/CISP.2015.7407919.

[22] Jiang F, Wang G, He P, Zheng CC, Xiao ZY, Wu Y. 2022. Application of canny operator threshold adaptive segmentation algorithm combined with digital image processing in tunnel face crevice extraction. The Journal of Supercomputing, 78(9), 11601-11620. doi: 10.1007/s11227-022-04330-9.

[23] Kapur JN, Sahoo PK, Wong AKC. 1985. A new method for gray-level picture thresholding using the entropy of the histogram. Computer Vision, Graphics, and Image Processing, 29(3), 273-285. doi: 10.1016/0734-189X(85)90125-2.

[24] Kingma DP, Ba J, Hammad MM. 2014. Adam: A method for stochastic optimization: 1412.6980. https://arxiv.org/abs/1412.6980v9.

[25] Kittler J, Illingworth J. 1986. Minimum error thresholding. Pattern Recognition, 19(1), 41-47. doi: 10.1016/0031-3203(86)90030-0.

[26] Lang Y, Zheng D. 2016. An Improved Sobel Edge Detection OperatorAdvances in Intelligent Systems Research", "Proceedings of the 2016 6th International Conference on Mechatronics, Computer and Education Informationization (MCEI 2016). November 11-13, 2016. Shenyang, China. Atlantis Press, 590-593. doi: 10.2991/mcei16.2016.123.

[27] Levachkine S, Velàzquez A, Alexandrov V, Kharinov M. 2002. Semantic analysis and recognition of raster-scanned color cartographic images. Graphics Recognition Algorithms and Applications. Berlin, Heidelberg: Springer Berlin Heidelberg, 178-189. doi: 10.1007/3-540-45868-9_15.

[28] Levinshtein A, Stere A, Kutulakos KN, Fleet DJ, Dickinson SJ, Siddiqi K. 2009. TurboPixels: Fast superpixels using geometric flows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(12), 2290-2297. doi: 10.1109/TPAMI.2009.96.

[29] Leyk S, Boesch R. 2010. Colors of the past: Color image segmentation in historical topographic maps based on homogeneity. GeoInformatica, 14(1), 1-21. doi: 10.1007/s10707-008-0074-z.

[30] Li ES, Zhu SL, Zhu BS, Zhao Y, Xia CG, Song LH. 2009. An adaptive edge-detection method based on the canny operator. 2009 International Conference on Environmental Science and Information Application Technology. July 4-5, 2009, Wuhan, China. IEEE, 465-469. doi: 10.1109/ESIAT.2009.49.

[31] Lin GS, Milan A, Shen CH, Reid I. 2017. RefineNet: Multi-path refinement networks for high-resolution semantic segmentation. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 5168-5177. doi: 10.1109/CVPR.2017.549.

[32] Liu HQ, Yao MB, Xiao XM, Xiong YG. 2023. RockFormer: A Ushaped transformer network for Martian rock segmentation. IEEE Transactions on Geoscience and Remote Sensing, 61, 4600116. doi: 10.1109/TGRS.2023.3235525.

[33] Likas A, Vlassis N, Verbeek JJ. 2003. The global k-means clustering algorithm. Pattern Recognition, 36(2), 451-461. doi: 10.1016/S0031-3203(02)00060-2.

[34] Liu Y, Wu LZ. 2018. High performance geological disaster recognition using deep learning. Procedia Computer Science, 139, 529-536. doi: 10.1016/j.procs.2018.10.237.

[35] Long J, Shelhamer E, Darrell T. 2015. Fully convolutional networks for semantic segmentation. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12, 2015, Boston, MA, USA. IEEE, 3431-3440. doi: 10.1109/CVPR.2015.7298965.

[36] Ma BJ, Pereira JLJ, Oliva D, Liu S, Kuo YH. 2023. Manta ray foraging optimizer-based image segmentation with a two-strategy enhancement. Knowledge-Based Systems, 262, 110247. doi: 10.1016/j.knosys.2022.110247.

[37] Ma ZJ, Mei G. 2021. Deep learning for geological hazards analysis: Data, models, applications, and opportunities. Earth-Science Reviews, 223, 103858. doi: 10.1016/j.earscirev.2021.103858.

[38] Otsu N. 1979. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1), 62-66. doi: 10.1109/TSMC.1979.4310076.

[39] Paszke A, Chaurasia A, Kim S, Culurciello E. 2016. ENet: A deep neural network architecture for real-time semantic segmentation, 1606.02147. https://arxiv.org/abs/1606.02147v1.

[40] Pham DL, Xu CY, Prince JL. 2000. Current methods in medical image segmentation. Annual Review of Biomedical Engineering, 2, 315-337. doi: 10.1146/annurev.bioeng.2.1.315.

[41] Qiu QJ, Ma K, Lv HR, Tao LF, Xie Z. 2023. Construction and application of a knowledge graph for iron deposits using text mining analytics and a deep learning algorithm. Mathematical Geosciences, 55(3), 423-456. doi: 10.1007/s11004-023-10050-4.

[42] Qiu QJ, Tan YJ, Ma K, Tian M, Xie Z, Tao LF. 2023. Geological symbol recognition on geological map using convolutional recurrent neural network with augmented data. Ore Geology Reviews, 153, 105262. doi: 10.1016/j.oregeorev.2022.105262.

[43] Rahimi H, Abedi M, Yousefi M, Bahroudi A, Elyasi GR. 2021. Supervised mineral exploration targeting and the challenges with the selection of deposit and non-deposit sites thereof. Applied Geochemistry, 128, 104940. doi: 10.1016/j.apgeochem.2021.104940.

[44] Rahman MA, Wang Y. 2016. Optimizing intersection-over-union in deep neural networks for image segmentation. Advances in Visual Computing. Cham, Springer International Publishing, 234-244. doi: 10.1007/978-3-319-50835-1_22.

[45] Rauch A, Sartori M, Rossi E, Baland P, Castelltort S. 2019. Trace information extraction (TIE): A new approach to extract structural information from traces in geological maps. Journal of Structural Geology, 126, 286-300. doi: 10.1016/j.jsg.2019.06.007.

[46] Recky M, Leberl F. 2010. Windows detection using K-means in CIE-lab color space. 2010 20th International Conference on Pattern Recognition. August 23-26, 2010, Istanbul, Turkey. IEEE, 356-359. doi: 10.1109/ICPR.2010.96.

[47] Ronneberger O, Fischer P, Brox T. 2015. U-Net:Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015. Cham, Switzerland: Springer International Publishing, 234-241. doi: 10.1007/978-3-319-24574-4_28.

[48] Rosenfeld A. 1981. The max Roberts operator is a hueckel-type edge detector. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-3(1), 101-103. doi: 10.1109/TPAMI.1981.4767056.

[49] Sezgin M, Sankur B. 2004. Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging, 13, 146-165 doi: 10.1117/1.1631315.

[50] Shamir RR, Duchin Y, Kim J, Sapiro G, Harel N. 2019. Continuous dice coefficient: A method for evaluating probabilistic segmentations, 1906. 11031. https://arxiv.org/abs/1906.11031v1.

[51] Song S, Chaudhuri K, Sarwate AD. 2013. Stochastic gradient descent with differentially private updates. 2013 IEEE Global Conference on Signal and Information Processing. December 3-5, 2013, Austin, TX, USA. IEEE, 245-248. doi: 10.1109/GlobalSIP.2013.6736861.

[52] Teimouri N, Dyrmann M, Jørgensen RN. 2019. A novel spatio-temporal FCN-LSTM network for recognizing various crop types using multitemporal radar images. Remote Sensing, 11(8), 990. doi: 10.3390/rs11080990.

[53] Tian M, Ma K, Liu ZH, Qiu QJ, Tan YJ, Xie Z. 2023. Recognition of geological legends on a geological profile via an improved deep learning method with augmented data using transfer learning strategies. Ore Geology Reviews, 153, 105270. doi: 10.1016/j.oregeorev.2022.105270.

[54] Tian Z, Huang WL, He T, He P, Qiao Y. 2016. Detecting text in natural image with connectionist text proposal network. Computer Vision ECCV 2016. Cham, Switzerland: Springer International Publishing, 56-72. doi: 10.1007/978-3-319-46484-8_4.

[55] Tieleman T. 2012. Lecture 6. 5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning, 4(2), 26.

[56] Tremeau A, Borel N. 1997. A region growing and merging algorithm to color segmentation. Pattern Recognition, 30(7), 1191-1203. doi: 10.1016/S0031-3203(96)00147-1.

[57] Ulupinar F, Medioni G. 1990. Refining edges detected by a LoG operator. Computer Vision, Graphics, and Image Processing, 51(3), 275-298. doi: 10.1016/0734-189X(90)90004-F.

[58] Van den Bergh M, Boix X, Roig G, Van Gool L. 2015. SEEDS: Superpixels extracted via energy-driven sampling. International Journal of Computer Vision, 111(3), 298-314. doi: 10.1007/s11263-014-0744-2.

[59] Vincent L, Soille P. 1991. Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(6), 583-598. doi: 10.1109/34.87344.

[60] Wang F, Jiang MQ, Qian C, Yang S, Li C, Zhang HG, Wang XG, Tang XO. 2017. Residual attention network for image classification. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 6450-6458. doi: 10.1109/CVPR.2017.683.

[61] Wei YC, Xiao HX, Shi HH, Jie ZQ, Feng JS, Huang TS. 2018. Revisiting dilated convolution: A simple approach for weakly- and semi-supervised semantic segmentation. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 7268-7277. doi: 10.1109/CVPR.2018.00759.

[62] Xu JL, Wen XP, Zhang HN, Luo DY, Li JB, Xu LL, Yu M. 2020. Automatic extraction of lineaments based on wavelet edge detection and aided tracking by hillshade. Advances in Space Research, 65(1), 506-517. doi: 10.1016/j.asr.2019.09.045.

[63] Yang L, Wu XY, Zhao DW, Li H, Zhai J. 2011. An improved Prewitt algorithm for edge detection based on noised image. 2011 4th International Congress on Image and Signal Processing. October 15-17, 2011, Shanghai, China. IEEE, 1197-1200. doi: 10.1109/CISP.2011.6100495.

[64] Yu F, Koltun V. 2015. Multi-scale context aggregation by dilated convolutions. 1511.07122. https://arxiv.org/abs/1511.07122v3.

[65] Yu HG, Tao JF, Qin CJ, Liu MY, Xiao DY, Sun H, Liu CL. 2022. A novel constrained dense convolutional autoencoder and DNN-based semi-supervised method for shield machine tunnel geological formation recognition. Mechanical Systems and Signal Processing, 165, 108353. doi: 10.1016/j.ymssp.2021.108353.

[66] Zhang WH, Wang X, You W, Chen JF, Dai P, Zhang PB. 2019. RESLS: Region and edge synergetic level set framework for image segmentation. IEEE Transactions on Image Processing, 29, 57-71. doi: 10.1109/TIP.2019.2928134.

[67] Zhang Y, Chen JQ, Li YL. 2022. Segmentation and quantitative analysis of geological fracture: A deep transfer learning approach based on borehole televiewer image. Arabian Journal of Geosciences, 15(3), 300. doi: 10.1007/s12517-022-09536-y.

[68] Zhang YJ. 2006. An overview of image and video segmentation in the last 40 years. Advances in Image and Video Segmentation, IGI Global, 1-16. doi: 10.4018/978-1-59140-753-9.ch001.

[69] Zheng SX, Lu JC, Zhao HS, Zhu XT, Luo ZK, Wang YB, Fu YW, Feng JF, Xiang T, Torr PHS, Zhang L. 2021. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 20-25, 2021, Nashville, TN, USA. IEEE, 6877-6886. doi: 10.1109/CVPR46437.2021.00681.

[70] Zhou TX, Ruan S, Vera P, Canu S. 2022. A Tri-Attention fusion guided multi-modal segmentation network. Pattern Recognition, 124, 108417. doi: 10.1016/j.patcog.2021.108417.

Funding

Fund of Key Laboratory of Geological Survey and Evaluation of Ministry of Education(GLAB 2023ZR01)

Fund of Key Laboratory of Geological Survey and Evaluation of Ministry of Education(GLAB2024ZR08)

the Fundamental Research Funds for the Central Universities.