VSegNet – A Variant SegNet for Improving Segmentation Accuracy in Medical Images with Class Imbalance and Limited Data

Abstract

Deep learning methods for many medical image segmentation task encounter challenges like smaller datasets and class imbalance. This study proposes a variant SegNet (vSegNet) designed to deliver significantly accurate and reliable segmentation results on such datasets. The novelty lies in designing encoder and decoder blocks with an appropriate number of convolution layers and using the Dice score and Hausdorff distance (HD) as compound loss function in learning. This study used public datasets consisting of chest X-rays, axial CT slices, foot ulcer images, and subset of SPIDER dataset to benchmark the segmentation task of the proposed neural network model with other popular networks like U-Net, SegNet, DeepLabv3+, VGG16, MobileNetV2, and fully convolutional network (FCN). For the segmentation of lungs in chest X-rays, vertebral body in CT, augmented data for the previous case, foot ulcer dataset, and segmentation of vertebrae, intervertebral disks, and spinal canal in SPIDER dataset (MRI dataset) respectively, the proposed vSegNet performed with a Dice score of 0.96 ± 0.01, 0.90 ± 0.20, 0.95 ± 0.02, 0.86 ± 0.07, and 0.95 ± 0.01 and the HD of 14.33 ± 7.74, 8.45 ± 7.08, 7.99 ± 6.05, 29.32 ± 25.64, and 8.45 ± 2.81 with respect to the ground truth on the test dataset. These results highlight the effectiveness of the proposed model in delivering both higher segmentation accuracy and improved boundary delineation. The proposed network, vSegNet, has been demonstrated as an effective model for semantic segmentation on class-imbalanced smaller datasets, surpassing all other networks considered in this study in terms of mIoU, BF score, Dice score, HD, accuracy, precision, recall, and F1 score on a variety of anatomical regions and medical imaging modalities.

Keywords

deep neural network / semantic segmentation / Dice score / Hausdorff distance / compound loss function

References

[1] Haralick, R. M., Shanmugam, K., & Dinstein, I. H. (1973). Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, 3(6), 610-621. https://doi.org/10.1109/TSMC.1973.4309314

[2] Gill, G., Toews, M., & Beichel, R. R. (2014). Robust initialization of active shape models for lung segmentation in CT scans: A feature—based atlas approach. International Journal of Biomedical Imaging, 2014(1), 479154. https://doi.org/10.1155/2014/479154

[3] Ronneberger, O., Fischer, P., & Brox, T. (2015). U—Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer—Assisted Intervention, 234-241. https://doi.org/10.1007/978-3-319-24574-4_28

[4] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems, 1-9.

[5] Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., ..., & Li, F. F. (2015). ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115, 211-252. https://doi.org/10.1007/s11263-015-0816-y

[6] Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large—scale image recognition. arXiv Preprint:1409.1556.

[7] Dubey, A. K., & Jain, V. (2020). Automatic facial recognition using VGG16 based transfer learning model. Journal of Information and Optimization Sciences, 41(7), 1589-1596. https://doi.org/10.1080/02522667.2020.1809126

[8] Howard, A. G. (2013). Some improvements on deep convolutional neural network based image classification. arXiv Preprint:1312.5402.

[9] Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017). SegNet: A deep convolutional encoder—decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481-2495. https://doi.org/10.1109/TPAMI.2016.2644615

[10] Theckedath, D., & Sedamkar, R. R. (2020). Detecting affect states using VGG16, ResNet50 and SE—ResNet50 networks. SN Computer Science, 1(2), 79. https://doi.org/10.1007/s42979-020-0114-9

[11] Qassim, H., Verma, A., & Feinzimer, D. (2018). Compressed residual—VGG16 CNN model for big data places image recognition. In 2018 IEEE 8th Annual Computing and Communication Workshop and Conference, 169-175. https://doi.org/10.1109/CCWC.2018.8301729

[12] Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018). DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 834-848. https://doi.org/10.1109/TPAMI.2017.2699184

[13] Chen, L. C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder—decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision, 801-818.

[14] Kim, M., & Lee, B. D. (2021). Automatic lung segmentation on chest X—rays using self—attention deep neural network. Sensors, 21(2), 369. https://doi.org/10.3390/s21020369

[15] Lu, H., Tian, S., Yu, L., Liu, L., Cheng, J., Wu, W., ..., & Zhang, D. (2022). DCACNet: Dual context aggregation and attention—guided cross deconvolution network for medical image segmentation. Computer Methods and Programs in Biomedicine, 214, 106566. https://doi.org/10.1016/j.cmpb.2021.106566

[16] Oktay, O., Schlemper, J., Folgoc, L. L., Lee, M., Heinrich, M., Misawa, K., ..., & Rueckert, D. (2018). Attention U—Net: Learning where to look for the pancreas. arXiv Preprint:1804.03999.

[17] Hansen, S., Gautam, S., Jenssen, R., & Kampffmeyer, M. (2022). Anomaly detection—inspired few—shot medical image segmentation through self—supervision with supervoxels. Medical Image Analysis, 78, 102385. https://doi.org/10.1016/j.media.2022.102385

[18] Dong, N., Kampffmeyer, M., Liang, X., Xu, M., Voiculescu, I., & Xing, E. (2022). Towards robust partially supervised multi—structure medical image segmentation on small—scale data. Applied Soft Computing, 114, 108074. https://doi.org/10.1016/j.asoc.2021.108074

[19] Hryniowski, A., & Wong, A. (2019). DeepLABNet: End—to—end learning of deep radial basis networks. Journal of Computational Vision and Imaging Systems, 5(1), 1—1.

[20] Cui, X., Chang, S., Li, C., Kong, B., Tian, L., Wang, H., ..., & Li, Z. (2021). DEAttack: A differential evolution based attack method for the robustness evaluation of medical image segmentation. Neurocomputing, 465, 38-52. https://doi.org/10.1016/j.neucom.2021.08.118

[21] Zhou, Q., Wang, Q., Bao, Y., Kong, L., Jin, X., & Ou, W. (2022). LAEDNet: A lightweight attention encoder—decoder network for ultrasound medical image segmentation. Computers and Electrical Engineering, 99, 107777. https://doi.org/10.1016/j.compeleceng.2022.107777

[22] Tao, G., Li, H., Huang, J., Han, C., Chen, J., Ruan, G., ..., & Cai, H. (2022). SeqSeg: A sequential method to achieve nasopharyngeal carcinoma segmentation free from background dominance. Medical Image Analysis, 78, 102381. https://doi.org/10.1016/j.media.2022.102381

[23] Tang, P., Yang, P., Nie, D., Wu, X., Zhou, J., & Wang, Y. (2022). Unified medical image segmentation by learning from uncertainty in an end—to—end manner. Knowledge—Based Systems, 241, 108215. https://doi.org/10.1016/j.knosys.2022.108215

[24] Milletari, F., Navab, N., & Ahmadi, S. A. (2016). V—Net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 Fourth International Conference on 3D Vision, 565-571. https://doi.org/10.1109/3DV.2016.79

[25] Karimi, D., & Salcudean, S. E. (2020). Reducing the Hausdorff distance in medical image segmentation with convolutional neural networks. IEEE Transactions on Medical Imaging, 39(2), 499-513. https://doi.org/10.1109/TMI.2019.2930068

[26] Baumgartner, C. F., Koch, L. M., Pollefeys,, M., & Konukoglu, E. (2018). An exploration of 2D and 3D deep learning techniques for cardiac MR image segmentation. In Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges, 111-119. https://doi.org/10.1007/978-3-319-75541-0_12

[27] Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., ..., & Sánchez, C. I. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42, 60-88. https://doi.org/10.1016/j.media.2017.07.005

[28] Yeung, M., Sala, E., Schönlieb, C. B., & Rundo, L. (2022). Unified focal loss: Generalising dice and cross entropy—based losses to handle class imbalanced medical image segmentation. Computerized Medical Imaging and Graphics, 95, 102026. https://doi.org/10.1016/j.compmedimag.2021.102026

[29] Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. arXiv Preprint: 1708.02002.

[30] Jaeger, S., Karargyris, A., Candemir, S., Folio, L., Siegelman, J., Callaghan, F., ..., & McDonald, C. J. (2014). Automatic tuberculosis screening using chest radiographs. IEEE Transactions on Medical Imaging, 33(2), 233-245. https://doi.org/10.1109/TMI.2013.2284099

[31] Jaeger, S., Karargyris, A., Candemir, S., Siegelman, J., Folio, L., Antani, S., ..., & McDonald, C. J. (2013). Automatic screening for tuberculosis in chest radiographs: A survey. Quantitative Imaging in Medicine and Surgery, 3(2), 89-99. https://doi.org/10.3978/j.issn.2223-4292.2013.04.03

[32] Candemir, S., Jaeger, S., Palaniappan, K., Musco, J. P., Singh, R. K., Xue, Z., ..., & McDonald, C. J. (2014). Lung segmentation in chest radiographs using anatomical atlases with nonrigid registration. IEEE Transactions on Medical Imaging, 33(2), 577-590. https://doi.org/10.1109/TMI.2013.2290491

[33] Rajaraman, S., Folio, L. R., Dimperio, J., Alderson, P. O., & Antani, S. K. (2021). Improved semantic segmentation of tuberculosis—Consistent findings in chest X—rays using augmented training of modality—specific U—Net models with weak localizations. Diagnostics, 11(4), 616. https://doi.org/10.3390/diagnostics11040616

[34] Jaeger, S., Candemir, S., Antani, S., Wáng, Y. X. J., Lu, P. X., & Thoma, G. (2014). Two public chest X—ray datasets for computer—aided screening of pulmonary diseases. Quantitative Imaging in Medicine and Surgery, 4(6), 475-477. https://doi.org/10.3978/j.issn.2223-4292.2014.11.20

[35] Aslan, M. S., Ali, A., Rara, H., Arnold, B., Farag, A. A., Fahmi, R., & Xiang, P. (2009). A novel 3D segmentation of vertebral bones from volumetric CT images using graph cuts. In Advances in Visual Computing: 5th International Symposium, 519-528. https://doi.org/10.1007/978-3-642-10520-3_49

[36] Aslan, M. S., Shalaby, A., & Farag, A. A. (2013). Clinically desired segmentation method for vertebral bodies. In 2013 IEEE 10th International Symposium on Biomedical Imaging, 840-843. https://doi.org/10.1109/ISBI.2013.6556606

[37] Wang, C., Anisuzzaman, D. M., Williamson, V., Dhar, M. K., Rostami, B., Niezgoda, J., ..., & Yu, Z. (2020). Fully automatic wound segmentation with deep convolutional neural networks. Scientific Reports, 10(1), 21897. https://doi.org/10.1038/s41598-020-78799-w

[38] van der Graaf, J. W., van Hooff, M. L., Buckens, C. F., Rutten, M., van Susante, J. L., Kroeze, R. J., ..., & Lessmann, N. (2024). Lumbar spine segmentation in MR images: A dataset and a public benchmark. Scientific Data, 11(1), 264. https://doi.org/10.1038/s41597-024-03090-w

[39] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. C. (2018). MobileNetV2: Inverted residuals and linear bottlenecks. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4510-4520. https://doi.org/10.1109/CVPR.2018.00474

[40] Shelhamer, E., Long, J., & Darrell, T. (2017). Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 640-651. https://doi.org/10.1109/TPAMI.2016.2572683

[41] Gaggion, N., Mansilla, L., Mosquera, C., Milone, D. H., & Ferrante, E. (2023). Improving anatomical plausibility in medical image segmentation via hybrid graph neural networks: Applications to chest X—ray analysis. IEEE Transactions on Medical Imaging, 42(2), 546-556. https://doi.org/10.1109/TMI.2022.3224660

[42] de Almeida, P. A. C., & Borges, D. L. (2023). A deep unsupervised saliency model for lung segmentation in chest X—ray images. Biomedical Signal Processing and Control, 86, 105334. https://doi.org/10.1016/j.bspc.2023.105334

[43] Junia, R. C., & Selvan, K. (2024). Deep learning—based automatic segmentation of COVID—19 in chest X—ray images using ensemble neural net sentinel algorithm. Measurement: Sensors, 33, 101117. https://doi.org/10.1016/j.measen.2024.101117

[44] Li, Y., Liang, W., Zhang, Y., & Tan, J. (2018). Automatic global level set approach for lumbar vertebrae CT image segmentation. BioMed Research International, 2018(1), 6319879. https://doi.org/10.1155/2018/6319879

[45] Wang, Z., Xiao, P., & Tan, H. (2023). Spinal magnetic resonance image segmentation based on U—Net. Journal of Radiation Research and Applied Sciences, 16(3), 100627. https://doi.org/10.1016/j.jrras.2023.100627

[46] Laiwalla, A. N., Ratnaparkhi, A., Zarrin, D., Cook, K., Li, I., Wilson, B., ..., & Macyszyn, L. (2023). Lumbar spinal canal segmentation in cases with lumbar stenosis using deep—U—Net ensembles. World Neurosurgery, 178, e135-e140. https://doi.org/10.1016/j.wneu.2023.07.009

[47] Scebba, G., Zhang, J., Catanzaro, S., Mihai, C., Distler, O., Berli, M., & Karlen, W. (2022). Detect—and—segment: A deep learning approach to automate wound image segmentation. Informatics in Medicine Unlocked, 29, 100884. https://doi.org/10.1016/j.imu.2022.100884

[48] Yap, M. H., Cassidy, B., Byra, M., Liao, T. Y., Yi, H., Galdran, A., ..., & Kendrick, C. (2024). Diabetic foot ulcers segmentation challenge report: Benchmark and analysis. Medical Image Analysis, 94, 103153. https://doi.org/10.1016/j.media.2024.103153