The development of neural networks with intra- and inter-block dense connections for improved liver tumor segmentation

Abstract

Automatic liver tumor segmentation is an essential part of computer-aided diagnosis systems. Despite the significant progress made by fully convolutional neural networks (FCNs) in recent years, existing methods fail to effectively segment multiscale tumors and accurately delineate tumor boundaries. This indicates that the single-connection network structure overly relies on the high-level semantic information of the image but fails to fully utilize the spatial information of the target across different levels. To solve these problems, we introduce a novel end-to-end segmentation network, named intra- and inter-block densely connected FCN (I2-DenseFCN). This network comprising intra-block and inter-block dense connections in the encoder and decoder, respectively, is developed to effectively handle the segmentation of multiscale tumors. We also designed a hybrid loss function to achieve more accurate tumor boundary delineation by adaptively adjusting the optimization of the cross-entropy and Lovász Softmax loss. In addition, we are the first to propose an intensity-adaptive image pre-processing method that effectively mitigates the differences between real-world medical scenarios and model inference for liver tumor segmentation tasks. Our training data, derived from the Liver Tumor Segmentation Challenge (LiTS) public dataset, have been expanded to 25,755 samples of liver and their tumors through data augmentation techniques. Experimental evaluations on the LiTS and 3DIRCADb databases demonstrated the superiority of the proposed I2-DenseFCN over classical methods. We believe this research will contribute to the field of tumor segmentation using AI-based medical image processing methods, which is a promising application in computer-aided diagnosis systems.

Keywords

Liver tumor segmentation / Deep learning / Multiscale tumors / Computed tomography imaging / Artificial intelligence / AI diagnostics

Funding

None.

Conflict of interest

The authors declare they have no competing interests.

References

[1] Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018; 68(6):394-424. doi: 10.3322/caac.21492

[2] Dou Q, Yu L, Chen H, et al. 3D deeply supervised network for automated segmentation of volumetric medical images. Med Image Anal. 2017; 41:40-54. doi: 10.1016/j.media.2017.05.001

[3] Seo H, Huang C, Bassenne M, Xiao R, Xing L. Modified U-Net (mU-Net) With incorporation of object-dependent high level features for improved liver and liver-tumor segmentation in CT images. IEEE Trans Med Imaging. 2020; 39(5):1316-1325. doi: 10.1109/TMI.2019.2948320

[4] Lippi G, Banfi G, Church S, et al. Preanalytical quality improvement. In pursuit of harmony, on behalf of European Federation for Clinical Chemistry and Laboratory Medicine (EFLM) Working group for preanalytical phase (WG-PRE). Clin Chem Lab Med CCLM. 2015; 53(3):357-370. doi: 10.1515/cclm-2014-1051

[5] Hawkins R. Managing the pre- and post-analytical phases of the total testing process. Ann Lab Med. 2012; 32(1):5-16. doi: 10.3343/alm.2012.32.1.5

[6] Compton CC, Robb JA, Anderson MW, et al. Preanalytics and precision pathology: Pathology practices to ensure molecular integrity of cancer patient biospecimens for precision medicine. Arch Pathol Lab Med. 2019; 143(11):1346-1363. doi: 10.5858/arpa.2019-0009-SA

[7] Gündisch S, Hauck S, Sarioglu H, et al. Variability of protein and phosphoprotein levels in clinical tissue specimens during the preanalytical phase. J Proteome Res. 2012; 11(12):5748-5762. doi: 10.1021/pr300560y

[8] Moghbel M, Mashohor S, Mahmud R, Saripan MIB. Review of liver segmentation and computer assisted detection/ diagnosis methods in computed tomography. Artif Intell Rev. 2018; 50(4):497-537. doi: 10.1007/s10462-017-9550-x

[9] Yin S, You X, Yang X, Peng Q, Zhu Z, Jing XY. A joint space-angle regularization approach for single 4D diffusion image super-resolution. Magn Reson Med. 2018; 80(5):2173-2187. doi: 10.1002/mrm.27184

[10] Soler L, Delingette H, Malandain G, et al. Fully automatic anatomical, pathological, and functional segmentation from CT scans for hepatic surgery. Comput Aided Surg. 2001; 6(3):131-142. doi: 10.1002/igs.1016

[11] Ciecholewski M, Ogiela MR. Automatic Segmentation of Single and Multiple Neoplastic Hepatic Lesions in CT Images. In: International Work-Conference on the Interplay between Natural and Artificial Computation. Germany: Springer; 2007. p. 63-71.

[12] Nugroho HA, Ihtatho D, Nugroho H. Contrast Enhancement for Liver Tumor Identification. In: MICCAI Workshop. Vol. 41. 2008. p. 201.

[13] Smeets D, Stijnen B, Loeckx D, De Dobbelaer B, Suetens P. Segmentation of liver metastases using a level set method with spiral-scanning technique and supervised fuzzy pixel classification. MIDAS J. 2008. doi: 10.54294/dxbugc

[14] Hoogi A, Beaulieu CF, Cunha GM, et al. Adaptive local window for level set segmentation of CT and MRI liver lesions. Med Image Anal. 2017; 37:46-55. doi: 10.1016/j.media.2017.01.002

[15] Qi Y, Xiong W, Leow WK, et al. Semi-automatic segmentation of liver tumors from CT scans using Bayesian rule-based 3D region growing. MIDAS J. 2008. doi: 10.54294/rofibw

[16] Christ PF, Ettlinger F, Grün F, et al. Automatic Liver and Tumor Segmentation of CT and MRI Volumes Using Cascaded Fully Convolutional Neural Networks. In: ArXiv Preprint ArXiv170205970; 2017.

[17] Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: ArXiv Preprint; 2015. doi: 10.48550/arXiv.1505.04597

[18] Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017; 42:60-88. doi: 10.1016/j.media.2017.07.005

[19] Bi L, Kim J, Kumar A, Feng D. Automatic Liver Lesion Detection Using Cascaded Deep Residual Networks. In: ArXiv Preprint ArXiv170402703; 2017.

[20] He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016. p. 770-778.

[21] Yuan Y, Chao M, Lo YC. Automatic skin lesion segmentation using deep fully convolutional networks with Jaccard distance. IEEE Trans Med Imaging. 2017; 36(9):1876-1886. doi: 10.1109/TMI.2017.2695227

[22] Vorontsov E, Tang A, Pal C, Kadoury S. Liver lesion segmentation informed by joint liver segmentation. In: 2018 IEEE 15^th International Symposium on Biomedical Imaging (ISBI 2018). 2018. p. 1332-1335. doi: 10.1109/ISBI.2018.8363817

[23] Hu Q, Chen Y, Xiao J, et al. Label-free Liver Tumor Segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023. p. 7422-7432.

[24] Liu H, Wei D, Lu D, Sun J, Wang L, Zheng Y. M3AE:Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalities. In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol 37. 2023. p. 1657-1665.

[25] Liu J, Zhang Y, Chen JN, et al. Clip-driven Universal Model for Organ Segmentation and Tumor Detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023. p. 21152-21164.

[26] Radford A, Kim JW, Hallacy C, et al. Learning Transferable Visual Models from Natural Language Supervision. In: International Conference on Machine Learning. PMLR; 2021. p. 8748-8763.

[27] Berman M, Triki AR, Blaschko MB. The Lovász-softmax Loss: A Tractable Surrogate for the Optimization of the Intersection-over-union Measure in Neural Networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. p. 4413-4421.

[28] Jabbar R. A study of preprocessing and segmentation techniques on cardiac medical images. Int J Eng. 2014; 3(4):336-341.

[29] Li X, Chen H, Qi X, Dou Q, Fu CW, Heng PA. H-DenseUNet: Hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans Med Imaging. 2018; 37(12):2663-2674. doi: 10.1109/TMI.2018.2845918

[30] Han X. Automatic Liver Lesion Segmentation Using a Deep Convolutional Neural Network Method. In: ArXiv Preprint ArXiv170407239; 2017.

[31] Lee H, Kim M, Do S. Practical Window Setting Optimization for Medical Image Deep Learning. In: ArXiv Preprint ArXiv181200572; 2018.

[32] Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely Connected Convolutional Networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017. p. 4700-4708.

[33] Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell. 2018; 40(4):834-848. doi: 10.1109/TPAMI.2017.2699184

[34] Bilinski P, Prisacariu V. Dense Decoder Shortcut Connections for Single-pass Semantic Segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. p. 6596-6605.

[35] Milletari F, Navab N, Ahmadi SA. V-net:Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV). United States: IEEE; 2016. p. 565-571.

[36] Bilic P, Christ P, Li HB, et al. The Liver Tumor Segmentation Benchmark (LiTS). Med Image Anal. 2023; 84:102680. doi: 10.1016/j.media.2022.102680

[37] Paszke A, Gross S, Chintala S, et al. Automatic Differentiation in Pytorch. In: Conference on Neural Information Processing Systems; 2017.

[38] Kingma DP, Ba J. Adam:A Method for Stochastic Optimization. In: ArXiv Preprint ArXiv14126980; 2014.

[39] Chlebus G, Meine H, Moltz JH, Schenk A. Neural Network-based Automatic Liver Tumor Segmentation with Random Forest-based Candidate Filtering. In: ArXiv Preprint ArXiv170600842; 2017.

[40] Bellver M, Maninis KK, Pont-Tuset J, Giró-i-Nieto X, Torres J, Van Gool L. Detection-Aided Liver Lesion Segmentation Using Deep Learning. In: ArXiv Preprint ArXiv171111069; 2017.

[41] Chlebus G, Schenk A, Moltz JH, van Ginneken B, Hahn HK, Meine H. Automatic liver tumor segmentation in CT with fully convolutional neural networks and object-based postprocessing. Sci Rep. 2018; 8(1):15497. doi: 10.1038/s41598-018-33860-7

[42] Chen X, Zhang R, Yan P. Feature Fusion Encoder Decoder Network for Automatic Liver Lesion Segmentation. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019); 2019. p. 430-433. doi: 10.1109/ISBI.2019.8759555