Deep learning with attention modules and residual transformations improves hepatocellular carcinoma (HCC) differentiation using multiphase CT

Abstract

Background: We hypothesize generative adversarial networks (GAN) combined with self-attention (SA) and aggregated residual transformations (ResNeXt) perform better than conventional deep learning models in differentiating hepatocellular carcinoma (HCC). Attention modules facilitate concentrating on salient features and suppressing redundancies, while residual transformations can reuse relevant features. Therefore, we aim to propose a GAN+SA+ResNeXt deep learning model to improve HCC prediction accuracy.

Methods: 228 multiphase CTs from 57 patients were retrospectively analyzed with local IRB's approval, where 30 patients were pathologically confirmed with HCC and the rest 27 were non-HCC. Pre-processing of automatic liver segmentation and Hounsfield unit (HU) normalization was performed, followed by deep learning training with five-fold cross validation in a conventional 3D GAN, a 3D GAN+A, and a 3D GAN+A+ ResNeXt, respectively (training: testing ∼ 4:1). Area under receiver operating characteristics curves (AUROC), accuracy, sensitivity and specificity of HCC prediction were evaluated.

Results: Results showed the proposed method had larger AUROC (95%), better accuracy (91%) and sensitivity (93%) with acceptable specificity (88%) and prediction time (0.04s). Deep GAN with attentions and residual transformations for HCC diagnosis using multiphase CT is feasible and favorable with improved accuracy and efficiency, which harbors clinical potentials in differentiating HCC from other benign or malignant liver lesions.

Keywords

dynamic contrast enhanced (DCE) CT / generative adversarial networks (GAN) / hepatocellular carcinoma (HCC)

References

[1] Rahib L, Smith BD, Aizenberg R, Rosenzweig AB, Fleshman JM, Matrisian LM. Projecting cancer incidence and deaths to 2030: the unexpected burden of thyroid, liver, and pancreas cancers in the United States. Cancer Res. 2014; 74(11): 2913-2921.

[2] Ehman EC, Behr SC, Umetsu SE et al. Rate of observation and inter-observer agreement for LI-RADS major features at CT and MRI in 184 pathology proven hepatocellular carcinomas. Abdom Radiol (NY) 2016; 41: 963–969.

[3] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521(7553): 436–444.

[4] Yasaka K, Akai H, Abe O, Kiryu S. Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study. Radiology 2018; 286: 887–896.

[5] Chernyak V, Fowler KJ, Kamaya A, et al. Liver Imaging Reporting and Data System (LI-RADS) Version 2018: Imaging of Hepatocellular Carcinoma in At-Risk Patients. Radiology. 2018; 289(3): 816-830.

[6] Hanna RF, Miloushev VZ, Tang A, et al. Comparative 13-year meta-analysis of the sensitivity and positive predictive value of ultrasound, CT, and MRI for detecting hepatocellular carcinoma. Abdom Radiol (NY). 2016; 41(1): 71-90.

[7] Yaghmai V, Besa C, Kim E, Gatlin JL, Siddiqui NA, Taouli B. Imaging assessment of hepatocellular carcinoma response to locoregional and systemic therapy. AJR Am J Roentgenol. 2013; 201(1): 80-96.

[8] Roberts LR, Sirlin CB, Zaiem F et al. Imaging for the diagnosis of hepatocellular carcinoma: a systematic review and meta-analysis. Hepatology 2018; 67: 401–421.

[9] Choi JY, Lee JM, Sirlin CB. CT and MR imaging diagnosis and staging of hepatocellular carcinoma: part II. Extracellular agents, hepatobiliary agents, and ancillary imaging features. Radiology 2014; 273: 30-50.

[10] Abajian A, Murali N, Savic LJ, et al. Predicting Treatment Response to Intra-arterial Therapies for Hepatocellular Carcinoma with the Use of Supervised Machine Learning-An Artificial Intelligence Concept. J Vasc Interv Radiol. 2018; 29(6): 850-857.e1.

[11] Nam D, Chapiro J, Paradis V, Seraphin TP, Kather JN. Artificial intelligence in liver diseases: Improving diagnostics, prognostics and response prediction. JHEP Rep. 2022; 4(4): 100443.

[12] Bi WL, Hosny A, Schabath MB, et al. Artificial intelligence in cancer imaging: clinical challenges and applications. CA Cancer J Clin 2019; 69: 127–157.

[13] Welch ML, McIntosh C, Haibe-Kains B, et al. Vulnerabilities of radiomic signature development: The need for safeguards. Radiother Oncol. 2019; 130: 2-9.

[14] Wei J, Jiang H, Zeng M, et al. Prediction of Microvascular Invasion in Hepatocellular Carcinoma via Deep Learning: A Multi-Center and Prospective Validation Study. Cancers (Basel). 2021; 13(10): 2368.

[15] Kim DW, Lee G, Kim SY, et al. Deep learning-based algorithm to detect primary hepatic malignancy in multiphase CT of patients at high risk for HCC. Eur Radiol. 2021; 31(9): 7047-7057.

[16] Saillard C, Schmauch B, Laifa O, et al. Predicting survival after hepatocellular carcinoma resection using deep learning on histological slides. Hepatology. 2020; 72(6): 2000-2013.

[17] Cheng N, Ren Y, Zhou J, et al. Deep learning-based classification of hepatocellular nodular lesions on whole-slide histopathologic Images. Gastroenterology. 2022; 162(7): 1948-1961.e7.

[18] Chaudhary K, Poirion OB, Lu L, Garmire LX. Deep Learning-Based Multi-Omics Integration Robustly Predicts Survival in Liver Cancer. Clin Cancer Res. 2018; 24(6): 1248-1259.

[19] Liu QP, Xu X, Zhu FP, Zhang YD, Liu XS. Prediction of prognostic risk factors in hepatocellular carcinoma with transarterial chemoembolization using multi-modal multi-task deep learning. EClinicalMedicine. 2020; 23: 100379.

[20] Wei L, Owen D, Rosen B, et al. A deep survival interpretable radiomics model of hepatocellular carcinoma patients. Phys Med. 2021; 82: 295-305.

[21] Peng J, Kang S, Ning Z, et al. Residual convolutional neural network for predicting response of transarterial chemoembolization in hepatocellular carcinoma from CT imaging. Eur Radiol. 2020; 30(1): 413-424.

[22] Fu S, Lai H, Huang M, et al. Multi-task deep learning network to predict future macrovascular invasion in hepatocellular carcinoma. EClinicalMedicine. 2021; 42: 101201.

[23] Wang W, Chen Q, Iwamoto Y, et al. Deep Learning-Based Radiomics Models for Early Recurrence Prediction of Hepatocellular Carcinoma with Multi-phase CT Images and Clinical Data. Annu Int Conf IEEE Eng Med Biol Soc. 2019; 2019: 4881–4884.

[24] Gu J, Zhao Z, Zeng Z, et al. Multi-Phase Cross-modal Learning for Noninvasive Gene Mutation Prediction in Hepatocellular Carcinoma. Annu Int Conf IEEE Eng Med Biol Soc. 2020; 2020: 5814–5817.

[25] Chen WF, Ou HY, Liu KH, et al. In-Series U-Net Network to 3D Tumor Image Reconstruction for Liver Hepatocellular Carcinoma Recognition. Diagnostics (Basel). 2020; 11(1): 11.

[26] Ouhmich F, Agnus V, Noblet V, Heitz F, Pessaux P. Liver tissue segmentation in multiphase CT scans using cascaded convolutional neural networks. Int J Comput Assist Radiol Surg. 2019; 14(8): 1275-1284.

[27] Feng B, Ma XH, Wang S, Cai W, Liu XB, Zhao XM. Application of artificial intelligence in preoperative imaging of hepatocellular carcinoma: Current status and future perspectives. World J Gastroenterol. 2021; 27(32): 5341-5350.

[28] Oestmann PM, Wang CJ, Savic LJ, et al. Deep learning-assisted differentiation of pathologically proven atypical and typical hepatocellular carcinoma (HCC) versus non-HCC on contrast-enhanced MRI of the liver. Eur Radiol. 2021; 31(7): 4981-4990.

[29] Goodfellow IJ, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Nets. Proceedings of the 27th International Conference on Neural Information Processing Systems 2014; 2: 2672–2680.

[30] Dong X, Lei Y, Wang T, et al. Automatic multiorgan segmentation in thorax CT images using U-net-GAN. Med Phys. 2019; 46(5): 2157-2168.

[31] Dong X, Lei Y, Tian S, et al. Synthetic MRI-aided multi-organ segmentation on male pelvic CT using cycle consistent deep attention network. Radiother Oncol. 2019; 141: 192-199.

[32] Kearney V, Chan JW, Wang T, et al. DoseGAN: a generative adversarial network for synthetic dose prediction using attention-gated discrimination and generation. Sci Rep. 2020; 10(1): 11073.

[33] He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. 2016 IEEE Conf. Comput. Vis. Pattern Recognit., 2016, p. 770–778.

[34] Szegedy C, Ioffe S, Vanhoucke V. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. CoRR 2016;abs/1602.07261.

[35] Xie S, Girshick R, Dollár P, Tu Z, He K. Aggregated Residual Transformations for Deep Neural Networks. 2017 IEEE Conf. Comput. Vis. Pattern Recognit., 2017, p. 5987–5995.

[36] Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM 2012; 60: 84–90.

[37] Schlemper J, Oktay O, Schaap M, et al. Attention gated networks: Learning to leverage salient regions in medical images. Med Image Anal. 2019; 53: 197-207.

[38] Kearney V, Chan JW, Wang T, Perry A, Yom SS, Solberg TD. Attention-enabled 3D boosted convolutional neural networks for semantic CT segmentation using deep supervision. Phys Med Biol. 2019; 64(13): 135001.

[39] Liu Y, Lei Y, Wang T, et al. CBCT-based synthetic CT generation using deep-attention cycleGAN for pancreatic adaptive radiotherapy. Med Phys. 2020; 47(6): 2472-2483.

[40] Zhang Y, Lei Y, Qiu RLJ, et al. Multi-needle Localization with Attention U-Net in US-guided HDR Prostate Brachytherapy. Med Phys. 2020; 47(7): 2735-2745.

[41] Zeng Q, Liu B, Xu Y, Zhou W. An attention-based deep learning model for predicting microvascular invasion of hepatocellular carcinoma using an intra-voxel incoherent motion model of diffusion-weighted magnetic resonance imaging. Phys Med Biol. 2021; 66(18).

[42] Gao F, Qiao K, Yan B, et al. Hybrid network with difference degree and attention mechanism combined with radiomics (H-DARnet) for MVI prediction in HCC. Magn Reson Imaging. 2021; 83: 27-40.

[43] Choi KJ, Jang JK, Lee SS, et al. Development and Validation of a Deep Learning System for Staging Liver Fibrosis by Using Contrast Agent-enhanced CT Images in the Liver. Radiology. 2018; 289(3): 688-697.

[44] Bilic P, Christ PF, Vorontsov E, et al. The Liver Tumor Segmentation Benchmark (LiTS) 2022. ArXiv, abs/1901.04056.

[45] Lecun L, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE 1998; 86(11): 2278-2324.

[46] Woo S, Park J, Lee J Y, et al. CBAM: Convolutional block attention module, Proceedings of the European Conference on Computer Vision (ECCV). 2018: 3–19.

[47] Ioffe S, Szegedy C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. ArXiv 2015; abs/1502.03167.

[48] Chen Y, Fan H, Xu B, et al. Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks With Octave Convolution. 2019 IEEE/CVF Int. Conf. Comput. Vis., 2019, p. 3434–3443.

[49] Hatamizadeh A et al. “UNETR: Transformers for 3D Medical Image Segmentation,” 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022, pp. 1748–1758.

[50] Hamm CA, Wang CJ, Savic LJ, et al. Deep learning for liver tumor diagnosis part I: development of a convolutional neural network classifier for multi-phasic MRI. Eur Radiol. 2019; 29(7): 3338-3347.

[51] Zhou J, Wang W, Lei B, et al. Automatic Detection and Classification of Focal Liver Lesions Based on Deep Convolutional Neural Networks: A Preliminary Study. Front Oncol. 2021; 10: 581210.

[52] Gao R, Zhao S, Aishanjiang K, et al. Deep learning for differential diagnosis of malignant hepatic tumors based on multi-phase contrast-enhanced CT and clinical data. J Hematol Oncol. 2021; 14(1): 154.

[53] Cao SE, Zhang LQ, Kuang SC, et al. Multiphase convolutional dense network for the classification of focal liver lesions on dynamic contrast-enhanced computed tomography. World J Gastroenterol. 2020; 26(25): 3660-3672.

[54] Wang M, Fu F, Zheng B, et al. Development of an AI system for accurately diagnose hepatocellular carcinoma from computed tomography imaging data. Br J Cancer 2021; 125: 1111–1121.

RIGHTS & PERMISSIONS

2025 The Author(s). Precision Radiation Oncology published by John Wiley & Sons Australia, Ltd on behalf of Shandong Cancer Hospital & Institute.