A fusion deep learning framework based on breast cancer grade prediction

Abstract

In breast cancer grading, the subtle differences between HE-stained pathological images and the insufficient number of data samples lead to grading inefficiency. With its rapid development, deep learning technology has been widely used for automatic breast cancer grading based on pathological images. In this paper, we propose an integrated breast cancer grading framework based on a fusion deep learning model, which uses three different convolutional neural networks as submodels to extract feature information at different levels from pathological images. Then, the output features of each submodel are learned by the fusion network based on stacking to generate the final decision results. To validate the effectiveness and reliability of our proposed model, we perform dichotomous and multiclassification experiments on the Invasive Ductal Carcinoma (IDC) pathological image dataset and a generated dataset and compare its performance with those of the state-of-the-art models. The classification accuracy of the proposed fusion network is 93.8%, the recall is 93.5%, and the F1 score is 93.8%, which outperforms the state-of-the-art methods.

Keywords

Breast cancer / Grade prediction / Fusion framework / Convolutional neural networks

References

[1] H. Yang, J.-Y. Kim, H. Kim, S.P. Adhikari,Guided soft attention network for clas-sification of breast cancer histopathology images, IEEE Trans. Med. Imaging 39 (5)(2019) 1306-1315.

[2] S. Karimi Jafarbigloo, H. Danyali, Nuclear atypia grading in breast cancer histopathological images based on cnn feature extraction and lstm classification, CAAI Trans. Intell. Technol. 6(4) (2021) 426-439.

[3] A. Sohail, A. Khan, H. Nisar, S. Tabassum, A. Zameer, Mitotic nuclei analysis in breast cancer histopathology images using deep ensemble classifier, Med. Image Anal. 72 (2021) 102-121.

[4] E. Agostinetto, F. Jacobs, V. Debien, A. De Caluwé, C.-F. Pop, X. Catteau, P. Aftimos, E. de Azambuja, L. Buisseret, Post-neoadjuvant treatment strategies for patients with early breast cancer, Cancers 14 (21) (2022) 5467.

[5] S. Sharma, R. Mehra, S. Kumar, Optimised cnn in conjunction with efficient pool-ing strategy for the multi-classification of breast cancer, IET Image Process. 15 (4)(2021) 936-946.

[6] R. Singh, T. Ahmed, A. Kumar, A.K. Singh, A.K. Pandey, S.K. Singh, Imbalanced breast cancer classification using transfer learning, ACM Trans. Comput. Biol. Bioin-form. 18 (1) (2020) 83-93.

[7] A. Abdelli, R. Saouli, K. Djemal, I. Youkana, Combined datasets for breast cancer grading based on multi-cnn architectures, in: 2020 Tenth International Conference on Image Processing Theory, Tools and Applications (IPTA), 2020, pp. 1-7.

[8] B. Xu, J. Liu, X. Hou, B. Liu, J. Garibaldi, I.O. Ellis, A. Green, L. Shen, G. Qiu,Atten-tion by selection: a deep selective attention approach to breast cancer classification, IEEE Trans. Med. Imaging 39 (6) (2019) 1930-1941.

[9] Y. Wang, B. Acs, S. Robertson, B. Liu, L. Solorzano, C. Wählby, J. Hartman, M. Rantalainen, Improved breast cancer histological grading using deep learning, Ann. Oncol. 33 (1) (2022) 89-98.

[10] K. Shankar, E. Perumal, A novel hand-crafted with deep learning features based fusion model for Covid-19 diagnosis and classification using chest x-ray images, Complex Intell. Syst. 7(3) (2021) 1277-1293.

[11] N. Choeda, J. Pruthi, Resnet fusion model for multi-class classification of chest x-rays, in: 2022 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COM-IT-CON), 2022, pp. 425-431.

[12] S. Garg, P. Singh, Transfer learning based lightweight ensemble model for imbal-anced breast cancer classification, ACM Trans. Comput. Biol. Bioinform. 20 (2)(2023) 1529-1539.

[13] S.-H. Wang, D.R. Nayak, D.S. Guttery, X. Zhang, Y.-D. Zhang, Covid-19 classification by ccshnet with deep fusion using transfer learning and discriminant correlation analysis, Inf. Fusion 68 (2021) 131-148.

[14] Z. Jiang, Z. Dong, L. Wang, W. Jiang, Method for diagnosis of acute lymphoblastic leukemia based on vit-cnn ensemble model, Comput. Intell. Neurosci. (2021).

[15] M. Shorfuzzaman, An explainable stacked ensemble of deep learning models for im-proved melanoma skin cancer detection, Multimed. Syst. 28 (4) (2022) 1309-1323.

[16] G. Aresta, T. Araújo, S. Kwok, S.S. Chennamsetty, M. Safwan, V. Alex, B. Marami, M. Prastawa, M. Chan, M. Donovan, et al., Bach: grand challenge on breast cancer histology images, Med. Image Anal. 56 (2019) 122-139.

[17] R. Yan, J. Li, X. Rao, Z. Lv, C. Zheng, J. Dou, X. Wang, F. Ren, F. Zhang,Nanet: nuclei-aware network for grading of breast cancer in He stained pathological images, in: 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2020, pp. 865-870.

[18] A. Basavanhally, S. Ganesan, M. Feldman, N. Shih, C. Mies, J. Tomaszewski, A. Madabhushi, Multi-field-of-view framework for distinguishing tumor grade in er+ breast cancer from entire histopathology slides, IEEE Trans. Biomed. Eng. 60 (8)(2013) 2089-2099.

[19] T. Wan, J. Cao, J. Chen, Z. Qin, Automated grading of breast cancer histopathology using cascaded ensemble with combination of multi-level image features, Neuro-computing 229 (2017) 34-44.

[20] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2016, pp. 770-778.

[21] G. Huang, Z. Liu, L. Van Der Maaten, K.Q. Weinberger, Densely connected convo-lutional networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2017, pp. 4700-4708.

[22] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception ar-chitecture for computer vision, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2016, pp. 2818-2826.

[23] S. Majumdar, P. Pramanik, R. Sarkar, Gamma function based ensemble of cnn mod-els for breast cancer detection in histopathology images, Expert Syst. Appl. 213 (2023) 119022.

[24] M. Ragab, A. Albukhari, J. Alyami, R.F. Mansour, Ensemble deep-learning-enabled clinical decision support system for breast cancer diagnosis and classification on ultrasound images, Biology 11 (3) (2022) 439.