A relationship-aware mutual learning method for lightweight skin lesion classification^☆

Abstract

In recent years, deep learning has made significant advancements in skin cancer diagnosis. However, most methods prioritize high prediction accuracy without considering the limitations of computational resources, making them impractical for wearable devices. In this case, knowledge distillation has emerged as an effective method, capable of significantly reducing a model's reliance on computational and storage resources. Nonetheless, previous research suffers from two limitations: 1) the student model can only passively receive knowledge from the teacher model, and 2) the teacher model does not effectively model sample relationships during training, potentially hindering the effective transfer of sample relationship-related knowledge during knowledge distillation. To address these issues, we employ two identical student models, each equipped with a sample relationship module. This design ensures that the student models can mutually learn while modeling sample relationships. We conducted extensive experiments on the ISIC 2019 dataset to validate the effectiveness of our method. The results demonstrate that our approach significantly improves the recognition of various types of skin diseases. Compared to state-of-the-art methods, our approach exhibits higher accuracy and better generalization capabilities.

Keywords

Skin cancer diagnosis / Wearable devices / Knowledge distillation / Mutual learning / Sample relationship

CRediT authorship contribution statement

Peng Liu: Writing - review & editing, Writing - original draft. Wenhua Qian: Methodology. Huaguang Li: Writing - review & editing. Jinde Cao: Methodology.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This work was supported by the Research Foundation of Yunnan Province No. 202001BB050043 and 202105AF150011, National Natural Science Foundation of China under Grants No. 62162065, Provincial Foundation for Leaders of Disciplines in Science and Technology No. 2019HB121.

References

[1] Z. Apalla, A. Lallas, E. Sotiriou, E. Lazaridou, D. Ioannides, Epidemiological trends in skin cancer, Dermatol. Pract. Concept. 7 (2) (2017) 1.

[2] G. Litjens, T. Kooi, B.E. Bejnordi, A.A.A. Setio, F. Ciompi, M. Ghafoorian, J.A. Van Der Laak, B. Van Ginneken, C.I. Sánchez, A survey on deep learning in medical image analysis, Med. Image Anal. 42 (2017) 60-88.

[3] T.J. Brinker, A. Hekler, J.S. Utikal, N. Grabe, D. Schadendorf, J. Klode, C. Berking, T. Steeb, A.H. Enk, C. Von Kalle, Skin cancer classification using convolutional neural networks: systematic review, J. Med. Internet Res. 20 (10) (2018) e11936.

[4] Q. Abbas, M.E. Celebi, Dermodeep-a classification of melanoma-nevus skin lesions using multi-feature fusion of visual features and deep neural network, Multimed. Tools Appl. 78 (16) (2019) 23559-23580.

[5] J. Kawahara, S. Daneshvar, G. Argenziano, G. Hamarneh, Seven-point checklist and skin lesion classification using multitask multimodal neural nets, IEEE J. Biomed. Health Inform. 23 (2) (2018) 538-546.

[6] N. Gessert, M. Nielsen, M. Shaikh, R. Werner, A. Schlaefer, Skin lesion classification using ensembles of multi-resolution efficientnets with meta data, MethodsX 7 (2020) 100864.

[7] C. Barata, C. Santiago, Improving the explainability of skin cancer diagnosis using cbir, in: International Conference on Medical Image Computing and Computer- Assisted Intervention, Springer, 2021, pp. 550-559.

[8] Y. Wang, J. Cai, D.C. Louie, Z.J. Wang, T.K. Lee, Incorporating clinical knowledge with constrained classifier chain into a multimodal deep network for melanoma detection, Comput. Biol. Med. 137 (2021) 104812.

[9] K. He, X. Zhang, S. Ren, J. Sun,Deep residual learning for image recognition, in:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.

[10] Z. Rahman, M.S. Hossain, M.R. Islam, M.M. Hasan, R.A. Hridhee, An approach for multiclass skin lesion classification based on ensemble learning, Inform. Med. Un-locked 25 (2021) 100659.

[11] P. Tang, Q. Liang, X. Yan, S. Xiang, D. Zhang, Gp-cnn-dtel: global-part cnn model with data-transformed ensemble learning for skin lesion classification, IEEE J. Biomed. Health Inform. 24 (10) (2020) 2870-2882.

[12] R.A. Pratiwi, S. Nurmaini, D.P. Rini, M.N. Rachmatullah, A. Darmawahyuni, Deep ensemble learning for skin lesions classification with convolutional neural network, IAES Int. J. Artif. Intell. 10 (3) (2021) 563.

[13] A. Aboulmira, H. Hrimech, M. Lachgar, et al., Ensemble learning methods for deep learning: application to skin lesions classification, in: 2022 11th International Sym-posium on Signal, Image, Video and Communications (ISIVC), IEEE, 2022, pp. 1-6.

[14] R. Ali, R.C. Hardie, B.N. Narayanan, S. De Silva,Deep learning ensemble methods for skin lesion analysis towards melanoma detection, in: 2019 IEEE National Aerospace and Electronics Conference (NAECON), IEEE, 2019, pp. 311-316.

[15] G. Hinton, O. Vinyals, J. Dean, Distilling the knowledge in a neural network, arXiv preprint, arXiv :1503.02531.

[16] J. Gou, B. Yu, S.J. Maybank, D. Tao, Knowledge distillation: a survey, Int. J. Comput. Vis. 129 (2021) 1789-1819.

[17] L. Wang, K.-J. Yoon, Knowledge distillation and student-teacher learning for visual intelligence: a review and new outlooks, IEEE Trans. Pattern Anal. Mach. Intell. 44 (6) (2021) 3048-3068.

[18] D. Qin, J.-J. Bu, Z. Liu, X. Shen, S. Zhou, J.-J. Gu, Z.-H. Wang, L. Wu, H.-F. Dai,Efficient medical image segmentation based on knowledge distillation, IEEE Trans. Med. Imaging 40 (12) (2021) 3820-3831.

[19] S. Abbasi, M. Hajabdollahi, P. Khadivi, N. Karimi, R. Roshandel, S. Shirani, S. Samavi, Classification of diabetic retinopathy using unlabeled data and knowledge distillation, Artif. Intell. Med. 121 (2021) 102176.

[20] W. Chen, L. Gao, X. Li, W. Shen, Lightweight convolutional neural network with knowledge distillation for cervical cells classification, Biomed. Signal Process. Con-trol 71 (2022) 103177.

[21] S. Back, S. Lee, S. Shin, Y. Yu, T. Yuk, S. Jong, S. Ryu, K. Lee, Robust skin disease classification by distilling deep neural network ensemble for the mobile diagnosis of herpes zoster, IEEE Access 9 (2021) 20156-20169.

[22] M.S. Khan, K.N. Alam, A.R. Dhruba, H. Zunair, N. Mohammed, Knowledge distilla-tion approach towards melanoma detection, Comput. Biol. Med. 146 (2022) 105581.

[23] A. Esteva, B. Kuprel, R.A. Novoa, J. Ko, S.M. Swetter, H.M. Blau, S. Thrun, Dermatologist-level classification of skin cancer with deep neural networks, Nature 542 (7639) (2017) 115-118.

[24] Q. Abbas, Development of a clinically-oriented expert system for differentiating melanocytic from non-melanocytic skin lesions, Int. J. Adv. Comput. Sci. Appl. 8 (7) (2017) 24-29.

[25] O. Sevli, A deep convolutional neural network-based pigmented skin lesion classi-fication application and experts evaluation, Neural Comput. Appl. 33 (18) (2021) 12039-12050.

[26] I. Iqbal, M. Younus, K. Walayat, M.U. Kakar, J. Ma, Automated multi-class classifi-cation of skin lesions through deep convolutional neural network with dermoscopic images, Comput. Med. Imaging Graph. 88 (2021) 101843.

[27] J. Zhang, Y. Xie, Y. Xia, C. Shen,Attention residual learning for skin lesion classifi-cation, IEEE Trans. Med. Imaging 38 (9) (2019) 2092-2103.

[28] S. Benyahia, B. Meftah, O. Lézoray, Multi-features extraction based on deep learning for skin lesion classification, Tissue Cell 74 (2022) 101701.

[29] J. Daghrir, L. Tlig, M. Bouchouicha, M. Sayadi,Melanoma skin cancer detection using deep learning and classical machine learning techniques: a hybrid approach, in: 2020 5th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), IEEE, 2020, pp. 1-5.

[30] R.R. Subramanian, D. Achuth, P.S. Kumar, K.N. Kumar Reddy, S. Amara, A.S. Chowdary,Skin cancer classification using convolutional neural networks, in: 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence), IEEE, 2021, pp. 13-19.

[31] A. Hameed, M. Umer, U. Hafeez, H. Mustafa, A. Sohaib, M.A. Siddique, H.A. Madni, Skin lesion classification in dermoscopic images using stacked convolutional neural network, J. Ambient Intell. Humaniz. Comput. 14 (4) (2023) 3551-3565.

[32] S. Qian, K. Ren, W. Zhang, H. Ning, Skin lesion classification using cnns with grouping of multi-scale attention and class-specific loss weighting, Comput. Meth-ods Programs Biomed. 226 (2022) 107166.

[33] M. Abdar, M. Samami, S.D. Mahmoodabad, T. Doan, B. Mazoure, R. Hashemife-sharaki, L. Liu, A. Khosravi, U.R. Acharya, V. Makarenkov, et al., Uncertainty quan-tification in skin cancer classification using three-way decision-based Bayesian deep learning, Comput. Biol. Med. 135 (2021) 104418.

[34] A. Romero, N. Ballas, S.E. Kahou, A. Chassang, C. Gatta, Y. Bengio, Fitnets: hints for thin deep nets, arXiv preprint, arXiv :1412.6550.

[35] J. Yim, D. Joo, J. Bae, J. Kim,A gift from knowledge distillation: fast optimization, network minimization and transfer learning, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 7130-7138.

[36] Y. Liu, J. Cao, B. Li, C. Yuan, W. Hu, Y. Li, Y. Duan,Knowledge distillation via in-stance relationship graph, in:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 7096-7104.

[37] W. Park, D. Kim, Y. Lu, M. Cho,Relational knowledge distillation, in:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 3967-3976.

[38] B. Peng, X. Jin, J. Liu, D. Li, Y. Wu, Y. Liu, S. Zhou, Z. Zhang,Correlation congruence for knowledge distillation, in:Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 5007-5016.

[39] G. Xu, Z. Liu, X. Li, C.C. Loy, Knowledge distillation meets self-supervision, in: Eu-ropean Conference on Computer Vision, Springer, 2020, pp. 588-604.

[40] X. Ding, Y. Wang, Z. Xu, Z.J. Wang, W.J. Welch, Distilling and transferring knowl-edge via cgan-generated samples for image classification and regression, Expert Syst. Appl. 213 (2023) 119060.

[41] Y. Zhang, T. Xiang, T.M. Hospedales, H. Lu, Deep mutual learning, in: Proceed-ings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 4320-4328.

[42] G. Song, W. Chai, Collaborative learning for deep neural networks, in: S. Bengio, H.Wallach, H.Larochelle, K.Grauman, N.Cesa-Bianchi, R.Garnett (Eds.), Advances in Neural Information Processing Systems, vol. 31, 2018.

[43] G. Wu, S. Gong,Peer collaborative learning for online knowledge distilla-tion, in:Proceedings of the AAAI Conference on Artificial Intelligence, 2021, pp. 10302-10310.

[44] Z. Hou, B. Yu, D. Tao,Batchformer: learning to explore sample relationships for robust representation learning,in:Proceedings of the IEEE/CVF Conference on Com-puter Vision and Pattern Recognition, 2022, pp. 7256-7266.

[45] Y. Wang, Y. Wang, J. Cai, T.K. Lee, C. Miao, Z.J. Wang, Ssd-kd: a self-supervised diverse knowledge distillation method for lightweight skin lesion classification using dermoscopic images, Med. Image Anal. 84 (2023) 102693.

[46] H. Zhang, M. Cisse, Y.N. Dauphin, D.Lopez-Paz, mixup: beyond empirical risk min-imization,in:International Conference on Learning Representations, 2018.

[47] D. Needell, R. Ward, N. Srebro, Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm, in: Z. Ghahramani, M.Welling, C.Cortes, N.Lawrence, K.Weinberger (Eds.), Advances in Neural Information Processing Sys-tems, vol. 27, 2014.