FedViTBloc: Secure and privacy-enhanced medical image analysis with federated vision transformer and blockchain

Abstract

The increasing prevalence of cancer necessitates advanced methodologies for early detection and diagnosis. Early intervention is crucial for improving patient outcomes and reducing the overall burden on healthcare systems. Traditional centralized methods of medical image analysis pose significant risks to patient privacy and data security, as they require the aggregation of sensitive information in a single location. Furthermore, these methods often suffer from limitations related to data diversity and scalability, hindering the development of universally robust diagnostic models. Recent advancements in machine learning, particularly deep learning, have shown promise in enhancing medical image analysis. However, the need to access large and diverse datasets for training these models introduces challenges in maintaining patient confidentiality and adhering to strict data protection regulations. This paper introduces FedViTBloc, a secure and privacy-enhanced framework for medical image analysis utilizing Federated Learning (FL) combined with Vision Transformers (ViT) and blockchain technology. The proposed system ensures patient data privacy and security through fully homomorphic encryption and differential privacy techniques. By employing a decentralized FL approach, multiple medical institutions can collaboratively train a robust deep-learning model without sharing raw data. Blockchain integration further enhances the security and trustworthiness of the FL process by managing client registration and ensuring secure onboarding of participants. Experimental results demonstrate the effectiveness of FedViTBloc in medical image analysis while maintaining stringent privacy standards, achieving 67% accuracy and reducing loss below 2 across 10 clients, ensuring scalability and robustness.

Keywords

AI / Blockchain / Decentralized / Federated Learning / Medical images / Machine Learning / ViT

CRediT authorship contribution statement

Gabriel Chukwunonso Amaizu: Writing - review & editing, Writing - original draft, Visualization, Validation, Software, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Akshita Maradapu Vera Venkata Sai: Writing - review & editing, Supervision, Funding acquisition. Sanjay Bhard-waj: Supervision, Investigation. Dong-Seong Kim: Supervision. Madhuri Siddula: Funding acquisition. Yingshu Li: Funding acquisition.

Declaration of Generative AI and AI-assisted technologies in the writing process

During the preparation of this work, the authors used chatGPT in order to perform English corrections. After using this tool/service, the authors reviewed and edited the content as needed and take(s) full responsibility for the content of the publication.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This research was supported by the Ministry of Science and ICT, Korea, under the Grand IT Research Center support program (IITP-2022-2020-0-01612) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation) and Priority Research Centers Program through the National Research Fund (NRF) Korea funded by the Ministry of Education, Science and Technology, South Korea (2018R1A6A1A03024003). This research was also supported in part by National Science Foundation (NSF) of USA (2200673) and the Office of Sponsored Programs & Research Seed Funding Program at Towson University, United States.

References

[1] What is cancer by WHO, 2023, URL https://www.who.int/health-topics/cancer#tab=tab_1. (Accessed 21 July 2023).

[2] World Health Organization, et al., World health statistics 2020, 2020.

[3] Types of skin cancer, American academy of dermatology association, 2023.

[4] Key statistics for melanoma skin cancer, American cancer society, 2023.

[5] Jacques Ferlay, Murielle Colombet, Isabelle Soerjomataram, Donald M Parkin, Marion Piñeros, Ariana Znaor, Freddie Bray,Cancer statistics for the year 2020: An overview, Int. J. Cancer 149 (4) (2021) 778-789.

[6] Marwan Ali Albahar, Skin lesion classification using convolutional neural network with novel regularizer, IEEE Access 7 (2019) 38306-38313, http://dx.doi.org/10.1109/ACCESS.2019.2906241.

[7] Mary K Tripp, Meg Watson, Sophie J Balk, Susan M Swetter, Jeffrey E Gershenwald, State of the science on prevention and screening to reduce melanoma incidence and mortality: The time is now, CA: Cancer J. Clin. 66 (6) (2016) 460-480.

[8] Muhammad Zia Ur Rehman, Fawad Ahmed, Suliman A. Alsuhibany, Sajjad Shaukat Jamal, Muhammad Zulfiqar Ali, Jawad Ahmad, Classification of skin cancer lesions using explainable deep learning, Sensors (ISSN: 1424-8220) 22 (18) (2022) http://dx.doi.org/10.3390/s22186915, URL https://www.mdpi.com/1424-8220/22/18/6915.

[9] Blagoj Ristevski, Ming Chen, Big data analytics in medicine and healthcare, J. Integr. Bioinform. 15 (3) (2018) 20170030, http://dx.doi.org/10.1515/jib-2017-0030.

[10] Chen Zhang, Yu Xie, Hang Bai, Bin Yu, Weihong Li, Yuan Gao, A survey on federated learning, Knowl.-Based Syst. (ISSN: 0950-7051) 216 (2021) 106775, http://dx.doi.org/10.1016/j.knosys.2021.106775, URL https://www.sciencedirect.com/science/article/pii/S0950705121000381.

[11] A. Rahman, M.S. Hossain, G. Muhammad, D. Kundu, T. Debnath, M. Rahman, M.S.I. Khan, P. Tiwari, S.S. Band, Federated learning-based AI approaches in smart healthcare: concepts, taxonomies, challenges and open issues, Clust. Comput. (2022) 1-41, http://dx.doi.org/10.1007/s10586-022-03658-4, Epub ahead of print.

[12] Ashish Rauniyar, Desta Haileselassie Hagos, Debesh Jha, Jan Erik Håkegård, Ulas Bagci, Danda B. Rawat, Vladimir Vlassov, Federated learning for medical applications: A taxonomy, current trends, challenges, and future research directions, 2022, arXiv:2208.03392.

[13] Fengpan Zhao, Yan Huang,Akshita Maradapu Vera Venkata Sai, Yubao Wu, A cluster-based solution to achieve fairness in federated learn- ing, in: 2020 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking, ISPA/BDCloud/So- cialCom/SustainCom, 2020, pp. 875-882, http://dx.doi.org/10.1109/ISPA-BDCloud-SocialCom-SustainCom51426.2020.00135.

[14] Md Fahimuzzman Sohan, Anas Basalamah, A systematic review on federated learning in medical image analysis, IEEE Access 11 (2023) 28628-28644, http://dx.doi.org/10.1109/ACCESS.2023.3260027.

[15] Manzoor Ahmed Hashmani, Syed Muslim Jameel, Syed Sajjad Hussain Rizvi, Saurabh Shukla, An adaptive federated machine learning-based intelligent system for skin disease detection: A step toward an intelligent dermoscopy device, Appl. Sci. (ISSN: 2076-3417) 11 (5) (2021) 2145, http://dx.doi.org/10.3390/app11052145.

[16] Yifan Li, Yuechen He, Yufei Fu,Sizhe Shan, Privacy preserved federated learning for skin cancer diagnosis, in: 2023 IEEE 3rd International Conference on Power, Electronics and Computer Applications, ICPECA, 2023, pp. 27-33, http://dx.doi.org/10.1109/ICPECA56706.2023.10075862.

[17] Jeffry Wicaksana, Zengqiang Yan, Xin Yang, Yang Liu, Lixin Fan, Kwang-Ting Cheng, Customized federated learning for multi-source decentralized medical image classification, IEEE J. Biomed. Health Inform. 26 (11) (2022) 5596-5607, http://dx.doi.org/10.1109/JBHI.2022.3198440.

[18] Jie Wen, Zhixia Zhang, Yang Lan, Zhihua Cui, Jianghui Cai, Wensheng Zhang, A survey on federated learning: challenges and applications, Int. J. Mach. Learn. Cybern. (ISSN: 1868-808X) 14 (2) (2023) 513-535, http://dx.doi.org/10.1007/s13042-022-01647-y, URL https://doi.org/10.1007/s13042-022-01647-y.

[19] Shengju Qian, Yi Zhu, Wenbo Li, Mu Li, Jiaya Jia, What Makes for Good Tokenizers in Vision Transformer? IEEE Trans. Pattern Anal. Mach. Intell. (2022) 1-13, http://dx.doi.org/10.1109/TPAMI.2022.3231442.

[20] Jie Ma, Yalong Bai, Bineng Zhong, Wei Zhang, Ting Yao, Tao Mei, Visualizing and Understanding Patch Interactions in Vision Transformer, IEEE Trans. Neural Netw. Learn. Syst. (2023) 1-10, http://dx.doi.org/10.1109/TNNLS.2023.3270479.

[21] Kai Han, Yunhe Wang, Hanting Chen, Xinghao Chen, Jianyuan Guo, Zhen-hua Liu, Yehui Tang, An Xiao, Chunjing Xu, Yixing Xu, Zhaohui Yang, Yiman Zhang, Dacheng Tao, A Survey on Vision Transformer, IEEE Trans. Pattern Anal. Mach. Intell. 45 (1) (2023) 87-110, http://dx.doi.org/10.1109/TPAMI.2022.3152247.

[22] Khalid Al-hammuri, Fayez Gebali, Awos Kanan, Ilamparithi Thirumarai Chelvan, Vision transformer architecture and applications in digital health: a tutorial and survey, Vis. Comput. Ind. Biomed. Art (ISSN: 2524-4442) 6 (1) (2023) 14, http://dx.doi.org/10.1186/s42492-023-00140-9.

[23] Suliman Aladhadh, Majed Alsanea, Mohammed Aloraini, Taimoor Khan, Shabana Habib, Muhammad Islam, An effective skin cancer classification mechanism via medical vision transformer, Sensors (ISSN: 1424-8220) 22 (11) (2022) 4008, http://dx.doi.org/10.3390/s22114008.

[24] Guang Yang, Suhuai Luo, Peter Greer, A novel vision transformer model for skin cancer classification, Neural Process. Lett. (ISSN: 1573-773 X) (2023) http://dx.doi.org/10.1007/s11063-023-11204-5.

[25] Chao Xin, Zhifang Liu, Keyu Zhao, Linlin Miao, Yizhao Ma, Xiaoxia Zhu, Qiongyan Zhou, Songting Wang, Lingzhi Li, Feng Yang, Suling Xu, Haijiang Chen, An improved transformer network for skin cancer classification, Comput. Biol. Med. (ISSN: 0010-4825) 149 (2022) 105939, http://dx.doi.org/10.1016/j.compbiomed.2022.105939, URL https://www.sciencedirect.com/science/article/pii/S0010482522006746.

[26] Ahmed Afif Monrat, Olov Schelén, Karl Andersson, A survey of blockchain from the perspectives of applications, challenges, and opportunities, IEEE Access 7 (2019) 117134-117151, http://dx.doi.org/10.1109/ACCESS.2019.2936094.

[27] Asaph Azaria, Ariel Ekblaw, Thiago Vieira, Andrew Lippman,MedRec: Using blockchain for medical data access and permission management, in: 2016 2nd International Conference on Open and Big Data, OBD, 2016, pp. 25-30, http://dx.doi.org/10.1109/OBD.2016.11.

[28] Thomas McGhin, Kim-Kwang Raymond Choo, Charles Zhechao Liu, De-biao He, Blockchain in healthcare applications: Research challenges and opportunities, J. Netw. Comput. Appl. (ISSN: 1084-8045) 135 (2019) 62-75, http://dx.doi.org/10.1016/j.jnca.2019.02.027, URL https://www.sciencedirect.com/science/article/pii/S1084804519300864.

[29] Mengfang Li, Yuanyuan Jiang, Yanzhou Zhang, Haisheng Zhu, Medical image analysis using deep learning algorithms, Front. Public Health (ISSN: 2296-2565) 11 (2023) http://dx.doi.org/10.3389/fpubh.2023.1273253, URL https://www.frontiersin.org/journals/public-health/articles/10.3389/fpubh.2023.1273253.

[30] Khaled Salah M. Habib Ur Rehman, Nishara Nizamuddin, Ala Al-Fuqaha, Blockchain for AI: Review and open research challenges, IEEE Access 7 (2019) 10127-10149, http://dx.doi.org/10.1109/ACCESS.2018.2890507.

[31] Reza Shokri, Vitaly Shmatikov,Privacy-preserving deep learning, in: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, CCS ’15, Association for Computing Machinery, New York, NY, USA, ISBN: 9781450338325, 2015, pp. 1310-1321, http://dx.doi.org/10.1145/2810103.2813687.

[32] Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H. Yang, Farhad Farokhi, Shi Jin, Tony Q.S. Quek, H. Vincent Poor, Federated learning with differential privacy: Algorithms and performance analysis, IEEE Trans. Inf. Forensics Secur. 15 (2020) 3454-3469, http://dx.doi.org/10.1109/TIFS.2020.2988575.

[33] Zaobo He, Akshita Maradapu Vera Venkata Sai, Daehee Huang, Hanzhou Zhang, Qilong Han, Differentially private approximate aggregation based on feature selection, J. Comb. Optim. 41 (2021) http://dx.doi.org/10.1007/s10878-020-00666-1.

[34] Amjad Qashlan, Priyadarsi Nanda, Manoranjan Mohanty, Differential privacy model for blockchain based smart home architecture, Future Gener. Comput. Syst. (ISSN: 0167-739X) 150 (2024) 49-63, http://dx.doi.org/10.1016/j.future.2023.08.010.

[35] Opacus AI, 2023, URL https://opacus.ai. (Accessed 21 July 2023).

[36] Flower dev, 2023, URL https://flower.dev/. (Accessed 21 July 2023).

[37] HAM10000 dataset, 2023, URL https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/DBW86T. (Accessed 21 July 2023).

[38] Solid check, 2024, URL https://solidcheck.io/. (Accessed 16 July 2024).