Privacy-preserved learning from non-i.i.d data in fog-assisted IoT: A federated learning approach

Abstract

With the prevalence of the Internet of Things (IoT) systems, smart cities comprise complex networks, including sensors, actuators, appliances, and cyber services. The complexity and heterogeneity of smart cities have become vulnerable to sophisticated cyber-attacks, especially privacy-related attacks such as inference and data poisoning ones. Federated Learning (FL) has been regarded as a hopeful method to enable distributed learning with privacy-preserved intelligence in IoT applications. Even though the significance of developing privacy-preserving FL has drawn as a great research interest, the current research only concentrates on FL with independent identically distributed (i.i.d) data and few studies have addressed the non-i. i.d setting. FL is known to be vulnerable to Generative Adversarial Network (GAN) attacks, where an adversary can presume to act as a contributor participating in the training process to acquire the private data of other contributors. This paper proposes an innovative Privacy Protection-based Federated Deep Learning (PP-FDL) framework, which accomplishes data protection against privacy-related GAN attacks, along with high classification rates from non-i. i.d data. PP-FDL is designed to enable fog nodes to cooperate to train the FDL model in a way that ensures contributors have no access to the data of each other, where class probabilities are protected utilizing a private identifier generated for each class. The PP-FDL framework is evaluated for image classification using simple convolutional networks which are trained using MNIST and CIFAR-10 datasets. The empirical results have revealed that PF-DFL can achieve data protection and the framework outperforms the other three state-of-the-art models with 3%-8% as accuracy improvements.

Keywords

Privacy preservation / Federated learning / Deep learning / Fog computing / Smart cities

References

[1] M.A. Al-Garadi, A. Mohamed, A.K. Al-Ali, X. Du, I. Ali, M. Guizani, A survey of machine and deep learning methods for internet of things (iot) security, IEEE Commun. Surv. Tutor. 22 (3) (2020) 1646-1685.

[2] Z. He, T. Zhang, R.B. Lee, Attacking and protecting data privacy in edge-cloud collaborative inference systems, IEEE Internet Things J. 8 (12) (2020) 9706-9716.

[3] M. Keshk, B. Turnbull, N. Moustafa, D. Vatsalan, K.-K.R. Choo, A privacy-preserving-framework-based blockchain and deep learning for protecting smart power networks, IEEE Trans. Ind. Inf. 16 (8) (2019) 5110-5118.

[4] L. Zhao, Q. Wang, Q. Zou, Y. Zhang, Y. Chen, Privacypreserving collaborative deep learning with unreliable participants, IEEE Trans. Inf. Forensics Secur. 15 (2019) 1486-1500.

[5] Z. Chen, A. Fu, Y. Zhang, Z. Liu, F. Zeng, R.H. Deng, Secure collaborative deep learning against gan attacks in the internet of things, IEEE Internet Things J. 8 (7)(2020) 5839-5849.

[6] X. Xu, Q. Huang, X. Yin, M. Abbasi, M.R. Khosravi, L. Qi, Intelligent offloading for collaborative smart city services in edge computing, IEEE Internet Things J. 7 (9)(2020) 7919-7927.

[7] H. Wu, Z. Zhang, C. Guan, K. Wolter, M. Xu, Collaborate edge and cloud computing with distributed deep learning for smart city internet of things, IEEE Internet Things J. 7 (9) (2020) 8099-8110.

[8] R. Hu, Y. Guo, H. Li, Q. Pei, Y. Gong, Personalized federated learning with differential privacy, IEEE Internet Things J. 7 (10) (2020) 9530-9539.

[9] M. Tiwari, S. Misra, P.K. Bishoyi, L.T. Yang, Devote: criticality-aware federated service provisioning in fog-based iot environments, IEEE Internet Things J. 8 (13)(2021) 10631-10638.

[10] R. Saha, S. Misra, P.K. Deb, Fogfl: fog-assisted federated learning for resource-constrained iot devices, IEEE Internet Things J. 8 (10) (2020) 8456-8463.

[11] J. Zhang, B. Chen, X. Cheng, H.T.T. Binh, S. Yu, Poisongan: generative poisoning attacks against federated learning in edge computing systems, IEEE Internet Things J. 8 (5) (2020) 3310-3322.

[12] S. Lu, Y. Yao, W. Shi, Clone: collaborative learning on the edges, IEEE Internet Things J. 8 (13) (2020) 10222-10236.

[13] Y. Huang, X. Qiao, P. Ren, L. Liu, C. Pu, S. Dustdar, J. Chen, A lightweight collaborative deep neural network for the mobile web in edge cloud, IEEE Transactions on Mobile Computing, IEEE (2020). PP(99), 1-1.

[14] R. Hadidi, J. Cao, M.S. Ryoo, H. Kim, Toward collaborative inferencing of deep neural networks on internet-of-things devices, IEEE Internet Things J. 7 (6) (2020) 4950-4960.

[15] N. Rodr’ıguez-Barroso, G. Stipcich, D. Jim’enez-L’opez, J.A. Ruiz-Mill’an, E. Mart’ınez-C’amara, G. Gonz’alez-Seco, M.V. Luz’on, M.A. Veganzones, F. Herrera, Federated learning and differential privacy: software tools analysis, the sherpa. ai fl framework and methodological guidelines for preserving data privacy, Inf. Fusion 64 (2020) 270-292.

[16] S.A. Hamad, Q.Z. Sheng, W.E. Zhang, S. Nepal, Realizing an internet of secure things: a survey on issues and enabling technologies, IEEE Commun. Surv. Tutor. 22 (2) (2020) 1372-1391.

[17] M.B. Sariyildiz, R.G. Cinbis, E. Ayday, Key protected classification for collaborative learning, Pattern Recogn. 104 (2020) 107327.

[18] A. Fu, X. Zhang, N. Xiong, Y. Gao, H. Wang, J. Zhang, Vfl, A verifiable federated learning with privacy-preserving for big data in industrial iot, IEEE Trans. Ind. Inf. 18 (5) (2020) 3316-3326.

[19] J. Baek, G. Kaddoum, Heterogeneous task offloading and resource allocations via deep recurrent reinforcement learning in partial observable multifog networks, IEEE Internet Things J. 8 (2) (2020) 1041-1056.

[20] J. Pang, Y. Huang, Z. Xie, Q. Han, Z. Cai, Realizing the heterogeneity: a self-organized federated learning framework for iot, IEEE Internet Things J. 8 (5) (2020) 3088-3098.

[21] S. Henna, A. Davy, Distributed and collaborative high speed inference deep learning for mobile edge with topological dependencies, IEEE Transactions on Cloud Computing, IEEE (2020). PP(99):1-1.

[22] B. McMahan, E. Moore, D. Ramage, S. Hampson, B.A. y Arcas, Communication-efficient learning of deep networks from decentralized data, in: Artificial Intelligence and Statistics, PMLR, 2017, pp. 1273-1282.

[23] B. Yin, H. Yin, Y. Wu, Z. Jiang, Fdc: a secure federated deep learning mechanism for data collaborations in the internet of things, IEEE Internet Things J. 7 (7) (2020) 6348-6359.

[24] W. Wu, P. Yang, W. Zhang, C. Zhou, X. Shen, Accuracy guaranteed collaborative dnn inference in industrial iot via deep reinforcement learning, IEEE Trans. Ind. Inf. 17 (7) (2020) 4988-4998.

[25] Y. Song, T. Liu, T. Wei, X. Wang, Z. Tao, M. Chen, Fda3: federated defense against adversarial attacks for cloud-based iiot applications, IEEE Trans. Ind. Inf. 17 (11)(2020) 7830-7838.

[26] H. Shen, M. Zhang, H. Wang, F. Guo, W. Susilo, A lightweight privacy-preserving fair meeting location determination scheme, IEEE Internet Things J. 7 (4) (2020) 3083-3093.

[27] Z. Ma, J. Ma, Y. Miao, X. Liu, K.-K.R. Choo, R. Yang, X. Wang, Lightweight privacy-preserving medical diagnosis in edge computing, IEEE Transactions on Services Computing (2020), https://doi.org/10.1109/TSC.2020.3004627.

[28] Q. Zhang, C. Xin, H. Wu, Privacy-preserving deep learning based on multiparty secure computation: a survey, IEEE Internet Things J. 8 (13) (2021) 10412-10429.

[29] Y. Lu, X. Huang, Y. Dai, S. Maharjan, Y. Zhang, Differentially private asynchronous federated learning for mobile edge computing in urban informatics, IEEE Trans. Ind. Inf. 16 (3) (2019) 2134-2143.

[30] C. Zhou, A. Fu, S. Yu, W. Yang, H. Wang, Y. Zhang, Privacypreserving federated learning in fog computing, IEEE Internet Things J. 7 (11) (2020) 10782-10793.

[31] Y. Li, Y. Zhou, A. Jolfaei, D. Yu, G. Xu, X. Zheng, Privacypreserving federated learning framework based on chained secure multiparty computing, IEEE Internet Things J. 8 (8) (2020) 6178-6186.

[32] Q. Feng, D. He, Z. Liu, H. Wang, K.-K.R. Choo, Securenlp: a system for multi-party privacy-preserving natural language processing, IEEE Trans. Inf. Forensics Secur. 15 (2020) 3709-3721.

[33] C. Ding, A. Zhou, Y. Liu, R. Chang, C.-H. Hsu, S. Wang, A Cloud-Edge Collaboration Framework for Cognitive Service, IEEE Transactions on Cloud Computing (2020), https://doi.org/10.1109/TCC.2020.2997008.

[34] Y.K. Teoh, S.S. Gill, A.K. Parlikad, Iot and Fog Computing Based Predictive Maintenance Model for Effective Asset Management in Industry 4.0 Using Machine Learning, IEEE Internet of Things Journal (2021), https://doi.org/10.1109/JIOT.2021.3050441.

[35] T. Zhang, Z. Shen, J. Jin, X. Zheng, A. Tagami, X. Cao, Achieving democracy in edge intelligence: a fog-based collaborative learning scheme, IEEE Internet Things J. 8 (4) (2020) 2751-2761.

[36] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: an imperative style, high-performance deep learning library, Adv. Neural Inf. Process., Syst. 32 (2019), https://doi.org/10.48550/arXiv.1912.01703.

[37] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE 86 (11) (1998) 2278-2324.

[38] A. Krizhevsky, G. Hinton, et al., Learning Multiple Layers of Features from Tiny Images 1(4), 2009, pp. 1-58.

[39] M. Teng, F. Wood, Bayesian distributed stochastic gradient descent, in: Proceedings of the 32nd International Conference on Neural Information Processing Systems, ACM, 2018, pp. 6380-6390.

[40] J. Ball’e, V. Laparra, E.P. Simoncelli, Density Modeling of Images Using a Generalized Normalization Transformation, 2015 arXiv preprint arXiv:1511.06281.

[41] S. Ioffe, C. Szegedy, Batch normalization: accelerating deep network training by reducing internal covariate shift,in:International Conference on Machine Learning, PMLR, 2015, pp. 448-456.

[42] S. Savazzi, M. Nicoli, V. Rampa, Federated learning with cooperating devices: a consensus approach for massive iot networks, IEEE Internet Things J. 7 (5) (2020) 4641-4654.

[43] X. Li, M. Jiang, X. Zhang, M. Kamp, Q. Dou, Fedbn: Federated Learning on Non-iid Features via Local Batch Normalization, 2021 arXiv preprint arXiv: 2102.07623.