DB-FL: DAG blockchain-enabled generalized federated dropout learning^☆

Abstract

To protect user privacy and data security, the integration of Federated Learning (FL) and blockchain has become an emerging research hotspot. However, the limited throughput and high communication complexity of traditional blockchains limit their application in large-scale FL tasks, and the synchronous traditional FL will also reduce the training efficiency. To address these issues, in this paper, we propose a Directed Acyclic Graph (DAG) blockchain-enabled generalized Federated Dropout (FD) learning strategy, which could improve the efficiency of FL while ensuring the model generalization. Specifically, the DAG maintained by multiple edge servers will guarantee the security and traceability of the data, and the Reputation-based Tips Selection Algorithm (RTSA) is proposed to reduce the blockchain consensus delay. Second, the semi-asynchronous training among Intelligent Devices (IDs) is adopted to improve the training efficiency, and a reputation-based FD technology is proposed to prevent overfitting of the model. In addition, a Hybrid Optimal Resource Allocation (HORA) algorithm is introduced to minimize the network delay. Finally, simulation results demonstrate the effectiveness and superiority of the proposed algorithms.

Keywords

Federated learning / Blockchain / Directed acyclic graph / Federated dropout / Resource allocation

CRediT authorship contribution statement

Sa Xiao: Writing - original draft, Visualization, Validation, Resources, Project administration, Methodology, Formal analysis, Data curation. Xiaoge Huang: Writing - review & editing, Funding acquisition, Conceptualization. Xuesong Deng: Investigation. Bin Cao: Software. Qianbin Chen: Supervision.

Declaration of Competing Interest

The authors declare that there are no conflict of interests, we do not have any possible conflicts of interest. Bin Cao is an editor for Digital Communications and Networks and was not involved in the editorial review or the decision to publish this article. All authors declare that there are no competing interests.

Acknowledgements

This work was supported in part by the National Key R&D Program of China under Grant 2021YFB1714100, and in part by the National Natural Science Foundation of China (NSFC) under Grant 62371082 and 62001076, and in part by the Natural Science Foundation of Chongqing under Grant CSTB2023NSCQ-MSX0726 and cstc2020jcyjmsxmX0878.

References

[1] J. Bian, et al., Machine learning in real-time Internet of things (IoT) systems: a sur-vey, IEEE Int. Things J. 9 (11) (2022) 8364-8386.

[2] V.-L. Nguyen, P.-C. Lin, B.-C. Cheng, R.-H. Hwang, Y.-D. Lin,Security and privacy for 6G: a survey on prospective technologies and challenges, IEEE Commun. Surv. Tutor. 23 (4) (2021) 2384-2428.

[3] S. Fu, Y. Wang, X. Feng, B. Di, C. Li, Reconfigurable intelligent surface assisted non-orthogonal multiple access network based on machine learning approaches, IEEE Netw. 38 (2) (2024) 272-279.

[4] Z. Chang, S. Liu, X. Xiong, Z. Cai, G. Tu, A survey of recent advances in edge-computing-powered artificial intelligence of things, IEEE Int. Things J. 8 (18) (2021) 13849-13875.

[5] D.C. Nguyen, M. Ding, P.N. Pathirana, A. Seneviratne, J. Li, H. Vincent Poor, Feder-ated learning for Internet of things: a comprehensive survey, IEEE Commun. Surv. Tutor. 23 (3) (2021) 1622-1658.

[6] D.C. Nguyen, S. Hosseinalipour, D.J. Love, P.N. Pathirana, C.G. Brinton, Latency op-timization for blockchain-empowered federated learning in multi-server edge com-puting, IEEE J. Sel. Areas Commun. 40 (12) (2022) 3373-3390.

[7] B. McMahan, E. Moore, D. Ramage, S. Hampson, B.A.Y. Arcas,Communication-efficient learning of deep networks from decentralized data, in:2017 International Conference on Artificial Intelligence and Statistics (AISTATS), 2017, pp. 1273-1282.

[8] Q. Ma, Y. Xu, H. Xu, Z. Jiang, L. Huang, H. Huang, FedSA: a semi-asynchronous federated learning mechanism in heterogeneous edge computing, IEEE J. Sel. Areas Commun. 39 (12) (2021) 3654-3672.

[9] B. Cao, et al., Blockchain systems, technologies, and applications: a methodology perspective, IEEE Commun. Surv. Tutor. 25 (1) (2023) 353-385.

[10] Y. Liu, F.R. Yu, X. Li, H. Ji, V.C.M. Leung, Blockchain and machine learning for communications and networking systems, IEEE Commun. Surv. Tutor. 22 (2) (2020) 1392-1431.

[11] H. Kim, J. Park, M. Bennis, S.-L. Kim, Blockchained on-device federated learning, IEEE Commun. Lett. 24 (6) (2020) 1279-1283.

[12] C. Xu, et al., Asynchronous federated learning on heterogeneous devices: a survey, Comput. Sci. Rev. 50 (2023) 100595.

[13] C. Xie, S. Koyejo, I. Gupta, Asynchronous federated optimization, in: NeurIPS Work-shop on Optimization for Machine Learning (OPT2020), 2020.

[14] Y. Zhang, et al., FedMDS: an efficient model discrepancy-aware semi-asynchronous clustered federated learning framework, IEEE Trans. Parallel Distrib. Syst. 34 (3) (2023) 1007-1019.

[15] M. Al-Quraan, et al., Edge-native intelligence for 6g communications driven by fed-erated learning: a survey of trends and challenges, IEEE Trans. Emerg. Top. Comput. Intell. 7 (3) (2023) 957-979.

[16] S. Duan, et al., Distributed artificial intelligence empowered by end-edge-cloud com-puting: a survey, IEEE Commun. Surv. Tutor. 25 (1) (2023) 591-624.

[17] J. Mills, J. Hu, G. Min, Client-side optimization strategies for communication-efficient federated learning, IEEE Commun. Mag. 60 (7) (2022) 60-66.

[18] A. Selamnia, B. Brik, S.M. Senouci, A. Boualouache, S. Hossain,Edge computing-enabled intrusion detection for c-v2x networks using federated learning, in: 2022 IEEE Global Communications Conference, 2022, pp. 2080-2085.

[19] Q. Wu, et al., HiFlash: communication-efficient hierarchical federated learning with adaptive staleness control and heterogeneity-aware client-edge association, IEEE Trans. Parallel Distrib. Syst. 34 (5) (2023) 1560-1579.

[20] L. Witt, M. Heyer, K. Toyoda, W. Samek, D. Li, Decentral and incentivized federated learning frameworks: a systematic literature review, IEEE Int. Things J. 10 (4) (2023) 3642-3663.

[21] J. Huang, L. Kong, G. Chen, Q. Xiang, X. Chen, X. Liu, Blockchain-based federated learning: a systematic survey, IEEE Netw. 37 (6) (2022) 150-157.

[22] L. Cui, X. Su, Y. Zhou, A fast blockchain-based federated learning framework with compressed communications, IEEE J. Sel. Areas Commun. 40 (12) (2022) 3358-3372.

[23] S. Liu, Y. Shang,Federated learning with anomaly client detection and decentralized parameter aggregation, in:2022 52nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), 2022, pp. 37-43.

[24] C. Liu, S. Guo, S. Guo, Y. Yan, X. Qiu, S. Zhang, LTSM: lightweight and trusted sharing mechanism of iot data in smart city, IEEE Int. Things J. 9 (7) (2022) 5080-5093.

[25] X. Ma, et al., A state-of-the-art survey on solving non-iid data in federated learning, Future Gener. Comput. Syst. 135 (2022) 244-258.

[26] M. Mendieta, et al., Local learning matters: rethinking data heterogeneity in fed-erated learning,in:2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 8387-8396.

[27] Z. Wang, et al., Asynchronous federated learning over wireless communication net-works, IEEE Trans. Wirel. Commun. 21 (9) (2022) 6961-6978.

[28] J. Nguyen, et al., Federated learning with buffered asynchronous aggregation,in:Proceedings of the 25th International Conference on Artificial Intelligence and Statis-tics (AISTATS), 2022, pp. 3581-3607.

[29] J. Konečn`y, Z. Qu, P. Richtárik, Semi-stochastic coordinate descent, Optim. Methods Softw. 32 (5) (2017) 993-1005.

[30] S. Popov, O. Saa, P. Finardi, Equilibria in the tangle, Comput. Ind. Eng. 136 (2019) 160-172.

[31] S. Caldas, J. Konečny, H.B. McMahan, A. Talwalkar, Expanding the reach of feder-ated learning by reducing client resource requirements, https://arxiv.org/abs/1812.07210, 2019. (Accessed 6 August 2024).

[32] S. Luo, X. Chen, Q. Wu, Z. Zhou, S. Yu, HFEL: joint edge association and resource allocation for cost-efficient hierarchical federated edge learning, IEEE Trans. Wirel. Commun. 19 (10) (2020) 6535-6548.

[33] W. Dinkelbach, On nonlinear fractional programming, Manag. Sci. 13 (7) (1967) 492-498.

[34] M. Ribero, H. Vikalo, G. de Veciana, Federated learning under intermittent client availability and time-varying communication constraints, IEEE J. Sel. Top. Signal Process. 17 (1) (2023) 98-111.

[35] L. Li, D. Huang, C. Zhang, An efficient DAG blockchain architecture for IoT, IEEE Int. Things J. 10 (2) (2024) 1286-1296.

[36] J. Feng, F.R. Yu, Q. Pei, J. Du, L. Zhu, Joint optimization of radio and computational resources allocation in blockchain-enabled mobile edge computing systems, IEEE Trans. Wirel. Commun. 19 (6) (2020) 4321-4334.

[37] L. Feng, Y. Zhao, S. Guo, X. Qiu, W. Li, P. Yu, BAFL: a blockchain-based asynchronous federated learning framework, IEEE Trans. Comput. 71 (5) (2021) 1092-1103.

[38] M. Cao, L. Zhang, B. Cao, Toward on-device federated learning: a direct acyclic graph-based blockchain approach, IEEE Trans. Neural Netw. Learn. Syst. 34 (4) (2023) 2028-2042.

[39] B. Cao, M. Li, L. Zhang, Y. Li, M. Peng, How does CSMA/CA affect the performance and security in wireless blockchain networks, IEEE Trans. Ind. Inform. 16 (6) (2020) 4270-4280.