FedACT: An adaptive chained training approach for federated learning in computing power networks

Abstract

Federated Learning (FL) is a novel distributed machine learning methodology that addresses large-scale parallel computing challenges while safeguarding data security. However, the traditional FL model in communication scenarios, whether for uplink or downlink communications, may give rise to several network problems, such as bandwidth occupation, additional network latency, and bandwidth fragmentation. In this paper, we propose an adaptive chained training approach (FedACT) for FL in computing power networks. First, a Computation-driven Clustering Strategy (CCS) is designed. The server clusters clients by task processing delays to minimize waiting delays at the central server. Second, we propose a Genetic-Algorithm-based Sorting (GAS) method to optimize the order of clients participating in training. Finally, based on the table lookup and forwarding rules of the Segment Routing over IPv6 (SRv6) protocol, the sorting results of GAS are written into the SRv6 packet header, to control the order in which clients participate in model training. We conduct extensive experiments on two datasets of CIFAR-10 and MNIST, and the results demonstrate that the proposed algorithm offers improved accuracy, diminished communication costs, and reduced network delays.

Keywords

Computing power network (CPN) / Federated learning (FL) / Segment routing IPv6 (SRv6) / Communication overheads / Model accuracy

References

[1] W.G. Hatcher, W. Yu, A survey of deep learning: platforms, applications and emerg-ing research trends, IEEE Access 6 (2018) 24411-24432.

[2] F. Song, Y.-T. Zhou, Y. Wang, T.-M. Zhao, I. You, H.-K. Zhang, Smart collaborative distribution for privacy enhancement in moving target defense, Inf. Sci. 479 (2019) 593-606.

[3] H. Xie, L. Wei, F. Fang, Research on privacy protection based on machine learning, in: 2021 International Wireless Communications and Mobile Computing (IWCMC), 2021, pp. 1003-1006.

[4] T. Li, A.K. Sahu, A. Talwalkar, V. Smith, Federated learning: challenges, methods, and future directions, IEEE Signal Process. Mag. 37 (3) (2020) 50-60.

[5] B. McMahan, E. Moore, D. Ramage, S. Hampson, B.A. y Arcas, Communication-efficient learning of deep networks from decentralized data, in: Artificial Intelligence and Statistics, PMLR, 2017, pp. 1273-1282.

[6] A. Mora, D. Fantini, P. Bellavista, Federated learning algorithms with heterogeneous data distributions: an empirical evaluation, in: 2022 IEEE/ACM 7th Symposium on Edge Computing (SEC), IEEE, 2022, pp. 336-341.

[7] Z. Lian, Q. Zeng, C. Su, Privacy-preserving blockchain-based global data sharing for federated learning with non-IID data, in: 2022 IEEE 42nd International Conference on Distributed Computing Systems Workshops (ICDCSW), IEEE, 2022, pp. 193-198.

[8] F. Sattler, S. Wiedemann, K.-R. Müller, W. Samek, Robust and communication-efficient federated learning from non-IID data, IEEE Trans. Neural Netw. Learn. Syst. 31 (9) (2019) 3400-3413.

[9] M. Duan, D. Liu, X. Chen, R. Liu, Y. Tan, L. Liang, Self-balancing federated learning with global imbalanced data in mobile systems, IEEE Trans. Parallel Distrib. Syst. 32 (1) (2020) 59-71.

[10] M. Duan, D. Liu, X. Chen, Y. Tan, J. Ren, L. Qiao, L. Liang, Astraea: self-balancing federated learning for improving classification accuracy of mobile deep learning ap-plications, in: 2019 IEEE 37th International Conference on Computer Design (ICCD), IEEE, 2019, pp. 246-254.

[11] X. Mo, J. Xu, Energy-efficient federated edge learning with joint communication and computation design, J. Commun. Inf. Netw. 6(2) (2021) 110-124.

[12] B. Lei, Z. Liu, X. Wang, M. Yang, Y. Chen, Computing network: a new multi-access edge computing, Telecommun. Sci. 35 (9) (2019) 44-51.

[13] X. Tang, C. Cao, Y. Wang, S. Zhang, Y. Liu, M. Li, T. He, Computing power net-work: the architecture of convergence of computing and networking towards 6G requirement, China Commun. 18 (2) (2021) 175-185.

[14] B. Lei, J. Wang, Q. Zhao, et al., Novel network virtualization architecture based on the convergence of computing, storage and transport resources, Telecommun. Sci. 36 (7) (2020) 42-54.

[15] H. Yao, L. Lu, X. Duan, Architecture and key technologies for computing-aware networking, ZTE Technol. J. 27 (3) (2021) 7-11.

[16] Y. Sun, J. Liu, H. Huang, X. Zhang, B. Lei, J. Peng, W. Wang, Computing power network: a survey, arXiv preprint, arXiv : 2210.06080.

[17] B. Lei, Q. Zhao, H. Zhao, et al., Overview of edge computing and computing power network, ZTE Commun. 27 (03) (2021) 3-6.

[18] W. Sun, Z. Li, Q. Wang, Y. Zhang, FedTAR: task and resource-aware federated learning for wireless computing power networks, IEEE Int. Things J. 10 (5) (2022) 4257-4270.

[19] P. Wang, W. Sun, H. Zhang, W. Ma, Y. Zhang, Distributed and secure federated learning for wireless computing power networks, IEEE Trans. Veh. Technol. 72 (7)(2023) 9381-9393.

[20] P. Loreti, A. Mayer, P. Lungaroni, F. Lombardo, C. Scarpitta, G. Sidoretti, L. Brac-ciale, M. Ferrari, S. Salsano, A. Abdelsalam, et al., SRv6-PM: a cloud-native architec-ture for performance monitoring of SRv6 networks, IEEE Trans. Netw. Serv. Manag. 18 (1) (2021) 611-626.

[21] H. Yoo, S. Byun, S. Yang, N. Ko,A service programmable network architecture based on SRv6, in:2022 13th International Conference on Information and Communica-tion Technology Convergence (ICTC), 2022, pp. 2068-2070.

[22] J. Zhou, H. Li, Q. Wu, Z. Lai, J. Liu, SR-TPP: extending IPv6 segment routing to enable trusted and private network paths, in: 2020 IEEE Symposium on Computers and Communications (ISCC), IEEE, 2020, pp. 1-6.

[23] M. Polverini, A. Cianfrani, M. Listanti, Interface counters in segment routing v6: a powerful instrument for traffic matrix assessment, in: 2018 9th International Con-ference on the Network of the Future (NOF), IEEE, 2018, pp. 76-82.

[24] D. Wu, L. Cui, A comprehensive survey on segment routing traffic engineering, Digit. Commun. Netw. 9(4) (2023) 990-1008.

[25] F. Song, Y. Ma, I. You, H. Zhang, Smart collaborative evolvement for virtual group creation in customized industrial IoT, IEEE Trans. Netw. Sci. Eng. 10 (5) (2023) 2514-2524.

[26] F. Sattler, K.-R. Müller, W. Samek, Clustered federated learning: model-agnostic dis-tributed multitask optimization under privacy constraints, IEEE Trans. Neural Netw. Learn. Syst. 32 (8) (2020) 3710-3722.

[27] Y. Fraboni, R. Vidal, L. Kameni, M. Lorenzi, Clustered sampling: low-variance and improved representativity for clients selection in federated learning,in:Interna-tional Conference on Machine Learning, PMLR, 2021, pp. 3407-3416.

[28] Y. Deng, F. Lyu, J. Ren, Y. Zhang, Y. Zhou, Y. Zhang, Y. Yang, Share: shaping data distribution at edge for communication-efficient hierarchical federated learn-ing, in: 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS), IEEE, 2021, pp. 24-34.

[29] Z. Chai, A. Ali, S. Zawad, S. Truex, A. Anwar, N. Baracaldo, Y. Zhou, H. Ludwig, F. Yan, Y. Cheng, TiFL: a tier-based federated learning system,in: Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing, 2020, pp. 125-136.

[30] O. Mersmann, B. Bischl, J. Bossek, H. Trautmann, M. Wagner, F. Neumann, Lo-cal search and the traveling salesman problem: a feature-based characterization of problem hardness,in:Learning and Intelligent Optimization: 6th International Con-ference, LION 6, Paris, France, January 16-20, 2012, in: Revised Selected Papers, Springer, 2012, pp. 115-129.

[31] O. Cheikhrouhou, I. Khoufi, A comprehensive survey on the multiple traveling sales-man problem: applications, approaches and taxonomy, Comput. Sci. Rev. 40 (2021) 100369.

[32] E. Osaba, X.-S. Yang, J. Del Ser, Traveling salesman problem: a perspective review of recent research and new results with bio-inspired metaheuristics, in: Nature-Inspired Computation and Swarm Intelligence, 2020, pp. 135-164.

[33] S.C. Ong, R.-H. Gau, Local loss-assisted dynamic client selection for image classification-oriented federated learning, in: ICC 2022-IEEE International Confer-ence on Communications, IEEE, 2022, pp. 4769-4774.

[34] Y. Ji, Z. Kou, X. Zhong, H. Li, F. Yang, S. Zhang,Client selection and bandwidth al-location for federated learning: an online optimization perspective, in: GLOBECOM 2022-2022 IEEE Global Communications Conference, IEEE, 2022, pp. 5075-5080.

[35] T. Huang, W. Lin, L. Shen, K. Li, A.Y. Zomaya, Stochastic client selection for feder-ated learning with volatile clients, IEEE Int. Things J. 9 (20) (2022) 20055-20070.

[36] W. Zhaohang, X. Geming, C. Jian, Y. Chaodong, Adaptive asynchronous federated learning for edge intelligence, in: 2021 International Conference on Machine Learn-ing and Intelligent Systems Engineering (MLISE), IEEE, 2021, pp. 285-289.

[37] F. Song, M. Zhu, Y. Zhou, I. You, H. Zhang, Smart collaborative tracking for ubiq-uitous power IoT in edge-cloud interplay domain, IEEE Int. Things J. 7(7) (2019) 6046-6055.

[38] Q. Li, Y. Diao, Q. Chen, B. He, Federated learning on non-IID data silos: an exper-imental study, in: 2022 IEEE 38th International Conference on Data Engineering (ICDE), IEEE, 2022, pp. 965-978.

[39] A.A. Al-Saedi, V. Boeva, E. Casalicchio, Reducing communication overhead of feder-ated learning through clustering analysis, in: 2021 IEEE Symposium on Computers and Communications (ISCC), IEEE, 2021, pp. 1-7.