Deep reinforcement learning based resource provisioning for federated edge learning

Abstract

With the rapid development of mobile internet technology and increasing concerns over data privacy, Federated Learning (FL) has emerged as a significant framework for training machine learning models. Given the advancements in technology, User Equipment (UE) can now process multiple computing tasks simultaneously, and since UEs can have multiple data sources that are suitable for various FL tasks, multiple tasks FL could be a promising way to respond to different application requests at the same time. However, running multiple FL tasks simultaneously could lead to a strain on the device’s computation resource and excessive energy consumption, especially the issue of energy consumption challenge. Due to factors such as limited battery capacity and device heterogeneity, UE may fail to efficiently complete the local training task, and some of them may become stragglers with high-quality data. Aiming at alleviating the energy consumption challenge in a multi-task FL environment, we design an automatic Multi-Task FL Deployment (MFLD) algorithm to reach the local balancing and energy consumption goals. The MFLD algorithm leverages Deep Reinforcement Learning (DRL) techniques to automatically select UEs and allocate the computation resources according to the task requirement. Extensive experiments validate our proposed approach and showed significant improvements in task deployment success rate and energy consumption cost.

Keywords

Task deployment / Federated learning / Deep reinforcement learning

CRediT authorship contribution statement

Xingyun Chen: Conceptualization, Methodology. Junjie Pang: Methodology, Writing - review & editing. Tonghui Sun: Methodology, Writing - review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

[1] Z. He, L. Wang, Z. Cai, Clustered federated learning with adaptive local differential privacy on heterogeneous iot data, IEEE Internet Things J. (2023).

[2] A. Tak, S. Cherkaoui, Federated edge learning: Design issues and challenges, IEEE Netw. 35 (2) (2021) 252-258, http://dx.doi.org/10.1109/MNET.011.2000478.

[3] A. Taïk, Z. Mlika, S. Cherkaoui, Data-aware device scheduling for federated edge learning, IEEE Trans. Cogn. Commun. Netw. 8 (1) (2022) 408-421, http://dx.doi.org/10.1109/TCCN.2021.3100574.

[4] Z. Xiong, Z. Cai, D. Takabi, W. Li, Privacy threat and defense for federated learning with non-iid data in aIoT, IEEE Trans. Ind. Inform. 18 (2) (2021) 1310-1321.

[5] L. Lyu, H. Yu, X. Ma, C. Chen, L. Sun, J. Zhao, Q. Yang, P.S. Yu, Privacy and robustness in federated learning: Attacks and defenses, IEEE Trans. Neural Netw. Learn. Syst. (2022) 1-21, http://dx.doi.org/10.1109/TNNLS.2022.3216981.

[6] Z. Xiong, W. Li, Z. Cai, Federated generative model on multi-source heterogeneous data in iot, in:Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37, No. 9, 2023, pp. 10537-10545.

[7] M. Miozzo, Z. Ali, L. Giupponi, P. Dini, Distributed and multi-task learning at the edge for energy efficient radio access networks, IEEE Access 9 (2021) 12491-12505, http://dx.doi.org/10.1109/ACCESS.2021.3050841.

[8] M. El Ghmary, Y. Hmimz, T. Chanyour, A. Ouacha, M.O. Cherkaoui Malki, Multi-task offloading and computational resources management in a mo-bile edge computing environment, in: 2020 5th International Conference on Cloud Computing and Artificial Intelligence: Technologies and Applica-tions, CloudTech, 2020, pp. 1-7, http://dx.doi.org/10.1109/CloudTech49835.2020.9365903.

[9] Z. Xiong, W. Li, Y. Li, Z. Cai, Exact-fun: An exact and efficient federated unlearning approach, in: 2023 IEEE International Conference on Data Mining, ICDM, IEEE, 2023, pp. 1439-1444.

[10] J. Lu, Q. Li, B. Guo, J. Li, Y. Shen, G. Li, H. Su, A multi-task oriented framework for mobile computation offloading, IEEE Trans. Cloud Comput. 10 (1) (2019) 187-201.

[11] J. Lee, H. Ko, J. Kim, S. Pack, Data: Dependency-aware task allocation scheme in distributed edge clouds, IEEE Trans. Ind. Inform. 16 (12) (2020) 7782-7790.

[12] Y. Liu, S. Wang, Q. Zhao, S. Du, A. Zhou, X. Ma, F. Yang, Dependency-aware task scheduling in vehicular edge computing, IEEE Internet Things J. 7 (6)(2020) 4961-4971.

[13] D. Chen, C.S. Hong, L. Wang, Y. Zha, Y. Zhang, X. Liu, Z. Han, Matching-theory-based low-latency scheme for multitask federated learning in MEC networks, IEEE Internet Things J. 8 (14) (2021) 11415-11426.

[14] Z. Li, H. Wu, Y. Lu, B. Ai, Z. Zhong, Y. Zhang, Matching game for multi-task federated learning in internet of vehicles, IEEE Trans. Veh. Technol. (2023).

[15] H. Lu, C. Gu, F. Luo, W. Ding, S. Zheng, Y. Shen, Optimization of task offloading strategy for mobile edge computing based on multi-agent deep reinforcement learning, IEEE Access 8 (2020) 202573-202584.

[16] Q. Luo, S. Hu, C. Li, G. Li, W. Shi, Resource scheduling in edge computing: A survey, IEEE Commun. Surv. Tutor. 23 (4) (2021) 2131-2165.

[17] S. Huang, S. Ontañón, A closer look at invalid action masking in policy gradient algorithms, 2020, arXiv preprint arXiv:2006.14171.

[18] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, O. Klimov, Proximal policy optimization algorithms, 2017, arXiv preprint arXiv:1707.06347.

[19] J. Schulman, P. Moritz, S. Levine, M. Jordan, P. Abbeel, High-dimensional continuous control using generalized advantage estimation, 2015, arXiv preprint arXiv:1506.02438.