Cybertwin driven resource allocation using optimized proximal policy based federated learning in 6G enabled edge environment^☆

Abstract

Sixth-generation (6G) communication system promises unprecedented data density and transformative applications over different industries. However, managing heterogeneous data with different distributions in 6G-enabled multi-access edge cloud networks presents challenges for efficient Machine Learning (ML) training and aggregation, often leading to increased energy consumption and reduced model generalization. To solve this problem, this research proposes a Weighted Proximal Policy-based Federated Learning approach integrated with ResNet50 and Scaled Exponential Linear Unit activation function (WPPFL-RS). The proposed method optimizes resource allocation such as CPU and memory, through enhancing the Cyber-twin technology to estimate the computing capacities of edge clouds. The proposed WPPFL-RS approach significantly minimizes the latency and energy consumption, solving complex challenges in 6G-enabled edge computing. This makes sure that efficient resource utilization and enhanced performance in heterogeneous edge networks. The proposed WPPFL-RS achieves a minimum latency of 8.20 s on 100 tasks, a significant improvement over the baseline Deep Reinforcement Learning (DRL), which recorded 11.39 s. This approach highlights its potential to enhance resource utilization and performance in 6G edge networks.

Keywords

Cybertwin / Federated learning / ResNet50 / Resource allocation / Scaled exponential linear unit / Weighted proximal policy

References

[1] M. Al-Ali, E. Yaacoub, Resource allocation scheme for eMBB and uRLLC coexistence in 6G networks, Wirel. Netw. 29 (6) (2023) 2519-2538.

[2] Q. Wang, Y. Liu, Y. Wang, X. Xiong, J. Zong, J. Wang, P. Chen, Resource allocation based on Radio Intelligence Controller for Open RAN towards 6G, IEEE Access 11 (2023) 97909-97919.

[3] N. Omheni, A. Amiri, F. Zarai, A detailed reinforcement learning framework for resource allocation in non-orthogonal multiple access enabled-B5G/6G networks, IET Netw. 13 (5-6) (2024) 455-470.

[4] M.M. Kamruzzaman, 6G wireless communication assisted security management us-ing cloud edge computing, Expert Syst. 40 (4) (2023) e13061.

[5] M. Ashwin, A.A. Alqahtani, A. Mubarakali, B. Sivakumar, Efficient resource manage-ment in 6G communication networks using hybrid quantum deep learning model, Comput. Electr. Eng. 106 (2023) 108565.

[6] R. Adeogun, G. Berardinelli, Multi-agent dynamic resource allocation in 6G in-X subnetworks with limited sensing information, Sensors 22 (13) (2022) 5062.

[7] L. Gu, M. Cui, L. Xu, X. Xu, Collaborative offloading method for digital twin empow-ered cloud edge computing on Internet of Vehicles, Tsinghua Sci. Technol. 28 (3)(2022) 433-451.

[8] M. Adhikari, A. Munusamy, N. Kumar, S.N. Srirama, Cybertwin-driven resource pro-visioning for IoE applications at 6G-enabled edge networks, IEEE Trans. Ind. Inform. 18 (7) (2021) 4850-4858.

[9] J. Huang, F. Yang, C. Chakraborty, Z. Guo, H. Zhang, L. Zhen, K. Yu, Opportunistic capacity based resource allocation for 6G wireless systems with network slicing, Future Gener. Comput. Syst. 140 (2023) 390-401.

[10] W. Han, X. Wang, J. Mei, EDI-driven multi-dimensional resource allocation for in-clusive 6G communications, IEEE Netw. Lett. 5 (2) (2023) 110-114.

[11] S.U. Jamil, M.A. Khan, S.U. Rehman, Resource allocation and task off-loading for 6G enabled smart edge environments, IEEE Access 10 (2022) 93542-93563.

[12] X. Wang, J. Lv, B.G. Kim, C. Maple, B.D. Parameshachari, A. Slowik, K. Li, Generative adversarial privacy for multimedia analytics across the IoT-edge continuum, IEEE Trans. Cloud Comput. 12 (4) (2024) 1260-1272.

[13] Q. Liu, R. Luo, H. Liang, Q. Liu, Energy-efficient joint computation offloading and resource allocation strategy for ISAC-aided 6G V2X networks, IEEE Trans. Green Commun. Netw. 7 (1) (2023) 413-423.

[14] Q. Guo, F. Tang, N. Kato, Federated reinforcement learning-based resource alloca-tion for D2D-aided digital twin edge networks in 6G industrial IoT, IEEE Trans. Ind. Inform. 19 (5) (2022) 7228-7236.

[15] S. Cho, Deep-learning-based resource allocation for 6G NOMA-assisted backscatter communications, IEEE Internet Things J. 11 (19) (2024) 32234-32243.

[16] T.J. Chua, W. Yu, J. Zhao, Resource allocation for mobile metaverse with the In-ternet of Vehicles over 6G wireless communications: a deep reinforcement learning approach, in: 2022 IEEE 8th World Forum on Internet of Things (WF-IoT), IEEE, 2022, pp. 1-7.

[17] Q. Guo, F. Tang, N. Kato, Federated reinforcement learning-based resource allocation in D2D-enabled 6G, IEEE Netw. 37 (5) (2022) 89-95.

[18] P. Qin, X. Wu, Z. Cai, X. Zhao, Y. Fu, M. Wang, S. Geng, Joint trajectory plan and resource allocation for UAV-enabled C-NOMA in air-ground integrated 6G heteroge-neous network, IEEE Trans. Netw. Sci. Eng. 10 (6) (2023) 3421-3434.

[19] D. Yang, W. Zhang, Q. Ye, C. Zhang, N. Zhang, C. Huang, H. Zhang, X. Shen, DetFed: dynamic resource scheduling for deterministic federated learning over time-sensitive networks, IEEE Trans. Mob. Comput. 23 (5) (2023) 5162-5178.

[20] O. Alnajar, A. Barnawi, TactiFlex: a Federated learning-enhanced in-content aware resource allocation flexible architecture for Tactile IoT in 6G networks, Eng. Appl. Artif. Intell. 136 (2024) 108934.

[21] Y. Zhou, Z. Jin, H. Shi, L. Shi, N. Lu, Flying IRS: QoE-driven trajectory optimization and resource allocation based on adaptive deployment for WPCNs in 6G IoT, IEEE Internet Things J. 11 (5) (2023) 9031-9046.

[22] Y. Fan, M. Fu, H. Jiang, X. Liu, Q. Liu, Y. Xu, L. Yi, W. Hu, Q. Zhuge, Point-to-multipoint coherent architecture with joint resource allocation for B5G/6G fron-thaul, IEEE Wirel. Commun. 29 (2) (2022) 100-106.

[23] V. Jain, B. Kumar, A. Gupta, Cybertwin-driven resource allocation using deep rein-forcement learning in 6G-enabled edge environment, J. King Saud Univ, Comput. Inf. Sci. 34 (8) (2022) 5708-5720.

[24] Z. Huang, D. Li, J. Cai, H. Lu, Collective reinforcement learning based resource al-location for digital twin service in 6G networks, J. Netw. Comput. Appl. 217 (2023) 103697.

[25] R. Wei, T. Qin, J. Huang, Y. Yang, J. Ren, L. Yang, Resource allocation scheduling scheme for task migration and offloading in 6G Cybertwin internet of vehicles based on DRL, IET Commun. 18 (18) (2024) 1244-1265.

[26] Z. Huang, F.R. Yu, J. Cai,Knowledge collaboration-based resource allocation in 6G IoT: a graph attention RL approach, IEEE Internet Things J. (2024).

[27] P. Bhattacharya, F. Patel, A. Alabdulatif, R. Gupta, S. Tanwar, N. Kumar, R. Sharma, A deep-Q learning scheme for secure spectrum allocation and resource management in 6G environment, IEEE Trans. Netw. Serv. Manag. 19 (4) (2022) 4989-5005.

[28] J. Huang, C. Yang, S. Zhang, F. Yang, O. Alfarraj, V. Frascolla, S. Mumtaz, K. Yu, Reinforcement learning based resource management for 6G-enabled mIoT with hy-pergraph interference model, IEEE Trans. Commun. 72 (7) (2024) 4179-4192.

[29] H.M.F. Noman, K. Dimyati, K.A. Noordin, E. Hanafi, A. Abdrabou, FeDRL-D2D: federated deep reinforcement learning-empowered resource allocation scheme for energy efficiency maximization in D2D-assisted 6G networks, IEEE Access 12 ( 2024) 109775-109792.

[30] S.A. Syed, K. Sheela Sobana Rani, G.B. Mohammad, G. Anil Kumar, K.K. Chennam, R. Jaikumar, Y. Natarajan, K. Srihari, U. Barakkath Nisha, V.P. Sundramurthy, Design of resources allocation in 6G cybertwin technology using the fuzzy neuro model in healthcare systems, J. Healthcare Eng. 2022 (1) (2022) 5691203.

[31] A. Nivetha, K.S. Preetha, Meta-algorithmic optimized power allocation in cybertwin-based sixth generation cooperative communication system, Results Eng. 23 (2024) 102740.

[32] M.M. Salim, Y. Pan, J.H. Park, Energy-efficient resource allocation in blockchain-based Cybertwin-driven 6G, J. Ambient Intell. Humaniz. Comput. 15 (1) (2024) 103-114.

[33] X. Wang, J. Lv, A. Slowik, B.G. Kim, B.D. Parameshachari, K. Li, G. Feng, Augmented intelligence of things for priority-aware task offloading in vehicular edge computing, IEEE Internet Things J. 11 (22) (2024) 36002-36013.