Channel assignment and power allocation for throughput improvement with PPO in B5G heterogeneous edge networks

Abstract

In Beyond the Fifth Generation (B5G) heterogeneous edge networks, numerous users are multiplexed on a channel or served on the same frequency resource block, in which case the transmitter applies coding and the receiver uses interference cancellation. Unfortunately, uncoordinated radio resource allocation can reduce system throughput and lead to user inequity, for this reason, in this paper, channel allocation and power allocation problems are formulated to maximize the system sum rate and minimum user achievable rate. Since the construction model is non-convex and the response variables are high-dimensional, a distributed Deep Reinforcement Learning (DRL) framework called distributed Proximal Policy Optimization (PPO) is proposed to allocate or assign resources. Specifically, several simulated agents are trained in a heterogeneous environment to find robust behaviors that perform well in channel assignment and power allocation. Moreover, agents in the collection stage slow down, which hinders the learning of other agents. Therefore, a preemption strategy is further proposed in this paper to optimize the distributed PPO, form DP-PPO and successfully mitigate the straggler problem. The experimental results show that our mechanism named DP-PPO improves the performance over other DRL methods.

Keywords

B5G / Heterogeneous edge networks / PPO / Channel assignment / Power allocation / Throughput

References

[1] W. Wu, F. Zhou, R.Q. Hu, B. Wang, Energy-efficient resource allocation for secure NOMA-enabled mobile edge computing networks, IEEE Trans. Commun. 68 (1)(2020) 493-505.

[2] X. Zhang, J. Wang, H.V. Poor, Statistical delay and error-rate bounded QoS provisioning over mmWave cell-free M-MIMO and FBC-HARQ-IR based 6G wireless networks, IEEE J. Sel. Area. Commun. 38 (8) (2020) 1661-1677.

[3] H. Zhang, M. Feng, K. Long, G.K. Karagiannidis, V.C.M. Leung, H.V. Poor, Energy efficient resource management in SWIPT enabled heterogeneous networks with NOMA, IEEE Trans. Commun. 19 (2) (2020) 835-845.

[4] Y. Huang, C. Liu, Y. Xiao, S. Liu, Separate power allocation and control method based on multiple power channels for wireless power transfer, IEEE Trans. Power Electron. 35 (9) (2020) 9046-9056.

[5] X. Wang, Y. Zhang, R. Shen, Y. Xu, F.C. Zheng, DRL-based energy-efficient resource allocation frameworks for uplink NOMA systems, IEEE Internet Things J. 7 (8) (2020) 7279-7294.

[6] M. Zhang, Y. Chen, Z. Xia, J. Du, W. Susilo, PPO-DFK: a privacy-preserving optimization of distributed fractional knapsack with application in secure footballer configurations, IEEE Syst. J. 15 (1) (2021) 759-770.

[7] R. Kettimuthu, V. Subramani, S. Srinivasan, T. Gopalasamy, D.K. Panda, P. Sadayappan,Selective preemption strategies for parallel job scheduling, in:Proceedings of the Parallel Processing, 2002. Canada.

[8] J. Choi, J. Jung, I.C. Park, Area-efficient approach for generating quantized Gaussian noise, IEEE Trans. Circ. Syst. 63 (7) (2016) 1005-1013.

[9] D. Cao, D. Zhang, P. Chen, N. Liu, W. Kang, D. Gündüz, Coded caching with asymmetric cache sizes and link qualities: the two-user case, IEEE Trans. Commun. 67 (9) (2019) 6112-6126.

[10] Z. Yang, X. Lei, Z. Ding, P. Fan, G.K. Karagiannidis, On the uplink sum rate of SCMA system with randomly deployed users, IEEE Wireless Commun. Letters 6 (3) (2017) 338-341.

[11] M. Monemizadeh, H. Fehri, On the devroye-mitran-tarokh and rini-tuninetti-devroye achievable rate regions for the cognitive interference channels, IEEE Commun. Lett. 21 (12) (2017) 2662-2665.

[12] W. Meng, Q. Zheng, Y. Shi, G. Pan, An off-policy trust region policy optimization method with monotonic improvement guarantee for deep reinforcement learning, IEEE Transact. Neural Networks Learn. Syst. 33 (5) (2022) 2223-2235.

[13] N. Bouhlel, A. Dziri, Kullback-leibler divergence between multivariate generalized Gaussian distributions, IEEE Signal Process. Lett. 26 (7) (2019) 1021-1025.

[14] M. Ayub, A. Hussain, G. Jawad, B.I. Kwon, Brushless operation of a wound-field synchronous machine using a novel winding scheme, IEEE Trans. Magn. 55 (6)(2019) 8201104.

[15] S. Bruce, Z. Li, H.C. Yang, S. Mukhopadhyay, Nonparametric distributed learning architecture for big data: algorithm and applications, IEEE Trans. on Big Data. 5 (2)(2019) 166-179.

[16] H. Guo, A. Liu, V.K.N. Lau, Analog gradient aggregation for federated learning over wireless networks: customized design and convergence analysis, IEEE Internet Things J. 8 (1) (2021) 197-210.

[17] S. Basodi, C. Ji, H. Zhang, Y. Pan, Gradient amplification: an efficient way to train deep neural networks, Big Data Mining and Analytics 3 (3) (2020) 196-207.

[18] Y. Wang, B. Xin, J. Chen, An adaptive memetic algorithm for the joint allocation of heterogeneous stochastic resources, IEEE Trans. Cybern. 52 (11) (2022) 11526-11538.

[19] M. Chen, L. Wang, J. Chen, X. Wei, L. Lei, A computing and content delivery network in the smart city: scenario, framework, and analysis, IEEE Network 33 (2)(2019) 89-95.

[20] M. Chen, X. Wei, J. Chen, L. Wang, L. Zhou, Integration and provision for city public service in smart city cloud union: architecture and analysis, IEEE Wireless Commun. 27 (2) (2020) 148-154.