UAV-assisted MEC offloading strategy with peak AOI boundary optimization: A method based on DDQN

Abstract

In response to the requirements for large-scale device access and ultra-reliable and low-latency communication in the power internet of things, unmanned aerial vehicle-assisted multi-access edge computing can be used to realize flexible access to power services and update large amounts of information in a timely manner. By considering factors such as machine communication traffic, MAC competition access, and information freshness, this paper develops a cross-layer computing framework in which the peak Age of Information (AoI) provides a statistical delay boundary in the finite blocklength regime. We also propose a deep machine learning-based multi-access edge computing offloading algorithm. First, a traffic arrival model is established in which the time interval follows the Beta distribution, and then a business service model is proposed based on the carrier sense multiple access with collision avoidance algorithm. The peak AoI boundary performance of multiple access is evaluated according to stochastic network calculus theory. Finally, an unmanned aerial vehicle-assisted multi-level offloading model with cache is designed, in which the peak AoI violation probability and energy consumption provide the optimization goals. The optimal offloading strategy is obtained using deep reinforcement learning. Compared with baseline schemes based on non-cooperative game theory with stochastic learning automata and random edge unloading, the proposed algorithm improves the overall performance by approximately 3.52 % and 20.73 %, respectively, and provides superior deterministic offloading performance by using the peak AoI boundary.

Keywords

Power internet of things / Ultra-reliable low-latency communication / Unmanned aerial vehicle / Multi-access edge computing / Age of information / Stochastic network calculus / Deep reinforcement learning

References

[1] Z. Zhai, L. Jia, Y. Wang, Y. Ma, W. Jing, Z. Zhang, Research on Ubiquitous power internet of things architecture, in: 2019 IEEE 3rd Conference on Energy Internet and Energy System Integration ( EI2), 2019, pp. 435-439.

[2] D. Han, T. Shi, Secrecy capacity maximization for a UAV-assisted MEC system, China Communications 17 (10) (2020) 64-81.

[3] A.M. Seid, G.O. Boateng, B. Mareri, G. Sun, W. Jiang, Multi-agent DRL for task offloading and resource allocation in multi-UAV enabled IoT edge network, IEEE Transactions on Network and Service Management 18 (4) (2021) 4531-4547.

[4] A.M. Seid, G.O. Boateng, S. Anokye, T. Kwantwi, G. Sun, G. Liu, Collaborative computation offloading and resource allocation in multi-UAV-assisted IoT networks: a deep reinforcement learning approach, IEEE Internet Things J. 8 (15)(2021) 12203-12218.

[5] H. Xiao, Z. Hu, K. Yang, Y. Du, D. Chen, An energy-Aware joint Routing and task allocation algorithm in MEC systems assisted by multiple UAVs, in: 2020 International Wireless Communications and Mobile Computing (IWCMC), 2020, pp. 1654-1659.

[6] Z. Chen, N. Xiao, D. Han, A multilevel mobile fog computing offloading model based on UAV-assisted and Heterogeneous network, Wireless Commun. Mobile Comput. 11 (1) (2020) 11.

[7] H. Baek, J. Lim, Time Mirroring based CSMA/CA for improving performance of UAV-relay network system, IEEE Syst. J. 13 (4) (2019) 4478-4481.

[8] 3GPP TR 37.868, Study on RAN Improvements for Machine Type Communication[S], 2011.

[9] O. Arouk, A. Ksentini, T. Taleb, Performance analysis of RACH Procedure with Beta traffic-Activated machine-type-communication, in: 2015 IEEE Global Communications Conference (GLOBECOM), 2015, pp. 1-6.

[10] N.Y. Ermolova, O. Tirkkonen, Using Beta distributions for modeling distances in random finite networks, IEEE Commun. Lett. 20 (2) (2016) 308-311.

[11] Y. Polyanskiy, H.V. Poor, S. Verdu, Channel coding rate in the finite blocklength regime, IEEE Trans. Inf. Theor. 56 (5) (2010) 2307-2359.

[12] Y. Hu, J. Gross, A. Schmeink, On the performance advantage of relaying under the finite blocklength regime, IEEE Commun. Lett. 19 (5) (2015) 779-782.

[13] W. Yang, G. Caire, G. Durisi, Y. Polyanskiy, Optimum power control at finite blocklength, IEEE Trans. Inf. Theor. 61 (9) (2015) 4598-4615.

[14] K. Wang, C. Pan, H. Ren, W. Xu, L. Zhang, A. Nallanathan, Packet error probability and effective throughput for ultra-reliable and low-latency UAV communications, IEEE Trans. Commun. 69 (1) (2021) 73-84.

[15] S.R. Pandey, K. Kim, M. Alsenwi, Y.K. Tun, Z. Han, C.S. Hong, Latency-sensitive service Delivery with UAV-assisted 5G networks, IEEE Wireless Communications Letters 10 (7) (2021) 1518-1522.

[16] C. Pan, H. Ren, Y. Deng, M. Elkashlan, A. Nallanathan, Joint blocklength and location optimization for URLLC-Enabled UAV relay systems, IEEE Commun. Lett. 23 (3) (2019) 498-501.

[17] Z. Dong, J. Wei, X. Chen, P. Zheng, Energy Efficiency optimization and resource allocation of cross-layer Broadband wireless communication system, IEEE Access 8 (2020) 50740-50754.

[18] J. Liu, X. Wang, B. Bai, H. Dai,Age-optimal trajectory planning for UAV-assisted data collection, in: IEEE INFOCOM 2018 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 2018, pp. 553-558.

[19] H. Wang, S. Duan, X. Xie, Scheduling method for multi-channel wireless networks based on optimization of age of information, J. Electron. Inf. Technol. (Feb. 2021) 1-8.

[20] M. Sun, X. Xu, X. Qin, P. Zhang, AoI-energy-aware UAV-assisted data collection for IoT networks: a deep reinforcement learning method, IEEE Internet Things J. 8 (24) (2021) 17275-17289.

[21] C. Xu, H.H. Yang, X. Wang, T.Q.S. Quek, Optimizing information freshness in computing-Enabled IoT networks, IEEE Internet Things J. 7 (2) (2020) 971-985.

[22] M. Mei, Q. Yang, M. Yao, M. Qin, K.S. Kwak,Performance of secure UAV transmission: delay-secrecy analysis with channel Uncertainty, in: 2021 IEEE Wireless Communications and Networking Conference (WCNC), 2021, pp. 1-6.

[23] G. Yang, M. Xiao, H.V. Poor, Low-latency Millimeter-Wave communications: traffic dispersion or network Densification? IEEE Trans. Commun. 66 (8) (2018) 3526-3539.

[24] H. Forssell, R. Thobaben, H. Al-Zubaidy, J. Gross, Physical layer Authentication in mission-critical MTC networks: a security and delay performance analysis, IEEE J. Sel. Area. Commun. 37 (4) (2019) 795-808.

[25] X. Zhang, J. Wang, H.V. Poor, AoI-driven statistical delay and error-rate bounded QoS Provisioning for mURLLC over UAV-Multimedia 6G mobile networks using FBC, IEEE J. Sel. Area. Commun. 39 (11) (2021) 3425-3443.

[26] S.R. Pandey, K. Kim, M. Alsenwi, Y.K. Tun, Z. Han, C.S. Hong, Latency-sensitive service Delivery with UAV-assisted 5G networks, IEEE Wireless Communications Letters 10 (7) (2021) 1518-1522.

[27] S. Wu, W. Mao, X. Wang, Performance study on a CSMA/CA-Based MAC protocol for multi-user MIMO wireless LANs, IEEE Trans. Wireless Commun. 13 (6) (2014) 3153-3166.

[28] O. Azuaje, PhD Forum: delay Guarantees of a mobile wireless sensor network using stochastic network calculus, in: 2021 IEEE 22nd International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM), 2021, pp. 221-222.

[29] O. Arouk, A. Ksentini, T. Taleb, Performance analysis of RACH Procedure with Beta traffic-Activated machine-type-communication, in: 2015 IEEE Global Communications Conference (GLOBECOM), 2015, pp. 1-6.

[30] L. Jiang, L. Yu, Z. Chen,Mobility performance analysis in wireless network based on a cross-layer network calculus model, in:2011 11th International Symposium on Communications & Information Technologies (ISCIT), 2011, pp. 126-130.

[31] Y. Zhu, D. Zhou, M. Sheng, J. Li, Z. Han, Stochastic delay analysis for Satellite data relay networks with Heterogeneous traffic and transmission links, IEEE Trans. Wireless Commun. 20 (1) (2021) 156-170.

[32] I.A. Elgendy, W.-Z. Zhang, Y. Zeng, H. He, Y.-C. Tian, Y. Yang, Efficient and secure multi-user multi-task computation offloading for mobile-edge computing in mobile IoT networks, in: IEEE Transactions on Network and Service Management, vol. 17, Dec. 2020, pp. 2410-2422, 4.

[33] Z. Huang, X. Xu, DQN-based relay Deployment and trajectory planning in Consensus-based multi-UAVs Tracking network, in: 2021 IEEE International Conference on Communications Workshops (ICC Workshops), 2021, pp. 1-7.

[34] A. Anand, G. de Viviana, S. Shekita, Joint scheduling of URLLC and eMBB traffic in 5G wireless networks, IEEE/ACM Trans. Netw. 28 (2) (2020) 477-490.

[35] Fang-ming Deng, et al.Yun Shan, X.I.E. Zhong-xin, The power inspection and unloading strategy of UAV based on game theory and reinforcement learning, Power Syst. Technol. 45 (9) (2021) 3649-3657.