Learning-based user association and dynamic resource allocation in multi-connectivity enabled unmanned aerial vehicle networks

Zhipeng Cheng; Minghui Liwang; Ning Chen; Lianfen Huang; Nadra Guizani; Xiaojiang Du

doi:10.1016/j.dcan.2022.05.026

›› 2024, Vol. 10 ›› Issue (1) :53 -62. DOI: 10.1016/j.dcan.2022.05.026

Special issue on intelligent communications technologies for B5G

research-article

Learning-based user association and dynamic resource allocation in multi-connectivity enabled unmanned aerial vehicle networks

Author information +

History +

PDF

Abstract

Unmanned Aerial Vehicles (UAVs) as aerial base stations to provide communication services for ground users is a flexible and cost-effective paradigm in B5G. Besides, dynamic resource allocation and multi-connectivity can be adopted to further harness the potentials of UAVs in improving communication capacity, in such situations such that the interference among users becomes a pivotal disincentive requiring effective solutions. To this end, we investigate the Joint UAV-User Association, Channel Allocation, and transmission Power Control (J-UACAPC) problem in a multi-connectivity-enabled UAV network with constrained backhaul links, where each UAV can determine the reusable channels and transmission power to serve the selected ground users. The goal was to mitigate co-channel interference while maximizing long-term system utility. The problem was modeled as a cooperative stochastic game with hybrid discrete-continuous action space. A Multi-Agent Hybrid Deep Rein- forcement Learning (MAHDRL) algorithm was proposed to address this problem. Extensive simulation results demonstrated the effectiveness of the proposed algorithm and showed that it has a higher system utility than the baseline methods.

Keywords

UAV-user association / Multi-connectivity / Resource allocation / Power control / Multi-agent deep reinforcement learning

Cite this article

Download citation ▾

Zhipeng Cheng, Minghui Liwang, Ning Chen, Lianfen Huang, Nadra Guizani, Xiaojiang Du. Learning-based user association and dynamic resource allocation in multi-connectivity enabled unmanned aerial vehicle networks. , 2024, 10(1): 53-62 DOI:10.1016/j.dcan.2022.05.026

登录浏览全文

4963

注册一个新账户忘记密码

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	M. Liwang, Z. Gao, X. Wang, Let's trade in the future! A futures-enabled fast resource trading mechanism in edge computing-assisted UAV networks, IEEE J. Sel. Area. Commun. 39 (11) (2021) 3252-3270.

[2]	L. Xie, J. Xu, Y. Zeng, Common throughput maximization for UAV-enabled interference channel with wireless powered communications, IEEE Trans. Commun. 68 (5) (2020) 3197-3212.

[3]	W. Mei, Q. Wu, R. Zhang, Cellular-connected UAV: uplink association, power control and interference coordination, IEEE Trans. Wireless Commun. 18 (11) (2019) 5380-5393.

[4]	M. Kamel, W. Hamouda, A. Youssef, Performance analysis of multiple association in ultra-dense networks, IEEE Trans. Commun. 65 (9) (2017) 3818-3831.

[5]	Z. Cheng, M. Liwang, N. Chen, H. Lin, Z. Gao, L. Huang, Learning-based joint user- AP association and resource allocation in ultra dense network, in: Proceedings of the 2020 IEEE 91st Veh. Technol. Conf, 2020, pp. 1-5.

[6]	M. Gapeyenko, V. Petrov, D. Moltchanov, S. Andreev, N. Himayat, Y. Koucheryavy, Flexible and reliable UAV-assisted backhaul operation in 5G mmWave cellular networks, IEEE J. Sel. Area. Commun. 36 (11) (2018) 2486-2496.

[7]	J. Cui, Y. Liu, A. Nallanathan, Multi-agent reinforcement learning-based resource allocation for UAV networks, IEEE Trans. Wireless Commun. 19 (2) (2020) 729-743.

[8]	3GPP TSG-RAN2 NR,R2-1700172: Evaluation on Packet Duplication in Multi- Connectivity, Jan. 2017.

[9]	A. Rabitsch, et al., Utilizing multi-connectivity to reduce latency and enhance availability for vehicle to infrastructure communication, IEEE Trans. Mobile Comput. 21 (5) (2022) 1874-1891.

[10]	R.P. Antonioli, J. Pettersson, T.F. Maciel, Split responsibility scheduler for multi- connectivity in 5G cellular networks, IEEE Netw. 34 (6) (2020) 212-219.

[11]	Q. Zhang, H. Wang, Z. Feng, Z. Han, Many-to-many matching-theory-based dynamic bandwidth allocation for UAVs, IEEE Internet Things J. 8 (12) (2021) 9995-10009.

[12]	L. Wang, Y.L. Che, J. Long, L. Duan, K. Wu, Multiple access Mmwave design for UAV-aided 5G communications, IEEE Wireless Commun. 26 (1) (2019) 64-71.

[13]	L. Wang, H. Yang, J. Long, K. Wu, J. Chen, Enabling ultra-dense UAV-aided network with overlapped spectrum sharing: potential and approaches, IEEE Netw. 32 (5) (2018) 85-91.

[14]	H. Wang, G. Ding, F. Gao, J. Chen, J. Wang, L. Wang, Power control in UAV- supported ultra dense networks: communications, caching, and energy transfer, IEEE Commun. Mag. 56 (6) (2018) 28-34.

[15]	E. Bozkaya, B. Canberk,BS-on-air: optimum UAV localization for resilient ultra dense networks, in: Proceeding of the 2018 IEEE INFOCOM WKSHPS, 2018, pp. 877-881.

[16]	X. Xi, X. Cao, P. Yang, J. Chen, T. Quek, D. Wu, Joint user association and UAV location optimization for UAV-aided communications, IEEE Wireless Commun. Lett. 8 (6) (2019) 1688-1691.

[17]	Y. Sun, T. Wang, S. Wang, Location optimization and user association for unmanned aerial vehicles assisted mobile networks, IEEE Trans. Veh. Technol. 68 (10) (2019) 10056-10065.

[18]	Y. Qian, F. Wang, J. Li, L. Shi, K. Cai, F. Shu, User association and path planning for UAV-aided mobile edge computing with energy restriction, IEEE Wireless Commun. Lett. 8 (5) (2019) 1312-1315.

[19]	S.A.W. Shah, M.G. Khafagy, T. Khattab, M.O. Hasna, K. Abualsaud, ICIC-enabled association and channel selection for UAV-BSs based on user locations and demands, in: Proceedings of the 2019 IEEE Wirel. Commun. Netw. Conf. Work, WCNCW, 2019, pp. 1-7.

[20]	C. Qiu, Z. Wei, Z. Feng, P. Zhang, Joint resource allocation, placement and user association of multiple UAV-mounted base stations with in-band wireless backhaul, IEEE Wireless Commun. Lett. 8 (6) (2019) 1575-1578.

[21]	C. Qiu, Z. Wei, X. Yuan, Z. Feng, P. Zhang, Multiple UAV-mounted base station placement and user association with joint fronthaul and backhaul optimization, IEEE Trans. Commun. 68 (9) (2020) 5864-5877.

[22]	X. Guan, Y. Huang, C. Dong, Q. Wu, User association and power allocation for UAV- assisted networks: a distributed reinforcement learning approach, China Commun. 17 (12) (2020) 110-122.

[23]	S. Yin, L. Li, F.R. Yu, Resource allocation and basestation placement in downlink cellular networks assisted by multiple wireless powered UAVs, IEEE Trans. Veh. Technol. 69 (2) (2020) 2171-2184.

[24]	Q. Li, M. Ding, C. Ma, C. Liu, Z. Lin, Y. Liang,A reinforcement learning based user association algorithm for UAV networks, in:Proceedings of 2018 28th Int. Telecommun. Networks Appl. Conf, ITNAC, 2018, pp. 1-6.

[25]	X. Liu, Y. Liu, Y. Chen, Reinforcement learning in multiple-UAV networks: deployment and movement design, IEEE Trans. Veh. Technol. 68 (8) (2019) 8036-8049.

[26]	J. Hu, H. Zhang, L. Song, Z. Han, H.V. Poor, Reinforcement learning for a cellular internet of UAVs: protocol design, trajectory control, and resource management, IEEE Wireless Commun. 27 (1) (2020) 116-123.

[27]	F. Wu, H. Zhang, J. Wu, L. Song, Cellular UAV-to-device communications: trajectory design and mode selection by multi-agent deep reinforcement learning, IEEE Trans. Commun. 68 (7) (2020) 4175-4189.

[28]	N. Zhao, Z. Liu, Y. Cheng, Multi-agent deep reinforcement learning for trajectory design and power allocation in multi-UAV networks, IEEE Access 8 (2020) 139670-139679.

[29]	U. Challita, W. Saad, C. Bettstetter, Interference management for cellular-connected UAVs: a deep reinforcement learning approach, IEEE Trans. Wireless Commun. 18 (4) (2019) 2125-2140.

[30]	C. Qiu, Z. Wei, Z. Feng, P. Zhang, Backhaul-aware trajectory optimization of fixed- wing UAV-mounted base station for continuous available wireless service, IEEE Access 8 (2020) 60940-60950.

[31]	A. Al-Hourani, S. Kandeepan, S. Lardner, Optimal LAP altitude for maximum coverage, IEEE Wireless Commun. Lett. 3 (6) (2014) 569-572.

[32]	J. Xiong, et al., Parametrized deep q-networks learning: reinforcement learning with discrete-continuous hybrid action space, arXiv:1810.06394. [Online] Available: https://arxiv.org/abs/1810.06394, 2018.

[33]	Haotian Fu, et al., Deep multi-agent reinforcement learning with discrete- continuous hybrid action spaces,in:Proceedings of the 28th 2019 IJCAI, 2019, pp. 2329-2335.

[34]	R. Lowe, Y. Wu, A. Tamar, J. Harb, O.P. Abbeel, I. Mordatch,Multi-agent actor- critic for mixed cooperative-competitive environments, in:Proceedings of the Adv. Neural Inf. Process. Syst, 2017, pp. 6379-6390.

[35]	Y. Zhang, Z. Mou, F. Gao, J. Jiang, R. Ding, Z. Han, UAV-enabled secure communications by multi-agent deep reinforcement learning, IEEE Trans. Veh. Technol. 69 (10) (2020) 11599-11611.

[36]	T. Rashid, M. Samvelyan, C.S. de Witt, G. Farquhar, J. Foerster, S. Whiteson, QMIX: monotonic value function factorisation for deep multi-agent reinforcement learning,in: Proceedings of the 35th ICML, 2018, pp. 4292-4301.

[37]	A. Gao, T. Geng, S.X. Ng, W. Liang, A continuous policy learning approach for hybrid offloading in backscatter communication, IEEE Commun. Lett. 25 (2) (2021) 523-527.

[38]	Y. Al-Eryani, M. Akrout, E. Hossain, Multiple access in cell-free networks: outage performance, dynamic clustering, and deep reinforcement learning-based design, IEEE J. Sel. Area. Commun. 39 (4) (2021) 1028-1042.