Eﬃcient user scheduling in mmWave networks: leveraging knowledge transfer with channel knowledge map✩

Chunlong He; Peihong He; Xingquan Li

doi:10.1016/j.dcan.2025.09.003

›› 2026, Vol. 12 ›› Issue (2) :319 -331. DOI: 10.1016/j.dcan.2025.09.003

Regular Papers

research-article

Eﬃcient user scheduling in mmWave networks: leveraging knowledge transfer with channel knowledge map^✩

Author information +

^a Guangdong Key Laboratory of Intelligent Information Processing, Shenzhen University, Shenzhen 518060, China

^b National Mobile Communications Research Laboratory, Southeast University, Nanjing 214135, China

^c Shenzhen Institute of Information Technology, Shenzhen 518172, China

^* E-mail address: 2300432039@email.szu.edu.cn (P. He).

Chunlong He received the M.S. degree in Communication and Information Science from Southwest Jiaotong University, Chengdu, China, in 2010 and the Ph.D. degree from Southeast University, Nanjing, China, in 2014. From September 2012 to September 2014, he was a Visiting Student with the School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA. Since 2015, he has been with the College of In-formation Engineering, Shenzhen University, where he is currently an Associate Professor. His research interests include communication and signal processing, green communica-tion systems, channel estimation algorithms, and limited feedback techniques. Dr. He is a member of the Institute of Electronics, Information, and Communication Engineering. He is currently an Associate Editor of IEEE Access.

Peihong He is currently pursuing a master’s degree at the School of Electronic and Information Engineering, Shenzhen University, Guangdong, China. His research interests mainly focus on channel knowledge maps and machine learning.

Xingquan Li received the Ph.D. degree from the College of Electronics and Informa-tion Engineering, Shenzhen University, Shenzhen, China, in 2019. He was a Postdoctoral Researcher with the College of Electronics and Information Engineering, Shenzhen Uni-versity. He is currently a Lecturer with the School of Microelectronics, Shenzhen Institute of Information Technology, Shenzhen. His research interests include cooperative commu-nications, green communications, and resource allocation.

Show less

History +

PDF

Abstract

This paper proposes a Deep Reinforcement Learning (DRL) algorithm for user scheduling in Millimeter Wave (mmWave) networks, which utilizes Channel Knowledge Map (CKM) for knowledge transfer to enhance the learning of scheduling strategies. The user scheduling and link configuration problems are modeled as a multi-queue system. Each queue represents the data demand of an individual user. This setup allows the base station to make dynamic scheduling decisions based on changing environmental conditions. This approach facilitates eﬃcient management of user-specific requirements while addressing the challenges posed by dynamic network environments. Our model incorporates relay selection, codebook selection, and beam tracking to support flexible and eﬃcient resource allocation. In contrast to traditional channel model-based optimization, we design algorithms for scheduling policy pre-training using CKMs, which provide information about the channel between specific pairs of locations. Specifically, we assume that the CKM is fully available to allow the complex scheduling network to have a better starting point or follow a more favorable gradient direction through knowledge migration. This integration of CKM with knowledge transfer significantly accelerates DRL convergence and enhances performance stability. Simulation results confirmed the eﬀectiveness of the proposed approach. Relative to the baseline methods, integrating CKM with knowledge transfer accelerated the convergence of the DRL algorithm by approximately 20%, maintained the delay within 30 milliseconds, and reduced the average queue length by nearly 30%.

Keywords

Millimeter wave / User scheduling / Knowledge transfer / Channel knowledge map / Deep reinforcement learning

Cite this article

Download citation ▾

Chunlong He, Peihong He, Xingquan Li. Eﬃcient user scheduling in mmWave networks: leveraging knowledge transfer with channel knowledge map^✩. , 2026, 12(2): 319-331 DOI:10.1016/j.dcan.2025.09.003

登录浏览全文

4963

注册一个新账户忘记密码

CRediT authorship contribution statement

Chunlong He: Writing-review & editing, Writing-original draft, Software, Funding acquisition, Conceptualization. Peihong He: Writing-original draft, Software, Methodology, Conceptualization. Xingquan Li: Writing-review & editing, Conceptualization.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This work was supported in part by the Shenzhen Basic Research Program under Grant JCYJ20220531103008018, and Grants 20231120142345001 and 20231127144045001, and the Natural Sci-ence Foundation of China under Grant U20A20156.

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	J. Zhao, R. Ren, Y. Wu, Q. Zhang, W. Xu, D. Wang, L. Fan, Seattention-residual based channel estimation for mmwave massive mimo systems in iov scenarios, Digit. Commun. Netw. 11 (3) (2025) 778-786.

[2]	T.S. Rappaport, S. Sun, R. Mayzus, H. Zhao, Y. Azar, K. Wang, G.N. Wong, J.K. Schulz, M. Samimi, F. Gutierrez,Millimeter wave mobile communications for 5G cellular: it will work!, IEEE Access 1 (2013) 335-349.

[3]	E. Driouch, W. Ajib, Downlink scheduling and resource allocation for cognitive radio MIMO networks, IEEE Trans. Veh. Technol. 62 (8) (2013) 3875-3885.

[4]	X. Rao, V.K.N. Lau, Distributed compressive CSIT estimation and feedback for FDD multi-user massive MIMO systems, IEEE Trans. Signal Process. 62 (12) (2014) 3261-3271.

[5]	Z. Zhu, K. Lin, A.K. Jain, J. Zhou, Transfer learning in deep reinforcement learning: a survey, IEEE Trans. Pattern Anal. Mach. Intell. 45 (11) (2023) 13344-13362.

[6]	W. Zhao, J.P. Queralta, T. Westerlund, Sim-to-real transfer in deep reinforcement learning for robotics: a survey, in: IEEE Symposium Series on Computational Intel-ligence (SSCI), Canberra, ACT, Australia, 2020, pp. 737-744.

[7]	Y. Zeng, X. Xu, Toward environment-aware 6G communications via channel knowl-edge map, IEEE Wirel. Commun. 28 (3) (2021) 84-91.

[8]	Z. Dai, D. Wu, Z. Dong, K. Li, D. Ding, S. Wang, Y. Zeng, Prototyping and experi-mental results for environment-aware millimeter wave beam alignment via channel knowledge map, IEEE Trans. Veh. Technol. 73 (11) (2024) 16805-16816.

[9]	C. Zhan, H. Hu, Z. Liu, J. Wang, N. Cheng, S. Mao, Aerial video streaming over 3D cellular networks: an environment and channel knowledge map approach, IEEE Trans. Wirel. Commun. 23 (2) (2024) 1432-1446.

[10]	C. Zhang, Z. Zhou, Y. Yang, Y. Qiu, Z. Yu, X. Xu, Y. Zeng, Real-time demo of ISAC-based channel knowledge map prototyping system, in: IEEE/CIC Int. Conf. Commun. China (ICCC), Hangzhou, China, 2024, pp. 295-296.

[11]	E. Moeen Taghavi, R. Hashemi, N. Rajatheva, M. Latva-Aho, Environment-aware joint active/passive beamforming for RIS-aided communications leveraging channel knowledge map, IEEE Commun. Lett. 27 (7) (2023) 1824-1828.

[12]	X. Xu, Y. Zeng, How much data is needed for channel knowledge map construction?, IEEE Trans. Wirel. Commun. 23 (10) (2024) 13011-13021.

[13]	D. Ding, D. Wu, Y. Zeng, S. Jin, R. Zhang, Environment-aware beam selection for IRS-aided communication with channel knowledge map, in: IEEE Globecom Workshops (GC Wkshps), Madrid, Spain, 2021, pp. 1-6.

[14]	D. Wu, Y. Zeng, S. Jin, R. Zhang, Environment-aware hybrid beamforming by leveraging channel knowledge map, IEEE Trans. Wirel. Commun. 23 (5) (2024) 4990-5005.

[15]	S.A. Serrano, J. Martinez-Carranza, L.E. Sucar, Knowledge transfer for cross-domain reinforcement learning: a systematic review, IEEE Access 12 (2024) 114552-114572.

[16]	E. Chalmers, E.B. Contreras, B. Robertson, A. Luczak, A. Gruber, Learning to predict consequences as a method of knowledge transfer in reinforcement learning, IEEE Trans. Neural Netw. Learn. Syst. 29 (6) (2018) 2259-2270.

[17]	J. Xu, J. Bu, J. Li, A knowledge transfer framework based on deep-reinforcement learning for multistage construction projects, IEEE Trans. Eng. Manag. 71 (2024) 11361-11374.

[18]	Y. Hou, Y.-S. Ong, L. Feng, J.M. Zurada, An evolutionary transfer reinforcement learning framework for multiagent systems, IEEE Trans. Evol. Comput. 21 (4) (2017) 601-615.

[19]	H. Shi, J. Li, J. Mao, K.-S. Hwang, Lateral transfer learning for multiagent reinforce-ment learning, IEEE Trans. Cybern. 53 (3) (2023) 1699-1711.

[20]	W. Liu, L. Dong, J. Liu, C. Sun, Knowledge transfer in multi-agent reinforcement learning with incremental number of agents, J. Syst. Eng. Electron. 33 (2) (2022) 447-460.

[21]	I. Jang, H. Kim, D. Lee, Y.-S. Son, S. Kim, Knowledge transfer for on-device deep reinforcement learning in resource constrained edge computing systems, IEEE Access 8 (2020) 146588-146597.

[22]	Q. Hu, D.M. Blough, Relay selection and scheduling for millimeter wave backhaul in urban environments, in: Proc.-IEEE Int. Conf. Mob. Ad Hoc Sens. Syst. (MASS), Orlando, FL, USA, 2017, pp. 206-214.

[23]	J. Park, Y.I. Eom, URS: user-based resource scheduling for multi-user surface com-puting systems, IEEE Trans. Consum. Electron. 65 (3) (2019) 426-433.

[24]	G. Lee, Y. Sung, A new approach to user scheduling in massive multi-user MIMO broadcast channels, IEEE Trans. Commun. 66 (4) (2018) 1481-1495.

[25]	L.P. Qian, Y. Wu, J. Wang, W. Zhang, Energy-eﬃcient distributed user schedul-ing in relay-assisted cellular networks, IEEE Trans. Wirel. Commun. 15 (6) (2016) 4060-4073.

[26]	X. Chen, F.-K. Gong, H. Zhang, G. Li, Cooperative user scheduling in massive MIMO systems, IEEE Access 6 (2018) 21910-21923.

[27]	J. Zhao, F. Hu, Y. Gong, D. Wang, Downlink resource intelligent scheduling in mmwave cell-free urban vehicle network, IEEE Trans. Veh. Technol. 73 (10) (2024) 15525-15537.

[28]	Y. Zhang, S. Basu, S. Shakkottai, R.W. Heath Jr, Mmwave codebook selection in rapidly-varying channels via multinomial Thompson sampling, in: Proc. 22nd ACM Int. Symp. Mobile Ad Hoc Netw. Comput., Shanghai, China, 2021, pp. 151-160.

[29]	Y. Wang, G. de Veciana, Temporal dynamics of mobile blocking in millimeter wave based wearable networks, in: Int. Symp. Model. Optim. Mob. Ad Hoc, Wirel. Net-works (WiOpt), Paris, France, 2017, pp. 1-8.

[30]	V. Mnih, Asynchronous methods for deep reinforcement learning, Proc. Int.Conf. Mach. Learn., vol. 48, 2016, pp. 1928-1937.

[31]	Y. Gu, Y. Cheng, C.L.P. Chen, X. Wang, Proximal policy optimization with policy feedback, IEEE Trans. Syst. Man Cybern. Syst. 52 (7) (2022) 4600-4610.

[32]	J. Pan, X. Wang, Y. Cheng, Q. Yu, Multisource transfer double DQN based on actor learning, IEEE Trans. Neural Netw. Learn. Syst. 29 (6) (2018) 2227-2238.

[33]	C. Qiu, Y. Hu, Y. Chen, B. Zeng, Deep deterministic policy gradient (DDPG)-based energy harvesting wireless communications, IEEE Internet Things J. 6 (5) (2019) 8577-8588.

[34]	S. Salehpour, A. Eskandari, A. Nedaei, J. Milimonfared, M. Aghaei, A novel deep reinforcement learning and trust region policy optimization based anomaly detec-tion in solar power generation, in: IEEE Conference Photovoltaic Specialists (PVSC), Seattle, WA, USA, 2024, pp. 1693-1695.

[35]	J. Schulman, P. Moritz, S. Levine, M. Jordan, P. Abbeel,High-dimensional contin-uous control using generalized advantage estimation, in:Proc. 4th Int. Conf. Learn, 2016, pp. 1-14.

[36]	C.B. Mwakwata, H. Malik, M. Mahtab Alam, Y. Le Moullec, S. Parand, S. Mumtaz, Narrowband Internet of Things (NB-IoT): from physical (PHY) and media access control (MAC) layers perspectives, Sensors 19 (11) (2019) 1-2613.

[37]	System architecture for the 5G system,document TS 23.501 V16.1.0, 3GPP, 2019, pp. 1-219.

[38]	A. M., et al., WP5: Propagation, antennas and multiantenna techniques—D5. 1: Channel modeling and characterization, Millim.-Wave Evol. Backhaul Access (Mi-WEBA), 2014.