Lightweight deep reinforcement learning for dynamic resource allocation in vehicular edge computing✩

Dapeng Wu; Sijun Wu; Yaping Cui; Ailing Zhong; Tong Tang; Ruyan Wang; Xinqi Lin

doi:10.1016/j.dcan.2025.06.005

›› 2025, Vol. 11 ›› Issue (5) :1530 -1542. DOI: 10.1016/j.dcan.2025.06.005

Regular Papers

research-article

Lightweight deep reinforcement learning for dynamic resource allocation in vehicular edge computing^✩

Dapeng Wu ^a^,^b^,^c
, Sijun Wu ^a^,^b^,^c
, Yaping Cui ^a^,^b^,^c
, Ailing Zhong ^a^,^b^,^c
, Tong Tang ^a^,^b^,^c
, Ruyan Wang ^a^,^b^,^c
, Xinqi Lin ^a^,^b^,^c^,^d^,^*

Author information +

History +

PDF

Abstract

Vehicular Edge Computing (VEC) enhances the quality of user services by deploying wealth of resources near vehicles. However, due to highly dynamic and complex nature of vehicular networks, centralized decision- making for resource allocation proves inadequate within VECs. Conversely, allocating resources via distributed decision-making consumes vehicular resources. To improve the quality of user service, we formulate a problem of latency minimization, further subdividing this problem into two subproblems to be solved through distributed decision-making. To mitigate the resource consumption caused by distributed decision-making, we propose Reinforcement Learning (RL) algorithm based on sequential alternating multi-agent system mechanism, which eﬀectively reduces the dimensionality of action space without losing the informational content of action, achieving network lightweighting. We discuss the rationality, generalizability, and inherent advantages of proposed mechanism. Simulation results indicate that our proposed mechanism outperforms traditional RL algorithms in terms of stability, generalizability, and adaptability to scenarios with invalid actions, all while achieving network lightweighting.

Keywords

Vehicular edge computing / Resource allocation / Reinforcement learning / Lightweight

Cite this article

Download citation ▾

Dapeng Wu, Sijun Wu, Yaping Cui, Ailing Zhong, Tong Tang, Ruyan Wang, Xinqi Lin. Lightweight deep reinforcement learning for dynamic resource allocation in vehicular edge computing^✩. , 2025, 11(5): 1530-1542 DOI:10.1016/j.dcan.2025.06.005

登录浏览全文

4963

注册一个新账户忘记密码

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	J. Peng, Y. Fan, G. Yin, R. Jiang, Collaborative optimization of energy management strategy and adaptive cruise control based on deep reinforcement learning, IEEE Trans. Transp. Electr. 9 (1) (2023) 34-44.

[2]	Y. Dai, D. Xu, K. Zhang, S. Maharjan, Y. Zhang, Deep reinforcement learning and permissioned blockchain for content caching in vehicular edge computing and net- works, IEEE Trans. Veh. Technol. 69 (4) (2020) 4312-4324.

[3]	B.L. Nguyen, D.T. Ngo, M.N. Dao, Q.-T. Duong, M. Okada, A joint scheduling and power control scheme for hybrid I2V/V2V networks, IEEE Trans. Veh. Technol. 69 (12) (2020) 15668-15681.

[4]	T.X. Tran, A. Hajisami, P. Pandey, D. Pompili, Collaborative mobile edge computing in 5G networks: new paradigms, scenarios, and challenges, IEEE Commun. Mag. 55 (4) (2017) 54-61.

[5]	Y. Wang, P. Lang, D. Tian, J. Zhou, X. Duan, Y. Cao, D. Zhao, A game-based compu-tation oﬄoading method in vehicular multiaccess edge computing networks, IEEE Internet Things J. 7 (6) (2020) 4987-4996.

[6]	H. Wang, X. Li, H. Ji, H. Zhang, Federated oﬄoading scheme to minimize latency in MEC-enabled vehicular networks, in: 2018 IEEE Globecom Workshops (GC Wkshps), 2018, pp. 1-6.

[7]	Y. Liu, Y. Dai, On the complexity of joint subcarrier and power allocation for multi-user OFDMA systems, IEEE Trans. Signal Process. 62 (3) (2014) 583-596.

[8]	V. Mnih, K. Kavukcuoglu, D. Silver, et al., Human-level control through deep rein- forcement learning, Nature 518 (2015) 529-533.

[9]	K. Jiang, Y. Cao, H. Zhou, X. Ren, Y. Zhu, H. Lin, Edge intelligence empowered Internet of vehicles: concept, framework, issues, implementation, and prospect, Chin. J. Internet of Things 7 (1) (2023) 37-48.

[10]	L. Canese, G.C. Cardarilli, L. Di Nunzio, R. Fazzolari, D. Giardino, M. Re, S. Spanò, Multi-agent reinforcement learning: a review of challenges and applications, Appl. Sci. 11 (2021) 4948.

[11]	G. Qiao, S. Leng, K. Zhang, Y. He, Collaborative task oﬄoading in vehicular edge multi-access networks, IEEE Commun. Mag. 56 (8) (2018) 48-54.

[12]	M.M.K. Tareq, O. Semiari, M.A. Salehi, W. Saad,Ultra reliable, low latency vehicle- to-infrastructure wireless communications with edge computing, in: 2018 IEEE Global Communications Conference (GLOBECOM), 2018, pp. 1-7.

[13]	C. Li, S. Wang, X. Huang, X. Li, R. Yu, F. Zhao, Parked vehicular computing for energy-eﬃcient Internet of vehicles: a contract theoretic approach, IEEE Internet Things J. 6 (4) (2019) 6079-6088.

[14]	Y. Cui, D. Zhang, T. Zhang, P. Yang, H. Zhu, A new approach on task oﬄoading scheduling for application of mobile edge computing, in: 2021 IEEE Wireless Com- munications and Networking Conference (WCNC), 2021, pp. 1-6.

[15]	L. Liang, H. Ye, G.Y. Li, Spectrum sharing in vehicular networks based on multi-agent reinforcement learning, IEEE J. Sel. Areas Commun. 37 (10) (2019) 2282-2292.

[16]	Y. Liu, H. Yu, S. Xie, Y. Zhang, Deep reinforcement learning for oﬄoading and re- source allocation in vehicle edge computing and networks, IEEE Trans. Veh. Technol. 68 (11) (2019) 11158-11168.

[17]	Y. He, N. Zhao, H. Yin, Integrated networking, caching, and computing for connected vehicles: a deep reinforcement learning approach, IEEE Trans. Veh. Technol. 67 (1) (2018) 44-55.

[18]	D. Wu, T. Liu, Z. Li, T. Tang, R. Wang, Delay-aware edge-terminal collaboration in green Internet of vehicles: a multiagent soft actor-critic approach, IEEE Trans. Green Commun. Netw. 7 (2) (2023) 1090-1102.

[19]	L. Geng, H. Zhao, J. Wang, A. Kaushik, S. Yuan, W. Feng, Deep-reinforcement- learning-based distributed computation oﬄoading in vehicular edge computing net- works, IEEE Internet Things J. 10 (14) (2023) 12416-12433.

[20]	J. Hong, J. Yin, Y. Liu, J. Peng, H. Jiang, Energy management and control strat- egy of photovoltaic/battery hybrid distributed power generation systems with an integrated three-port power converter, IEEE Access 7 (2019) 82838-82847.

[21]	A. Rodriguez-Ramos, C. Sampedro, H. Bavle, P.d.l. Puente, P. Campoy, A deep rein- forcement learning strategy for UAV autonomous landing on a moving platform, J. Intell. Robot. Syst. 93 (2019) 351-366.

[22]	S. Bi, L. Huang, H. Wang, Y.-J.A. Zhang, Lyapunov-guided deep reinforcement learn- ing for stable online computation oﬄoading in mobile-edge computing networks, IEEE Trans. Wirel. Commun. 20 (11) (2021) 7519-7537.

[23]	S. Fujimoto, H.V. Hoof, D. Meger, Addressing function approximation error in actor- critic methods, https://arxiv.org/abs/1802.09477, 2018. (Accessed 13 May 2023).

[24]	A. Alwarafy, M. Abdallah, B. Ciftler, A. Al-Fuqaha, M. Hamdi, Deep reinforcement learning for radio resource allocation and management in next generation hetero- geneous wireless networks: a survey, https://www.techrxiv.org/doi/full/10.36227/ techrxiv.14672643.v1, 2021. (Accessed 13 May 2023).

[25]	T. Zahavy, M. Haroush, N. Merlis, D.J. Mankowitz, S. Mannor, Learn what not to learn: action elimination with deep reinforcement learning, https://doi.org/10.48550/arxiv.1809.02121, 2019. (Accessed 13 May 2023).

[26]	S. Huang, S. Ontaón, A closer look at invalid action masking in policy gradient al- gorithms, https://arxiv.org/abs/2006.14171, 2022. (Accessed 13 May 2023).

[27]	J. Foerster, N. Nardelli, G. Farquhar, T. Afouras, P.H.S. Torr, P. Kohli, S. Whiteson, Stabilising experience replay for deep multi-agent reinforcement learning, https://arxiv.org/abs/1702.08887, 2018. (Accessed 13 May 2023).

[28]	M. Parvini, M.R. Javan, N. Mokari, B. Abbasi, E.A. Jorswieck, Aoi-aware resource al- location for platoon-based C-V2X networks via multi-agent multi-task reinforcement learning, IEEE Trans. Veh. Technol. 72 (8) (2023) 9880-9896.

[29]	S. Ahmadi,3GPP LTE/LTE-advanced radio access technologies, in: J.D. Gibson (Ed.), Mobile Communications Handbook, CRC Press, Boca Raton, 2017, pp. 451-480.