Deep reinforcement learning-based forwarding node selection algorithm in Internet of vehicles^☆

Abstract

Due to open communication environment, Internet of Vehicles (IoV) are vulnerable to many attacks, including the gray hole attack, which can disrupt the process of transmitting messages. And this results in the degradation of routing performance. To address this issue, a double deep Q-networks-based stable routing for resisting gray hole attack (DOSR) is proposed in this paper. The aim of the DOSR algorithm is to maximize the message delivery ratio as well as to minimize the transmission delay. For this, the distance ratio, message loss ratio, and connection ratio are taken into consideration when choosing a relay node. Then, to choose the relay node is formulated as an optimization problem, and a double deep Q-networks are utilized to solve the optimization problem. Experimental results show that DOSR outperforms QLTR and TLRP by significant margins: in scenarios with 400 vehicles and 10% malicious nodes, the message delivery ratio (MDR) of DOSR is 8.3% higher than that of QLTR and 5.1% higher than that of TLRP; the average transmission delay (ATD) is reduced by 23.3% compared to QLTR and 17.9% compared to TLRP. Additionally, sensitivity analysis of hyperparameters confirms the convergence and stability of DOSR, demonstrating its robustness in dynamic IoV environments.

Keywords

Internet of vehicles / Stable routing / Deep reinforcement learning / Forwarding candidate set

References

[1] Y. He, F. Huang, D. Wang, B. Chen, T. Li, R. Zhang, Performance analysis and op-timization design of aav-assisted vehicle platooning in noma-enhanced internet of vehicles, IEEE Trans. Intell. Transp. Syst. 26 (6) (2025) 8810-8819.

[2] D. Annunziata, D. Chiaro, P. Qi, F. Piccialli, On the road to aiot: a framework for vehicle road cooperation, IEEE Internet Things J. 12 (5) (2025) 5783-5791.

[3] G. He, T. He, R. Chen, B. Xu, H. Zhu, L. Zhang, N. Guo, Multidevice collabora-tive authentication for internet of things, IEEE Internet Things J. 12 (14) (2025) 27753-27768.

[4] S.P. Chaturvedi, R. Mukherjee, S. Kumar, A. Yadav, Revolutionizing xtea: unveil-ing prextea and trixtea-enhanced efficiency and security in internet of things, IEEE Internet Things J. 12 (4) (2025) 3971-3979.

[5] B. Liu, Y. Sheng, X. Shao, Y. Ji, W. Han, E. Wang, S. Xiong, Collaborative intelligence enabled routing in green IoV: a grid and vehicle density prediction-based protocol, IEEE Trans. Green Commun. Netw. 7 (2) (2023) 1012-1022.

[6] K. Mershad, Surfer: a secure sdn-based routing protocol for internet of vehicles, IEEE Internet Things J. 8 (9) (2021) 7407-7422.

[7] R. Han, Q. Guan, F.R. Yu, J. Shi, F. Ji, Congestion and position aware dynamic routing for the Internet of vehicles, IEEE Trans. Veh. Technol. 69 (12) (2020) 16082-16094.

[8] C. Chen, L. Liu, T. Qiu, J. Jiang, Q. Pei, H. Song, Routing with traffic awareness and link preference in internet of vehicles, IEEE Trans. Intell. Transp. Syst. 23 (1) (2022) 200-214.

[9] H. Xia, S.-s. Zhang, Y. Li, Z.-k. Pan, X. Peng, X.-z. Cheng, An attack-resistant trust inference model for securing routing in vehicular ad hoc networks, IEEE Trans. Veh. Technol. 68 (7) (2019) 7108-7120.

[10] H. Xia, Z. Li, Y. Zheng, A. Liu, Y.-J. Choi, H. Sekiya, A novel light-weight subjective trust inference framework in MANETs, IEEE Trans. Sustain. Comput. 5 (2) (2020) 236-248.

[11] G. Oluchi Anyanwu, C.I. Nwakanma, J.-M. Lee, D.-S. Kim, Optimization of rbf-svm kernel using grid search algorithm for ddos attack detection in sdn-based vanet, IEEE Internet Things J. 10 (10) (2023) 8477-8490.

[12] T.T. Nguyen, V.J. Reddi, Deep reinforcement learning for cyber security, IEEE Trans. Neural Netw. Learn. Syst. 34 (8) (2023) 3779-3795.

[13] B.D. Evans, H.A. Engelbrecht, H.W. Jordaan, High-speed autonomous racing using trajectory-aided deep reinforcement learning, IEEE Robot. Autom. Lett. 8 (9) (2023) 5353-5359.

[14] R. Huang, M. Guo, C. Gu, S. He, J. Chen, M. Sun, Toward scalable and efficient hierar-chical deep reinforcement learning for 5G RAN slicing, IEEE Trans. Green Commun. Netw. 7 (4) (2023) 2153-2162.

[15] T.-N. Tran, T.-V. Nguyen, K. Shim, D.B. da Costa, B. An, A deep reinforcement learning-based qos routing protocol exploiting cross-layer design in cognitive radio mobile ad hoc networks, IEEE Trans. Veh. Technol. 71 (12) (2022) 13165-13181.

[16] A. Ullah, X. Yao, S. Shaheen, H. Ning, Advances in position based routing towards ITS enabled fog-oriented VANET-a survey, IEEE Trans. Intell. Transp. Syst. 21 (2)(2020) 828-840.

[17] L. Luo, L. Sheng, H. Yu, G. Sun, Intersection-based V2X routing via reinforcement learning in vehicular ad hoc networks, IEEE Trans. Intell. Transp. Syst. 23 (6) (2022) 5446-5459.

[18] S. Gangopadhyay, V.K. Jain, A position-based modified OLSR routing protocol for flying ad hoc networks, IEEE Trans. Veh. Technol. 72 (9) (2023) 12087-12098.

[19] L.-L. Wang, J.-S. Gui, X.-H. Deng, F. Zeng, Z.-F. Kuang, Routing algorithm based on vehicle position analysis for internet of vehicles, IEEE Internet Things J. 7 (12)(2020) 11701-11712.

[20] H. Gao, C. Liu, Y. Li, X. Yang, V2VR: reliable hybrid-network-oriented V2V data transmission and routing considering RSUs and connectivity probability, IEEE Trans. Intell. Transp. Syst. 22 (6) (2021) 3533-3546.

[21] N. Li, J.-F. Martínez-Ortega, V.H. Díaz, J.A.S. Fernandez, Probability prediction-based reliable and efficient opportunistic routing algorithm for VANETs, IEEE/ACM Trans. Netw. 26 (4) (2018) 1933-1947.

[22] X. Zhang, X. Cao, L. Yan, D.K. Sung, A street-centric opportunistic routing protocol based on link correlation for urban VANETs, IEEE Trans. Mob. Comput. 15 (7) (2016) 1586-1599.

[23] M. Rios, GeOpps-N: opportunistic routing for VANET in a public transit system, IEEE Latin Amer. Trans. 14 (4) (2016) 1630-1637.

[24] J. Ryu, S. Kim, Reputation-based opportunistic routing protocol using Q-learning for MANET attacked by malicious nodes, IEEE Access 11 (2023) 47701-47711.

[25] H. Fatemidokht, M.K. Rafsanjani, B.B. Gupta, C.-H. Hsu, Efficient and secure routing protocol based on artificial intelligence algorithms with UAV-assisted for vehicular ad hoc networks in intelligent transportation systems, IEEE Trans. Intell. Transp. Syst. 22 (7) (2021) 4757-4769.

[26] F.H. Kumbhar, S.Y. Shin, DT-VAR: decision tree predicted compatibility-based ve-hicular ad-hoc reliable routing, IEEE Wirel. Commun. Lett. 10 (1) (2021) 87-91.

[27] Z. Hassan, A. Mehmood, C. Maple, M.A. Khan, A. Aldegheishem, Intelligent detec-tion of black hole attacks for secure communication in autonomous and connected vehicles, IEEE Access 8 (2020) 199618-199628.

[28] B. Su, L. Tong, Transmission protocol of emergency messages in vanet based on the trust level of nodes, IEEE Access 11 (2023) 68243-68256.

[29] J. Lyu, C. Chen, H. Tian,Secure routing based on geographic location for resist-ing blackhole attack in three-dimensional VANETs, in:2020 IEEE/CIC International Conference on Communications in China (ICCC), 2020, pp. 1168-1173.

[30] V. S, P. R, R. R, V. K, Securing the vehicular communication against malicious node using various types of security/privacy algorithms: a survey,in:2022 6th Interna-tional Conference on Electronics, Communication and Aerospace Technology, 2022, pp. 633-637.

[31] A. Rezapour, W.-G. Tzeng, RL-Shield: mitigating target link-flooding attacks using sdn and deep reinforcement learning routing algorithm, IEEE Trans. Dependable Secure Comput. 19 (6) (2022) 4052-4067.

[32] E.M. Mianji, G.-M. Muntean, I. Tal,Trustworthy routing in VANET: a Q-learning approach to protect against black hole and gray hole attacks, in: 2023 IEEE 97th Vehicular Technology Conference (VTC2023-Spring), 2023, pp. 1-6.

[33] H. Wang, J. Han, K. Xue, J. Yang, J. Li, Q. Sun, J. Lu, Caaf: an ndn-based cache-aware adaptive forwarding strategy for reliable content delivery in vanets, IEEE Trans. Mob. Comput. 24 (7) (2025) 6361-6375.

[34] G. Du, Y. Zou, X. Zhang, J. Fan, Improved deep reinforcement learning for efficient motion control of autonomous vehicle with domain-centralized electronic and elec-trical architecture, IEEE Internet Things J. 12 (11) (2025) 15620-15635.

[35] Z. Sifeng, T. Xiaohua, Z. Zonghui, Q. Rui, Z. Hai, Content placement and edge col-laborative caching scheme based on deep reinforcement learning for internet of vehicles, IEEE Trans. Intell. Transp. Syst. 26 (6) (2025) 8050-8064.

[36] K. Zhang, Y. Li, Z. Zhang, F. Ye, Ddqn-based hybrid routing protocol for uwsns with void repair mechanism, IEEE Sens. J. 25 (11) (2025) 20718-20731.

[37] D.M. Casas-Velasco, O.M.C. Rendon, N.L.S. da Fonseca, DRSIR: a deep reinforcement learning approach for routing in software-defined networking, IEEE Trans. Netw. Serv. Manag. 19 (4) (2022) 4807-4820.