Research on active defense decision-making method for cloud boundary networks based on reinforcement learning of intelligent agent

Abstract

The cloud boundary network environment is characterized by a passive defense strategy, discrete defense actions, and delayed defense feedback in the face of network attacks, ignoring the influence of the external environment on defense decisions, thus resulting in poor defense effectiveness. Therefore, this paper proposes a cloud boundary network active defense model and decision method based on the reinforcement learning of intelligent agent, designs the network structure of the intelligent agent attack and defense game, and depicts the attack and defense game process of cloud boundary network; constructs the observation space and action space of reinforcement learning of intelligent agent in the non-complete information environment, and portrays the interaction process between intelligent agent and environment; establishes the reward mechanism based on the attack and defense gain, and encourage intelligent agents to learn more effective defense strategies. the designed active defense decision intelligent agent based on deep reinforcement learning can solve the problems of border dynamics, interaction lag, and control dispersion in the defense decision process of cloud boundary networks, and improve the autonomy and continuity of defense decisions.

Keywords

Active defense decision-making / Cloud boundary network security / Intelligent agent reinforcement learning / Offensive and defensive game

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (62106053), in part by the Guangxi Natural Science Foundation (2020GXNSFBA159042), in part by Innovation Project of Guangxi Graduate Education (YCSW2023478), in part by the Guangxi Education Department Program (2021KY0347), and part by the Doctoral Fund of Guangxi University of Science and Technology (XiaoKe Bo19Z33).

References

[1] Z. Fang, H. Fu, T. Gu, Z. Qian, T. Jaeger, P. Hu, P. Mohapatra, A model checking-based security analysis framework for IoT systems, High-Confid. Comput. 1 (2021) 100004.

[2] H. Zhang, J. Wang, D. Yu, J. Han, T. Li, Active defense strategy selection based on static Bayesian game, in:Third International Conference on Cyberspace Technology (CCT 2015), 2015, pp. 1-7.

[3] H. Zhang, L. Jiang, S. Huang, J. Wang, Y. Zhang, Attack-defense differential game model for network defense strategy selection, IEEE Access 7 (2019) 50618-50629.

[4] S. Huang, H. Zhang, J. Wang, J. Huang, Markov differential game for network defense decision-making method, IEEE Access 6 (2018) 39621-39634.

[5] L. Zhao, S. Hu, Q. Wang, J. Jiang, C. Shen, X. Luo, P. Hu, Shielding collaborative learning: Mitigating poisoning attacks through client-side detection, IEEE Trans. Dependable Secur. Comput. 18 (2021) 2029-2041.

[6] T.T. Nguyen, V.J. Reddi, Deep reinforcement learning for cyber security, IEEE Trans. Neural Netw. Learn. Syst. (2021) 1-17.

[7] K. Hammar, R. Stadler, Learning intrusion prevention policies through optimal stopping, in: 2021 17th International Conference on Network and Service Management, CNSM, IEEE, Izmir, Turkey, 2021, pp. 509-517.

[8] J. Parras, M. Huettenrauch, S. Zazo, G. Neumann, Deep reinforcement learning for attacking wireless sensor networks, Sensors 21 (2021) 4060.

[9] H.M. Rouzbahani, H. Karimipour, L. Lei, Multi-layer defense algorithm against deep reinforcement learning-based intruders in smart grids, Int. J. Electr. Power Energy Syst. 146 (2023) 108798.

[10] A.T.D. Perera, P. Kamalaruban, Applications of reinforcement learning in energy systems, Renew. Sustain. Energy Rev. 137 (2021) 110618.

[11] X. Xu, H. Hu, Y. Liu, J. Tan, H. Zhang, H. Song, Moving target defense of routing randomization with deep reinforcement learning against eavesdropping attack, Digit. Commun. Netw. 8 (2022) 373-387.

[12] X. Deng, J. Zhang, H. Zhang, P. Jiang, Deep-reinforcement-learning-based resource allocation for cloud gaming via edge computing, IEEE Internet Things J. 10 (2023) 5364-5377.

[13] L. Huang, M. Fu, H. Qu, S. Wang, S. Hu, A deep reinforcement learning-based method applied for solving multi-agent defense and attack problems, Expert Syst. Appl. 176 (2021) 114896.

[14] F. Zhong, P. Hu, G. Zhang, H. Li, X. Cheng, Reinforcement learning based adversarial malware example generation against black-box detectors, Comput. Secur. 121 (2022) 102869.

[15] A hidden attack sequences detection method based on dynamic reward deep deterministic policy gradient, (n.d.), 2023, (accessed March 19, 2023).

[16] Y. Badr, Enabling intrusion detection systems with dueling double deep Q-learning, Digit. Transform. Soc. 1 (2022) 115-141.

[17] A. Andrew, S. Spillard, J. Collyer, N. Dhir, Developing optimal causal cyber-defence agents via cyber security simulation, 2022.

[18] H. Alavizadeh, J. Jang-Jaccard, S.Y. Enoch, H. Al-Sahaf, I. Welch, S.A. Camtepe, D.D. Kim, A survey on cyber situation awareness systems: Framework, techniques, and insights, ACM Comput. Surv. (2022) 3530809.

[19] S. Yan, J. Ren, W. Wang, L. Sun, W. Zhang, Q. Yu, A survey of adversarial attack and defense methods for malware classification in cyber security, IEEE Commun. Surv. Tutor. 25 (2023) 467-496.

[20] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, O. Klimov, Proximal policy optimization algorithms, 2017, ArXiv.Org..

[21] Playing atari with deep reinforcement learning. - aminer, (n.d.), 2023, (accessed March 8, 2023).