Communication-efficient federated multi-domain learning for network anomaly detection^✩

Abstract

Efficient and accurate anomaly detection in a network is of great significance for maintaining network and device security. Most anomaly detection methods assume that different anomalous network data distributions are the same or similar and ignore data privacy preservation. In this paper, a novel Federated Learning (FL) is proposed that it can quickly detect different types of anomalies in Non-Independent and Identically Distributed (Non-IID) data. First, we design a multi-domain machine learning model for multi-domain data, named Aegean, which consists of two modules: an ensemble AutoEncoder (AE) and a Generative Adversarial Network (GAN). Second, because data from different domains are non-IID, we model the anomaly detection problem as a dual problem, which can be recast as a robust optimization problem. The robust optimization problem is non-convex and therefore difficult to solve. As a remedy, we formulate and solve a dual problem by taking the Lagrangian dual function of the original problem. Experiments demonstrate that Aegean significantly outperforms the current state-of-the-art methods, with a 16% F1 score improvement over that of a One-Class Support Vector Machine (OCSVM). The designed FL significantly reduces the communication overhead of FedAvg without sacrificing anomaly detection performance.

Keywords

Federated learning / Multi-domain / Network anomaly detection / Communication

CRediT authorship contribution statement

Chunjiong Zhang: Writing – original draft, Software, Resources, Methodology, Data curation, Conceptualization. Gaoyang Shan: Validation, Software, Formal analysis, Data curation. Byeong-hee Roh: Writing – review & editing, Validation, Supervision, Project administration, Investigation, Funding acquisition.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

This work is supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2023-2018-0-01431) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation); the Brain Korea 21 (BK21) FOUR program of the National Research Foundation of Korea funded by the Ministry of Education (NRF5199991514504).

References

[1] A. Gerodimos, L. Maglaras, M.A. Ferrag, N. Ayres, I. Kantzavelou, Iot: communication protocols and security threats, Int. Things Cyber—Phys. Syst. 3 (2023) 1-13.

[2] Y. Li, Y. Liu, H. Yin, Z. Guo, Y. Wang, Trident: defensing synergetic denial—of—service attacks in underwater named data networking, IEEE Int. Things J. 10 (23) (2023) 20633-20648.

[3] A. Diro, S. Kaisar, A.V. Vasilakos, A. Anwar, A. Nasirian, G. Olani, Anomaly detection for space information networks: a survey of challenges, techniques, and future directions, Comput. Secur. 139 (2024) 103705.

[4] Z. Liu, Y. Zhou, Y. Xu, Z. Wang, Simplenet: a simple network for image anomaly detection and localization, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 20402-20411.

[5] X. Wang, Y. Wang, Z. Javaheri, L. Almutairi, N. Moghadamnejad, O.S. Younes, Federated deep learning for anomaly detection in the internet of things, Comput. Electr. Eng. 108 (2023) 108651.

[6] F. Meng, T. Ren, E. Guo, H. Chen, X. Liu, H. Zhang, J. Li, Estimation of ground motion parameters via multi—task deep neural networks, Nat. Hazards 120 (7) (2024) 6737-6754.

[7] Z. Xiong, W. Li, Z. Cai, Federated generative model on multi—source heterogeneous data in iot, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, 2023, pp. 10537-10545.

[8] X. Wu, Z. Hu, J. Pei, H. Huang, Serverless federated auprc optimization for multi—party collaborative imbalanced data mining, in: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023, pp. 2648-2659.

[9] Y. Jiang, M. Li, W. Wu, X. Wu, X. Zhang, X. Huang, R.Y. Zhong, G.G. Huang, Multi—domain ubiquitous digital twin model for information management of complex infrastructure systems, Adv. Eng. Inform. 56 (2023) 101951.

[10] M. Hassan, M.A. Gregory, S. Li, Multi—domain federation utilizing software defined networking—a review, IEEE Access 11 (2023) 19202-19227.

[11] A. Azab, M. Khasawneh, S. Alrabaee, K.—K.R. Choo, M. Sarsour, Network traffic classification: techniques, datasets, and challenges, Dig. Comm. Netw. 10 (3) (2024) 676-692.

[12] L. Yin, S. Lin, Z. Sun, R. Li, Y. He, Z. Hao, A game—theoretic approach for federated learning: a trade—off among privacy, accuracy and energy, Dig. Comm. Netw. 10 (2) (2024) 389-403.

[13] X.—X. Wei, H. Huang, Edge devices clustering for federated visual classification: a feature norm based framework, IEEE Trans. Image Process. 32 (2023) 995-1010.

[14] F.D. Vaca, Q. Niyaz, An ensemble learning based wi—fi network intrusion detection system (wnids), in: 2018 IEEE 17th International Symposium on Network Computing and Applications (NCA), IEEE, 2018, pp. 1-5.

[15] Y. Mirsky, T. Doitshman, Y. Elovici, A. Shabtai, Kitsune: an ensemble of autoencoders for online network intrusion detection, in: Network and Distributed System Security Symposium, 2018.

[16] W. Liao, Y. Guo, X. Chen, P. Li, A unified unsupervised Gaussian mixture variational autoencoder for high dimensional outlier detection, in: 2018 IEEE International Conference on Big Data, 2018.

[17] B. Zong, Q. Song, M.R. Min, W. Cheng, C. Lumezanu, D. Cho, H. Chen, Deep autoencoding Gaussian mixture model for unsupervised anomaly detection, in: International Conference on Learning Representations, 2018.

[18] G. Andresini, A. Appice, N. Di Mauro, C. Loglisci, D. Malerba, Exploiting the autoencoder residual error for intrusion detection, in: 2019 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW), IEEE, 2019, pp. 281-290.

[19] S. Rezvy, M. Petridis, A. Lasebae, T. Zebin, Intrusion detection and classification with autoencoded deep neural network, in: International Conference on Security for Information Technology and Communications, Springer, 2018, pp. 142-156.

[20] H. Tekchandani, S. Verma, N. Londhe, Performance improvement of mediastinal lymph node severity detection using gan and inception network, Comput. Methods Programs Biomed. 194 (2020) 105478.

[21] H. Zenati, M. Romain, C.—S. Foo, B. Lecouat, V. Chandrasekhar, Adversarially learned anomaly detection, in: 2018 IEEE International Conference on Data Mining, IEEE, 2018, pp. 727-736.

[22] B. Huang, W. Chen, X. Wu, C.—L. Lin, P.N. Suganthan, High—quality face image generated with conditional boundary equilibrium generative adversarial networks, Pattern Recognit. Lett. 111 (2018) 72-79.

[23] A.B.L. Larsen, S.K. Sønderby, H. Larochelle, O. Winther, Autoencoding beyond pixels using a learned similarity metric, in: International Conference on Machine Learning, 2016, pp. 1558-1566.

[24] G. Pang, C. Shen, A. van den Hengel, Deep anomaly detection with deviation networks, in: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 353-362.

[25] S. Ben—David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, J.W. Vaughan, A theory of learning from different domains, Mach. Learn. 79 (1—2) (2010) 151-175.

[26] D. Fourure, R. Emonet, E. Fromont, D. Muselet, N. Neverova, A. Tremeau, C. Wolf, Multi—task, multi—domain learning: application to semantic segmentation and pose regression, Neurocomputing 251 (AUG.16) (2017) 68-80.

[27] Y. Li, N. Vasconcelos, Efficient multi—domain learning by covariance normalization, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 5424-5433.

[28] A.M. Abdelmoniem, C.—Y. Ho, P. Papageorgiou, M. Canini, A comprehensive empirical study of heterogeneity in federated learning, IEEE Int. Things J. 10 (16) (2023) 14071-14083.

[29] T. Li, A.K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, V. Smith, Federated optimization in heterogeneous networks, in: Conference on Machine Learning and Systems, 2020.

[30] T.D. Nguyen, S. Marchal, M. Miettinen, H. Fereidooni, N. Asokan, A. Sadeghi, DÏot: a federated self—learning anomaly detection system for iot, in: 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), 2019, pp. 756-767.

[31] M.H. Mahmoud, A. Albaseer, M. Abdallah, N. Al—Dhahir, Federated learning resource optimization and client selection for total energy minimization under outage, latency, and bandwidth constraints with partial or no csi, IEEE Open J. Commun. Soc. 4 (2023) 936-953.

[32] X. Lian, C. Zhang, H. Zhang, C.—J. Hsieh, W. Zhang, J. Liu, Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent, in: Advances in Neural Information Processing Systems, 2017, pp. 5330-5340.

[33] S. Sun, W. Chen, J. Bian, X. Liu, T.—Y. Liu, Ensemble—compression: a new method for parallel training of deep neural networks, in: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer, 2017, pp. 187-202.

[34] A. Koloskova, T. Lin, S.U. Stich, M. Jaggi, Decentralized deep learning with arbitrary communication compression, in: International Conference on Learning Representations, 2019.

[35] D. Novoa—Paradela, O. Fontenla—Romero, B. Guijarro—Berdiñas, Fast deep autoencoder for federated learning, Pattern Recognit. 143 (2023) 109805.

[36] R. Xiong, W. Ren, S. Zhao, J. He, Y. Ren, K.—K.R. Choo, G. Min, Copifl: a collusion—resistant and privacy—preserving federated learning crowdsourcing scheme using blockchain and homomorphic encryption, Future Gener. Comput. Syst. 156 (2024) 95-104.

[37] J. Wang, G. Xie, Y. Huang, J. Lyu, F. Zheng, Y. Zheng, Y. Jin, Fedmed—gan: federated domain translation on unsupervised cross—modality brain image synthesis, Neurocomputing 546 (2023) 126282.

[38] B. McMahan, E. Moore, D. Ramage, S. Hampson, B.A. y Arcas, Communication—efficient learning of deep networks from decentralized data, in: Artificial Intelligence and Statistics, 2017, pp. 1273-1282.

[39] J. Zhang, B. Li, C. Chen, L. Lyu, S. Wu, S. Ding, C. Wu, Delving into the adversarial robustness of federated learning, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, 2023, pp. 11245-11253.

[40] X. Li, Z. Song, J. Yang, Federated adversarial learning: a framework with convergence analysis, in: International Conference on Machine Learning, PMLR, 2023, pp. 19932-19959.

[41] F. Sattler, K.—R. Müller, T. Wiegand, W. Samek, On the byzantine robustness of clustered federated learning, in: 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, 2020, pp. 8861-8865.

[42] T. Overman, G. Blum, D. Klabjan, A primal—dual algorithm for hybrid federated learning, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, 2024, pp. 14482-14489.

[43] X. Peng, Z. Huang, Y. Zhu, K. Saenko, Federated adversarial domain adaptation, in: International Conference on Learning Representations, 2019.

[44] R. Du, S. Xu, R. Zhang, L. Xu, H. Xia, A dynamic adaptive iterative clustered federated learning scheme, Knowl.—Based Syst. 276 (2023) 110741.

[45] M. Mehta, C. Shao, A greedy agglomerative framework for clustered federated learning, IEEE Trans. Ind. Inform. 19 (12) (2023) 11856-11867.

[46] X. Zhou, X. Ye, I. Kevin, K. Wang, W. Liang, N.K.C. Nair, S. Shimizu, Z. Yan, Q. Jin, Hierarchical federated learning with social context clustering—based participant selection for internet of medical things applications, IEEE Trans. Comput. Soc. Syst. 10 (4) (2023) 1742-1751.

[47] C. Zhang, B.—h. Roh, G. Shan, Poster: dynamic clustered federated framework for multi—domain network anomaly detection, in: Companion of the 19th International Conference on Emerging Networking EXperiments and Technologies, CoNEXT 2023, Association for Computing Machinery, New York, NY, USA, 2023, pp. 71-72.

[48] O.R.A. Almanifesti, C.—O. Chow, M.—L. Tham, J.H. Chuah, J. Kanesan, Communication and computation efficiency in federated learning: a survey, Int. Things 22 (2023) 100742.

[49] L. Ruff, R.A. Vandermeulen, N. Görnitz, A. Binder, E. Müller, K.—R. Müller, M. Kloft, Deep semi—supervised anomaly detection, in: International Conference on Learning Representations, 2020, https://openreview.net/forum?id=HkgH0TEYwH.

[50] J. Fan, G. Tang, K. Wu, Z. Zhao, Y. Zhou, S. Huang, Score—vae: root cause analysis for federated—learning—based iot anomaly detection, IEEE Int. Things J. 11 (1) (2024) 1041-1053.

[51] H. Xu, S. Han, X. Li, Z. Han, Anomaly traffic detection based on communication—efficient federated learning in space—air—ground integration network, IEEE Trans. Wirel. Commun. 22 (12) (2023) 9346-9360.

[52] Z. Chen, P. Tian, W. Liao, W. Yu, Zero knowledge clustering based adversarial mitigation in heterogeneous federated learning, IEEE Trans. Netw. Sci. Eng. 8 (2) (2020) 1070-1083.

[53] Y. Salem, M.M. Hamarsheh, Forensically analyzing iot smart camera using maoidff—iot framework, Forensic Sci. Int. Digit. Investig. 51 (2024) 301829.