Building robust traffic classifier under low quality data: A federated contrastive learning approach^✩

Abstract

In the big data era, the surge in network traffic volume poses challenges for network management and cyberse- curity. Network Traffic Classification (NTC) employs deep learning to categorize traffic data, aiding security and analysis systems as Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS). However, current NTC methods, based on isolated network simulations, usually fail to adapt to new protocols and applications and ignore the effects of network conditions and user behavior on traffic patterns. To improve network traffic management insights, federated learning frameworks have been proposed to aggregate diverse traffic data for collaborative model training. This approach faces challenges like data integrity, label noise, packet loss, and skewed data distributions. While label noise can be mitigated through the use of sophisticated traffic labeling tools, other issues such as packet loss and skewed data distributions encountered in Network Packet Brokers (NPB) can severely impede the efficacy of federated learning algorithms. In this paper, we introduced the Robust Traffic Classifier with Federated Contrastive Learning (FC-RTC), combining federated and contrastive learning methods. Using the Supcon-Loss function from contrastive learning, FC-RTC distinguishes between similar and dissimilar samples. Training by sample pairs, FC-RTC effectively updates when receiving corrupted traffic data with packet loss or disorder. In cases of sample imbalance, contrastive loss functions for similar samples reduce model bias towards higher proportion data. By addressing uneven data distribution and packet loss, our system enhances its capability to adapt and perform accurately in real-world network traffic analysis, meeting the spe- cific demands of this complex field.

Keywords

Federated learning / Network traffic classification / Contrastive learning / Robust machine learning / Packet loss

References

[1] A. Pescape, Claffy C. K. Dainotti, Issues and future directions in traffic classification, IEEE Netw. Mag. Comput. Commun. 26 (1) (2012) 35-40.

[2] I.T. Union, Measuring digital development: facts and figures 2023, https://www.itu.int/en/ITU-D/Statistics/Pages/facts/default.aspx, 2023. (Accessed 10 November 2023).

[3] A. Amine, O.A. Mohamed, B. Benatallah, Network Security Technologies: Design and Applications, Information Science Reference, 2014.

[4] J. Cai, Y. Li, B. Liu, Z. Wu, S. Zhu, Q. Chen, Q. Lei, H. Hou, Z. Guo, H. Jiang, S. Guo, F. Wang, S. Huang, S. Zhu, X. Fan, S. Tao, Developing deep lstms with later temporal attention for predicting covid-19 severity, clinical outcome, and antibody level by screening serological indicators over time, IEEE J. Biomed. Health Inform. 28 (7) (2024) 4204-4215.

[5] J. Cai, R. Zhong, Y. Li, Antenna selection for multiple-input multiple-output systems based on deep convolutional neural networks, PLoS ONE 14 (5) (2019) 1-16.

[6] J. Cai, J. Hu, X. Tang, T.-Y. Hung, Y.-P. Tan, Deep historical long short-term memory network for action recognition, Neurocomputing 407 (2020) 428-438.

[7] J. Liu, Z. Chen, J. Zhou, A. Xue, D. Peng, Y. Gu, H. Chen, Research on a ship trajectory classification method based on deep learning, Chin. J. Inf. Fusion 1 (1) (2024) 3-15.

[8] M. Yang, X. Fan, Yolov8-lite: a lightweight object detection model for real-time au- tonomous driving systems, IECE Trans. Emerg. Top. Artif. Intell. 1 (1) (2024) 1-16.

[9] X. Jin, J. Li, B. Wang, H. Ma, L. Gao, H. Fu, Visual feature extraction and tracking method based on corner flow detection, IECE Trans. Intell. Syst. 1 (1) (2024) 3-9.

[10] N. Malekghaini, E. Akbari, M.A. Salahuddin, N. Limam, R. Boutaba, B. Mathieu, S. Moteau, S. Tuffin, Data drift in dl: lessons learned from encrypted traffic classifica- tion,in: 2022 IFIP Networking Conference (IFIP Networking), 2022, pp. 1-9.

[11] B. McMahan, E. Moore, D. Ramage, S. Hampson, B.A.y. Arcas,Communication- efficient learning of deep networks from decentralized data, in: A. Singh, J.Zhu (Eds.), Proceedings of the 20th International Conference on Artificial Intelligence and Statistics,in:Proceedings of Machine Learning Research, vol. 54, PMLR, 2017, pp. 1273-1282.

[12] R. Xie, J. Cao, E. Dong, M. Xu, K. Sun, Q. Li, L. Shen, M. Zhang, Rosetta: enabling robust TLS encrypted traffic classification in diverse network environments with TCP-aware traffic augmentation,in:32nd USENIX Security Symposium (USENIX Se- curity 23), USENIX Association, Anaheim, CA, 2023, pp. 625-642.

[13] R. Sundara Raman, L.-H. Merino, K. Bock, M. Fayed, D. Levin, N. Sullivan, L. Va- lenta, Global, passive detection of connection tampering, in: Proceedings of the ACM SIGCOMM 2023 Conference, ACM SIGCOMM ’23, Association for Computing Ma- chinery, New York, NY, USA, 2023, pp. 622-636.

[14] S.C. Smith, R.J. Hammell, K.W. Wong, C.J. Mateo, An experimental exploration of the impact of multi-level packet loss on network intrusion detection, in: Software Engineering Research and Applications, 2016.

[15] P.H. Le-Khac, G. Healy, A.F. Smeaton, Contrastive representation learning: a frame- work and review, IEEE Access 8 (2020) 193907-193934.

[16] Z. Chen, G. Cheng, Z. Xu, K. Xu, Y. Shan, J. Zhang, A3c system: one-stop automated encrypted traffic labeled sample collection, construction and correlation in multi- systems, Appl. Sci. 12 (22) (2022).

[17] T. Karagiannis, A. Broido, M. Faloutsos, K. Claffy, Transport layer identification of p2p traffic, in: ACM SIGCOMM Conference on Internet Measurement, 2004.

[18] Y. Wang, Y. Xiang, S.Z. Yu, Automatic application signature construction from un- known traffic, in: IEEE International Conference on Advanced Information Network- ing & Applications, 2010.

[19] A. Madhukar, C.L. Williamson, A longitudinal study of p2p traffic classification, in: 14th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS 2006), 11-14 September 2006, Mon-terey, California, USA, 2006, pp. 11-14.

[20] Z. Cao, G. Xiong, Y. Zhao, Z. Li, L. Guo, A survey on encrypted traffic classification, in: L. Batten, G.Li, W.Niu, M.Warren (Applications and Techniques in Infor- mation Security,Eds.), Springer Berlin Heidelberg, Berlin, Heidelberg, 2014, pp. 73-81.

[21] T. Bujlow, T. Riaz, J.M. Pedersen, A method for classification of network traffic based on c5.0 machine learning algorithm, in: 2012 International Conference on Computing, Networking and Communications (ICNC), 2012.

[22] S. Huang, K. Chen, C. Liu, A. Liang, H. Guan,A statistical-feature-based approach to Internet traffic classification using machine learning, in:Proceedings of the In- ternational Conference on Ultra Modern Telecommunications, ICUMT 2009, 12-14 October 2009, St. Petersburg, Russia, 2009.

[23] A. Azab, M. Khasawneh, S. Alrabaee, K.-K.R. Choo, M. Sarsour, Network traffic clas- sification: techniques, datasets and challenges, Digit. Commun. Netw. 10 (3) (2024) 676-692.

[24] Z. Chen, G. Cheng, Z. Xu, S. Guo, Y. Zhou, Y. Zhao, Length matters: scalable fast encrypted Internet traffic service classification based on multiple protocol data unit length sequence with composite deep learning, Digit. Commun. Netw. 8 (3) (2022) 289-302.

[25] W. Wang, M. Zhu, X. Zeng, X. Ye, Y. Sheng,Malware traffic classification using convolutional neural network for representation learning, in:2017 International Conference on Information Networking (ICOIN), 2017, pp. 712-717.

[26] Z. Chen, G. Cheng, Z. Wei, Z. Xu, N. Fu, Y. Zhou, Higher layers, better results: application layer feature engineering in encrypted traffic classification,in:Inter- national Conference on Wireless Algorithms, Systems, and Applications, Springer, 2022, pp. 548-556.

[27] J. Dai, X. Xu, H. Gao, X. Wang, F. Xiao, Shape: a simultaneous header and payload encoding model for encrypted traffic classification, IEEE Trans. Netw. Serv. Manag. 20 (2) (2023) 1993-2012.

[28] H. Zhang, L. Yu, X. Xiao, Q. Li, F. Mercaldo, X. Luo, Q. Liu, Tfe-gnn: a temporal fusion encoder using graph neural networks for fine-grained encrypted traffic classi- fication,in:Proceedings of the ACM Web Conference 2023, WWW ’23, Association for Computing Machinery, New York, NY, USA, 2023, pp. 2066-2075.

[29] M. Abbasi, A. Shahraki, J. Prieto, A.G. Arrieta, J.M. Corchado, Unleashing the poten- tial of knowledge distillation for iot traffic classification, IEEE Trans. Mach. Learn. Commun. Netw. 2 (2024) 221-239.

[30] M. Shen, K. Ye, X. Liu, L. Zhu, J. Kang, S. Yu, Q. Li, K. Xu, Machine learning-powered encrypted network traffic analysis: a comprehensive survey, IEEE Commun. Surv. Tutor. 25 (1) (2023) 791-824.

[31] M. Jain, G. Kaur, V. Saxena, A k-means clustering and svm based hybrid concept drift detection technique for network anomaly detection, Expert Syst. Appl. 193 (2022) 116510.

[32] D.C. Nguyen, M. Ding, P.N. Pathirana, A. Seneviratne, J. Li, H. Vincent Poor, Feder- ated learning for Internet of things: a comprehensive survey, IEEE Commun. Surv. Tutor. 23 (3) (2021) 1622-1658.

[33] A. van den Oord, Y. Li, O. Vinyals, Representation learning with contrastive predic- tive coding, arXiv:1807.03748, 2019.

[34] T. Chen, S. Kornblith, M. Norouzi, G. Hinton,A simple framework for contrastive learning of visual representations, in: H. D.III, A.Singh (Eds.), Proceedings of the 37th International Conference on Machine Learning,in:Proceedings of Machine Learning Research, vol. 119, PMLR, 2020, pp. 1597-1607.

[35] K. He, H. Fan, Y. Wu, S. Xie, R. Girshick,Momentum contrast for unsupervised visual representation learning, in:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

[36] Y. Wu, D. Zeng, Z. Wang, Y. Shi, J. Hu, Distributed contrastive learning for medical image segmentation, Med. Image Anal. 81 (2022) 102564.

[37] Y. Zhang, Y. Xu, S. Wei, Y. Wang, Y. Li, X. Shang, Doubly contrastive representation learning for federated image recognition, Pattern Recognit. 139 (2023) 109507.

[38] L. Zhu, Z. Liu, S. Han, Deep leakage from gradients, arXiv:1906.08935, 2019.

[39] S. Yang, W. Zheng, M. Xie, X. Zhang, Research of federated learning application methods and social responsibility, IEEE Trans. Big Data 10 (6) (2024) 989-1000.

[40] T. Zou, Y. Liu, Y. Kang, W. Liu, Y. He, Z. Yi, Q. Yang, Y.-Q Zhang, Defending batch- level label inference and replacement attacks in vertical federated learning, IEEE Trans. Big Data 10 (6) (2024) 1016-1027.

[41] L. Han, D. Fan, J. Liu, W. Du,Federated learning differential privacy preservation method based on differentiated noise addition, in:2023 8th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA), 2023, pp. 285-289.

[42] P. Yao, H. Wang, C. Zheng, J. Yang, L. Wang,Efficient federated learning aggregation protocol using approximate homomorphic encryption, in:2023 26th International Conference on Computer Supported Cooperative Work in Design (CSCWD), 2023, pp. 1884-1889.

[43] U. Majeed, S.S. Hassan, C.S. Hong,Cross-silo model-based secure federated transfer learning for flow-based traffic classification, in:2021 International Conference on Information Networking (ICOIN), 2021, pp. 588-593.

[44] S. Latif, Z. Zou, Z. Idrees, J. Ahmad, A novel attack detection scheme for the industrial Internet of things using a lightweight random neural network, IEEE Access 8 (2020) 89337-89350.

[45] B. Li, Y. Wu, J. Song, R. Lu, T. Li, L. Zhao, Deepfed: federated deep learning for intrusion detection in industrial cyber-physical systems, IEEE Trans. Ind. Inform. 17 (8) (2021) 5615-5624.

[46] Y. Luan, Network traffic anomaly detection based on federated learning, in: 2024 4th International Conference on Neural Networks, Information and Communication Engineering (NNICE), 2024, pp. 224-228.

[47] C. Sun, B. Chen, Y. Bu, D. Zhang, Traffic classification method based on federated semi-supervised learning, in: Proceedings of the 2022 6th International Conference on Electronic Information Technology and Computer Engineering, EITCE ’22, Asso- ciation for Computing Machinery, New York, NY, USA, 2023, pp. 875-882.

[48] Z. Wang, Z. Li, M. Fu, Y. Ye, P. Wang, Network traffic classification based on feder- ated semi-supervised learning, J. Syst. Archit. 149 (2024) 103091.

[49] Z. Zhao, Y. Guo, J.H. Wang, H. Wang, C. Zhang, C. An, Cl-etc: a contrastive learning method for encrypted traffic classification,in: 2022 IFIP Networking Conference (IFIP Networking), 2022, pp. 1-9.

[50] R. Hjelm, A. Fedorov, S. Lavoie-Marchildon, K. Grewal, P. Bachman, A. Trischler, Y. Bengio, Learning deep representations by mutual information estimation and maxi- mization, arXiv:1808.06670, 2019.

[51] P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, D. Krishnan, Supervised contrastive learning, arXiv:2004. 11362, 2021.

[52] S. Wang, T. Tuor, T. Salonidis, K.K. Leung, C. Makaya, T. He, K. Chan, Adaptive fed- erated learning in resource constrained edge computing systems, IEEE J. Sel. Areas Commun. 37 (6) (2019) 1205-1221.

[53] P. Sun, H. Che, Z. Wang, Y. Wang, T. Wang, L. Wu, H. Shao, Pain-fl: personal- ized privacy-preserving incentive for federated learning, IEEE J. Sel. Areas Commun. 39 (12) (2021) 3805-3820.

[54] G. Aceto, D. Ciuonzo, A. Montieri, A. Nascita, A. Pescapé,Encrypted multitask traffic classification via multimodal deep learning, in: IEEE International Conference on Communications (ICC), 2021, p. 2021.

[55] A.H. Lashkari, G. Draper-Gil, M.S.I. Mamun, A.A. Ghorbani,Characterization of en- crypted and vpn traffic using time-related features, in:The International Conference on Information Systems Security and Privacy (ICISSP), 2016.

[56] M. Lopez-Martin, B. Carro, A. Sanchez-Esguevillas, J. Lloret, Network traffic classifier with convolutional and recurrent neural networks for Internet of things, IEEE Access 5 (2017) 18042-18050.

[57] W. Wang, M. Zhu, J. Wang, X. Zeng, Z. Yang, End-to-end encrypted traffic classifica- tion with one-dimensional convolution neural networks, in: 2017 IEEE International Conference on Intelligence and Security Informatics (ISI), 2017, pp. 43-48.

[58] S. Rezaei, X. Liu,Multitask learning for network traffic classification, in:2020 29th International Conference on Computer Communications and Networks (ICCCN), 2020, pp. 1-9.

[59] Y. Zhao, J. Chen, D. Wu, J. Teng, S. Yu,Multi-task network anomaly detection using federated learning, in:The Tenth International Symposium, 2019.

[60] H. Sun, Y. Xiao, J. Wang, J. Wang, Q. Qi, J. Liao, X. Liu, Common knowledge based and one-shot learning enabled multi-task traffic classification, IEEE Access 7 (2019) 39485-39495.