Network traffic classification: Techniques, datasets, and challenges

Abstract

In network traffic classification, it is important to understand the correlation between network traffic and its causal application, protocol, or service group, for example, in facilitating lawful interception, ensuring the quality of service, preventing application choke points, and facilitating malicious behavior identification. In this paper, we review existing network classification techniques, such as port-based identification and those based on deep packet inspection, statistical features in conjunction with machine learning, and deep learning algorithms. We also explain the implementations, advantages, and limitations associated with these techniques. Our review also extends to publicly available datasets used in the literature. Finally, we discuss existing and emerging challenges, as well as future research directions.

Keywords

Network classification / Machine learning / Deep learning / Deep packet inspection / Traffic monitoring

References

[1] E. Al Neyadi, S. Al Shehhi, A. Al Shehhi, N. Al Hashimi, Q. Mohammad, S. Alrabaee, Discovering public wi-fi vulnerabilities using raspberry pi and kali linux, in: 2020 12th Annual Undergraduate Research Conference on Applied Computing (URC), IEEE, 2020, pp. 1-4.

[2] International Telecommunication Union, Measuring digital development facts and figures. https://www.itu.int/en/ITU-D/Statistics/Documents/facts/FactsFigures2019.pdf (accessed 25 June 2003).

[3] H. Mohajeri Moghaddam, Skypemorph: Protocol Obfuscation for Censorship Resistance, Master's Thesis, University of Waterloo, 2013.

[4] A. Azab,Classification of Network Information Flow Analysis (CONIFA) to Detect New Application Versions, Ph.D. thesis, Federation University, 2015.

[5] S. AlDaajeh, H. Saleous, S. Alrabaee, E. Barka, F. Breitinger, K.-K.R. Choo, The role of national cybersecurity strategies on the improvement of cybersecurity education, Comput. Secur. 119 (2022). ARTN 102754.

[6] S. Alrabaee, M. Al-Kfairy, E. Barka, Efforts and suggestions for improving cybersecurity education, in: 2022 IEEE Global Engineering Education Conference (EDUCON), IEEE, 2022, pp. 1161-1168.

[7] A. Azab, Packing resistant solution to group malware binaries, Int. J. Secur. Network. 15 (3) (2020) 123-132.

[8] S. Alrabaee, A stratified approach to function fingerprinting in program binaries using diverse features, Expert Syst. Appl. 193 (2022), 116384.

[9] P. Casey, M. Topor, E. Hennessy, S. Alrabaee, M. Aloqaily, A. Boukerche, Applied comparative evaluation of the metasploit evasion module, in: 2019 IEEE Symposium on Computers and Communications (ISCC), IEEE, 2019, pp. 1-6.

[10] A. Khraisat, A. Alazab, M. Hobbs, J.H. Abawajy, A. Azab, Trends in crime toolkit development, in: Network Security Technologies: Design and Applications, IGI global, 2014, pp. 28-43.

[11] S. Alrabaee, M. Debbabi, L. Wang, A survey of binary code fingerprinting approaches: taxonomy, methodologies, and features, ACM Comput. Surv. 55 (1)(2022) 1-41.

[12] M. Finsterbusch, C. Richter, E. Rocha, J.-A. Muller, K. Hanssgen, A survey of payload-based traffic classification approaches, IEEE Commun. Tutorial. 16 (2)(2014) 1135-1156.

[13] S. Valenti, D. Rossi, A. Dainotti, A. Pescapè, A. Finamore, M. Mellia, Reviewing Traffic Classification, Springer Berlin Heidelberg, 2013, pp. 123-147.

[14] F. Pacheco, E. Exposito, M. Gineste, C. Baudoin, J. Aguilar, Towards the deployment of machine learning solutions in network traffic classification: a systematic survey, IEEE Commun. Tutorial. 21 (2) (2019) 1988-2014.

[15] O. Salman, I.H. Elhajj, A. Kayssi, A. Chehab, A review on machine learning-based approaches for internet traffic classification, Annal Telecommun. 75 (11) (2020) 673-710.

[16] H. Tahaei, F. Afifi, A. Asemi, F. Zaki, N.B. Anuar, The rise of traffic classification in iot networks: a survey, J. Netw. Comput. Appl. 154 (2020), 102538.

[17] P. Wang, X. Chen, F. Ye, Z. Sun, A survey of techniques for mobile service encrypted traffic classification using deep learning, IEEE Access 7 (2019) 54024-54033.

[18] J. Zhao, X. Jing, Z. Yan, W. Pedrycz, Network traffic classification for data fusion: a survey, Inf. Fusion 72 (2021) 22-47.

[19] Internet Assigned Numbers Authority (IANA), Service name and transport protocol port number registry. https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml (accessed 1 July 2020).

[20] A.W. Moore, K. Papagiannaki, Toward the accurate identification of network applications, in: C. Dovrolis (Ed.), Passive and Active Network Measurement, Springer Berlin Heidelberg, Berlin, Heidelberg, 2005, pp. 41-54.

[21] A. Madhukar, C. Williamson, A longitudinal study of p2p traffic classification, in: 14th IEEE International Symposium on Modeling, Analysis, and Simulation, 2006, pp. 179-188.

[22] S. Sen, O. Spatscheck, D. Wang, Accurate, scalable in-network identification of p2p traffic using application signatures,in:Proceedings of the 13th International Conference on World Wide Web, WWW ’04, Association for Computing Machinery, New York, USA, 2004, pp. 512-521.

[23] A. Azab, P. Watters, R. Layton,Characterising network traffic for skype forensics, in:2012 Third Cybercrime and Trustworthy Computing Workshop, 2012, pp. 19-27.

[24] P. Khandait, N. Hubballi, B. Mazumdar, Efficient keyword matching for deep packet inspection based network traffic classification, in: 2020 International Conference on COMmunication Systems & NETworkS (COMSNETS), IEEE, 2020, pp. 567-570.

[25] X. Wang, J. Jiang, Y. Tang, B. Liu, X. Wang, Strid2fa: scalable regular expression matching for deep packet inspection, in: 2011 IEEE International Conference on Communications, ICC, 2011, pp. 1-5.

[26] S. Fernandes, R. Antonello, T. Lacerda, A. Santos, D. Sadok, T. Westholm, Slimming down deep packet inspection systems, in: IEEE INFOCOM Workshops, 2009, pp. 1-6.

[27] N. Hubballi, M. Swarnkar, $bitcoding$: network traffic classification through encoded bit level signatures, IEEE/ACM Trans. Netw. 26 (5) (2018) 2334-2346.

[28] N. Hubballi, M. Swarnkar, M. Conti, Bitprob: probabilistic bit signatures for accurate application identification, IEEE Trans. Network Service Manage. 17 (3)(2020) 1730-1741.

[29] M. Hall, Correlation-based Feature Selection for Machine Learning, Ph.D. thesis, The University of Waikato, 2000.

[30] M. Dash, H. Liu, H. Motoda, Consistency based feature selection, in: Pacific-asia Conference on Knowledge Discovery and Data Mining, Springer, 2000, pp. 98-109.

[31] C.A. Ratanamahatana, D. Gunopulos, Scaling up the naive bayesian classifier: using decision trees for feature selection, Appl. Artif. Intell. 17 (5-6) (2003) 475-487.

[32] H. Liu, R. Setiono, Chi2: feature selection and discretization of numeric attributes,in: Proceedings of 7th IEEE International Conference on Tools with Artificial Intelligence, 1995, pp. 388-391.

[33] I.H. Witten, E. Frank, Data mining: practical machine learning tools and techniques with java implementations, SIGMOD Rec 31 (1) (2002) 76-77.

[34] P. Christen, Evaluation of Matching Quality and Complexity, Springer Berlin Heidelberg, Berlin, Heidelberg, 2012, pp. 163-184.

[35] A. Azab, R. Layton, M. Alazab, J. Oliver,Mining malware to detect variants, in:2014 Fifth Cybercrime and Trustworthy Computing Conference, 2014, pp. 44-53.

[36] Y. Reich, S.J. Fenves, in: D. H. Fisher, M.J. Pazzani, P. Langley (Eds.), Chapter 12 -the Formation and Use of Abstract Concepts in design Concept Formation, 1991, pp. 323-353.

[37] C. Cortes, V. Vapnik, Support-vector networks, Mach. Learn. 20 (3) (1995) 273-297.

[38] D.D. Lewis, Naive (bayes) at forty: the independence assumption in information retrieval,in: C. Nédellec, C. Rouveirol (Machine Learning:Eds.), ECML-98, Springer Berlin Heidelberg, Berlin, Heidelberg, 1998, pp. 4-15.

[39] J.R. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1993.

[40] T. Bujlow, T. Riaz, J.M. Pedersen, A method for classification of network traffic based on c5.0 machine learning algorithm, in: 2012 International Conference on Computing, Networking and Communications (ICNC), 2012, pp. 237-241.

[41] L. Breiman, Random forests, Mach. Learn. 45 (1) (2001) 5-32.

[42] S. Huang, K. Chen, C. Liu, A. Liang, H. Guan,A statistical-feature-based approach to internet traffic classification using machine learning, in:2009 International Conference on Ultra Modern Telecommunications Workshops, 2009, pp. 1-6.

[43] N. Williams, S. Zander, G. Armitage, Evaluating Machine Learning Algorithms for Automated Network Application Identification, Tech. Rep, Centre for Advanced Internet Architectures (CAIA), 2006.

[44] Z. Fan, R. Liu, Investigation of machine learning based network traffic classification, in: 2017 International Symposium on Wireless Communication Systems, ISWCS, 2017, pp. 1-6.

[45] R. Yuan, Z. Li, X. Guan, L. Xu, An svm-based machine learning method for accurate internet traffic classification, Inf. Syst. Front 12 (2) (2010) 149-156.

[46] A. Jenefa, M.B. Moses, An Upgraded c5.0 Algorithm for Network Application Identification, in: 2018 2nd International Conference on Trends in Electronics and Informatics, ICOEI, 2018, pp. 789-794.

[47] K.L. Dias, M.A. Pongelupe, W.M. Caminhas, L. de Errico, An innovative approach for real-time network traffic classification, Comput. Network. 158 (2019) 143-157.

[48] R. Alshammari, A.N. Zincir-Heywood, An Investigation on the Identification of Voip Traffic: Case Study on Gtalk and Skype,in: 2010 International Conference on Network and Service Management, 2010, pp. 310-313.

[49] R. Alshammari, A.N. Zincir-Heywood,Machine Learning Based Encrypted Traffic Classification: Identifying Ssh and Skype, in: 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, 2009, pp. 1-8.

[50] G. Sun, T. Chen, Y. Su, C. Li, Internet traffic classification based on incremental support vector machines, Mobile Network. Appl. 23 (4) (2018) 789-796.

[51] J. Cao, D. Wang, Z. Qu, H. Sun, B. Li, C.-L. Chen, An improved network traffic classification model based on a support vector machine, Symmetry 12 (2) (2020) 301.

[52] A.S. Khatouni, N. Zincir-Heywood, Integrating machine learning with off-the-shelf traffic flow features for http/https traffic classification, in: 2019 IEEE Symposium on Computers and Communications (ISCC), 2019, pp. 1-7.

[53] Argus: the network audit record generation and utilization system. https://qosient.com/argus/downloads.shtml (accessed 25 June 2020).

[54] Silk (system for internet-level knowledge). https://tools.netsa.cert.org/silk/ (accessed 25 June 2020).

[55] A. Finamore, M. Mellia, M. Meo, M.M. Munafo, P.D. Torino, D. Rossi, Experiences of internet traffic monitoring with tstat, IEEE Network 25 (3) (2011) 8-14.

[56] S. Burschka, B. Dupasquier,Tranalyzer: versatile high performance network traffic analyser, in: 2016 IEEE Symposium Series on Computational Intelligence (SSCI), 2016, pp. 1-8.

[57] S. Dong, Multi class SVM algorithm with active learning for network traffic classification, Expert Syst. Appl. 176 (2021), 114885.

[58] A.A. Afuwape, Y. Xu, J.H. Anajemba, G. Srivastava, Performance evaluation of secured network traffic classification using a machine learning approach, Comput. Stand. Interfac. 78 (4) (2021), 103545.

[59] E. Ganesan, I.-S. Hwang, A.T. Liem, M.S. Ab-Rahman, Sdn-enabled fiwi-iot smart environment network traffic classification using supervised ml models, Photonics 8 (6) (2021) 201.

[60] J. Li, S. Zhang, Y. Lu, J. Yan,Real-time p2p traffic identification, in: IEEE GLOBECOM 2008-2008 IEEE Global Telecommunications Conference, 2008, pp. 1-5.

[61] L. Bernaille, R. Teixeira, I. Akodkenou, A. Soule, K. Salamatian, Traffic classification on the fly, SIGCOMM Comput. Commun. Rev. 36 (2) (2006) 23-26.

[62] C. Gu, S. Zhang, Y. Sun, Realtime encrypted traffic identification using machine learning, J. SW 6 (6) (2011) 1009-1016.

[63] Y. Liu, J. Chen, P. Chang, X. Yun, A Novel Algorithm for Encrypted Traffic Classification Based on Sliding Window of Flow’s First N Packets, in: 2017 2nd IEEE International Conference on Computational Intelligence and Applications (ICCIA), 2017, pp. 463-470.

[64] L. Peng, B. Yang, Y. Chen, Z. Chen, Effectiveness of statistical features for early stage internet traffic identification, Int. J. Parallel Program. 44 (1) (2016) 181-197.

[65] P. Branch, J. But, Rapid and generalized identification of packetized voice traffic flows, in: 37th Annual IEEE Conference on Local Computer Networks, 2012, pp. 85-92.

[66] A. Azab, R. Layton, M. Alazab, P. Watters,Skype traffic classification using cost sensitive algorithms, in:2013 Fourth Cybercrime and Trustworthy Computing Workshop, 2013, pp. 14-21.

[67] A. Azab, O. Maruatona, P. Watters, AVOCAD: adaptive terrorist comms surveillance and interception using machine learning, in: 2019 18th IEEE International Conference on Trust, Security and Privacy in Computing and Communications/13th IEEE International Conference on Big Data Science and Engineering, TrustCom/BigDataSE, 2019, pp. 85-94.

[68] A. Azab, M. Alazab, M. Aiash, Machine Learning Based Botnet Identification Traffic, in: 2016 IEEE Trustcom BigDataSE ISPA, 2016, pp. 1788-1794.

[69] A. Azab, The effectiveness of cost sensitive machine learning algorithms in classifying zeus flows, J. Info. Comput. Security. 17 (3-4) (2021) 332-350.

[70] T.T. Nguyen, G. Armitage, A survey of techniques for internet traffic classification using machine learning, IEEE Commun. Tutorial. 10 (4) (2008) 56-76.

[71] M. Halkidi, Y. Batistakis, M. Vazirgiannis, Cluster validity methods: Part i, SIGMOD Rec 31 (2) (2002) 40-45.

[72] Y. Wang, Y. Xiang, J. Zhang, S. Yu,A novel semi-supervised approach for network traffic clustering, in:2011 5th International Conference on Network and System Security, 2011, pp. 169-175.

[73] R. Dubin, A. Dvir, O. Pele, O. Hadar, I. Richman, O. Trabelsi, Real Time Video Quality Representation Classification of Encrypted Http Adaptive Video Streaming-the Case of Safari, 2016 arXiv:1602.00489.

[74] Y. Du, R. Zhang, Design of a method for encrypted p2p traffic identification using k-means algorithm, Telecommun. Syst. 53 (1) (2013) 163-168.

[75] H. Singh,Performance analysis of unsupervised machine learning techniques for network traffic classification, in:2015 Fifth International Conference on Advanced Computing Communication Technologies, 2015, pp. 401-404.

[76] J. Zhang, Y. Xiang, W. Zhou, Y. Wang, Unsupervised traffic classification using flow statistical properties and ip packet payload, J. Comput. Syst. Sci. 79 (5) (2013) 573-585.

[77] A. Alalousi, R.R. Othman, M. Abualhaj, M. Anbar, S. Yaakob, A preliminary performance evaluation of k-means, knn and em unsupervised machine learning methods for network flow classification, Int. J. Electr. Comput. Eng. 6 (2016) 778.

[78] J. Höchst, L. Baumgärtner, M. Hollick, B. Freisleben, Unsupervised traffic flow classification using a neural autoencoder, in: 2017 IEEE 42nd Conference on Local Computer Networks, LCN, 2017, pp. 523-526.

[79] H. Alizadeh, A. Khoshrou, A. Zúquete, Traffic classification and verification using unsupervised learning of Gaussian mixture models, in: 2015 IEEE International Workshop on Measurements Networking, 2015, pp. 1-6.

[80] T. Wiradinata, A. Paramita, Clustering and feature selection technique for improving internet traffic classification using k-nn, J. Adv. Comput. Network. 4 (1) (2016) 24-27.

[81] J. Zhang, C. Chen, Y. Xiang, W. Zhou, A.V. Vasilakos, An effective network traffic classification method with unknown flow detection, IEEE Trans. Network Service Manage. 10 (2) (2013) 133-147.

[82] T. Glennan, C. Leckie, S. Erfani, Improved classification of known and unknown network traffic flows using semi-supervised machine learning, in: Information Security and Privacy, vol. 9723, Springer International Publishing, 2016, pp. 493-501.

[83] T. Bakhshi, B. Ghita, On internet traffic classification: a two-phased machine learning approach, J. Comput. Network. Commun. 2016 (2016) 21.

[84] A. Fahad, A. Almalawi, Z. Tari, K. Alharthi, F.S. Al Qahtani, M. Cheriet, Semtra: a semi-supervised approach to traffic flow labeling with minimal human effort, Pattern Recogn. 91 (2019), 1-12.

[85] C. Rotsos, J. Van Gael, A.W. Moore, Z. Ghahramani, Probabilistic graphical models for semi-supervised traffic classification, in: Proceedings of the 6th International Wireless Communications and Mobile Computing Conference, IWCMC ’10, Association for Computing Machinery, New York, USA, 2010, pp. 752-757.

[86] J. Erman, A. Mahanti, M. Arlitt, I. Cohen, C. Williamson, Offline/realtime traffic classification using semi-supervised learning, Perform. Eval 64 (9) (2007) 1194-1213.

[87] J. Gao, F. Liang, W. Fan, Y. Sun, J. Han, A graph-based consensus maximization approach for combining multiple supervised and unsupervised models, IEEE Trans. Knowl. Data Eng. 25 (1) (2013) 15-28.

[88] A. Almalawi, A. Fahad, Z. Tari, M.A. Cheema, I. Khalil, knnvwc: an efficient k-nearest neighbours approach based on various-widths clustering, in: 2016 IEEE 32nd International Conference on Data Engineering, ICDE, 2016, pp. 1572-1573.

[89] S. Zhao, Y. Zhang, P. Chang, Network traffic classification using tri-training based on statistical flow characteristics, in: 2017 IEEE Trustcom/BigDataSE/ICESS, 2017, pp. 323-330.

[90] J. Zhang, X. Chen, Y. Xiang, W. Zhou, J. Wu, Robust network traffic classification, IEEE/ACM Trans. Netw. 23 (4) (2015) 1257-1270.

[91] F. Noorbehbahani, S. Mansoori, A new semi-supervised method for network traffic classification based on x-means clustering and label propagation, in: 2018 8th International Conference on Computer and Knowledge Engineering, ICCKE, 2018, pp. 120-125.

[92] T. Ede, R. Bortolameotti, A. Continella, J. Ren, D. Dubois, M. Lindorfer, D. Choffnes, M. Steen, A. Peter, Flowprint: semi-supervised mobile-app fingerprinting on encrypted network traffic,in:Proc. Network and Distributed System Security Symposium (NDSS 2020), USA, 2020.

[93] M.A. Lopez, D.M. Mattos, O.C.M. Duarte, G. Pujolle, A fast unsupervised preprocessing method for network monitoring, Annal Telecommun. 74 (3) (2019) 139-155.

[94] Z. Wang, The applications of deep learning on traffic identification, BlackHat USA 24 (11) (2015) 1-10.

[95] W. Wang, M. Zhu, X. Zeng, X. Ye, Y. Sheng,Malware traffic classification using convolutional neural network for representation learning, in:2017 International Conference on Information Networking (ICOIN), 2017, pp. 712-717.

[96] W. Wang, M. Zhu, J. Wang, X. Zeng, Z. Yang, End-to-end Encrypted Traffic Classification with One-Dimensional Convolution Neural Networks, in: 2017 IEEE International Conference on Intelligence and Security Informatics, ISI, 2017, pp. 43-48.

[97] M. Lopez-Martin, B. Carro, A. Sanchez-Esguevillas, J. Lloret, Network traffic classifier with convolutional and recurrent neural networks for internet of things, IEEE Access 5 (2017) 18042-18050.

[98] M. Lotfollahi, R. Shirali hossein zade, M. Jafari Siavoshani, M. Saberian, Deep packet: a novel approach for encrypted traffic classification using deep learning, Soft Comput. 24 (2020) 1999-2012.

[99] H. Huang, H. Deng, J. Chen, L. Han, W. Wang, Automatic multi-task learning system for abnormal network traffic detection, Int. J. Eng. Technol. Learn. 13 (4)(2018) 4-20.

[100] P. Wang, F. Ye, X. Chen, Y. Qian, Datanet: deep learning based encrypted network traffic classification in sdn home gateway, IEEE Access 6 (2018) 55380-55391.

[101] G. Aceto, D. Ciuonzo, A. Montieri, A. Pescapé, Mobile encrypted traffic classification using deep learning: experimental evaluation, lessons learned, and challenges, IEEE Trans. Network Service Manage. 16 (2) (2019) 445-458.

[102] G. Aceto, D. Ciuonzo, A. Montieri, A. Pescapé, Mobile encrypted traffic classification using deep learning, in: 2018 Network Traffic Measurement and Analysis Conference, TMA, 2018, pp. 1-8.

[103] H. Zhou, Y. Wang, X. Lei, Y. Liu, A method of improved cnn traffic classification, in: 2017 13th International Conference on Computational Intelligence and Security, CIS, 2017, pp. 177-181.

[104] V. Tong, H.A. Tran, S. Souihi, A. Mellouk, A novel quic traffic classifier based on convolutional neural networks, in: 2018 IEEE Global Communications Conference, GLOBECOM, 2018, pp. 1-6.

[105] Z. Chen, K. He, J. Li, Y. Geng,Seq2img: a sequence-to-image based approach towards ip traffic classification using convolutional neural networks, in: 2017 IEEE International Conference on Big Data (Big Data), 2017, pp. 1271-1276.

[106] G. Aceto, D. Ciuonzo, A. Montieri, A. Pescapè, Mimetic: mobile encrypted traffic classification using multimodal deep learning, Comput. Network. 165 (2019), 106944.

[107] G. Aceto, D. Ciuonzo, A. Montieri, A. Pescapé, Toward effective mobile encrypted traffic classification through deep learning, Neurocomputing 409 (2020) 306-315.

[108] Z. Bu, B. Zhou, P. Cheng, K. Zhang, Z.-H. Ling, Encrypted network traffic classification using deep and parallel network-in-network models, IEEE Access 8 (2020) 132950-132959.

[109] C. Liu, L. He, G. Xiong, Z. Cao, Z. Li, Fs-net,A flow sequence network for encrypted traffic classification, in: IEEE INFOCOM 2019 -IEEE Conference on Computer Communications, 2019, pp. 1171-1179.

[110] C. Liu, Z. Cao, G. Xiong, G. Gou, S.-M. Yiu, L. He, Mampf: encrypted traffic classification based on multi-attribute markov probability fingerprints,in:2018 IEEE/ACM 26th International Symposium on Quality of Service, IWQoS, 2018, pp. 1-10.

[111] G. Aceto, D. Ciuonzo, A. Montieri, A. Pescapè, Mimetic: mobile encrypted traffic classification using multimodal deep learning, Comput. Network. 165 (2019), 106944.

[112] A.W. Moore, D. Zuev, Internet traffic classification using bayesian analysis techniques, SIGMETRICS Perform. Eval. Rev. 33 (1) (2005) 50-60.

[113] Wangwei, Ustc-tfc. https://github.com/echowei/DeepTraffic/tree/master/1.malware_traffic_classification/1.DataSet(USTC-TFC2016, 2016 (accessed 1 July 2020).

[114] The Canadian Institute for Cybersecurity, Vpn-nonvpn dataset (ISCXVPN2016). https://www.unb.ca/cic/datasets/vpn.html (accessed 1 July 2020).

[115] S. García, M. Grill, J. Stiborek, A. Zunino, An empirical comparison of botnet detection methods, Comput. Secur. 45 (2014) 100-123.

[116] MAWI, Mawi working group traffic archive. http://mawi.wide.ad.jp/mawi/ (accessed 1 July 2020).

[117] W.R. Group, Auckland ii. https://wand.net.nz/wits/auck/2/auckland_ii.php (accessed 1 July 2020).

[118] Unibs, Unibs, Data sharing. http://netweb.ing.unibs.it/ntw/tools/traces/ (accessed 1 July 2020).

[119] P. Lizhi, Z. Hongli, Y. Bo, C. Yuehui, W. Tong, Traffic labeller: collecting internet traffic samples with accurate application information, China Communications 11 (1) (2014) 69-78.

[120] Anon17, Network traffic dataset of anonymity services. https://web.cs.dal.ca/shahbar/data.html (accessed 1 July 2020).

[121] G. Aceto, D. Ciuonzo, A. Montieri, V. Persico, A. Pescapé, Mirage: mobile-app traffic capture and ground-truth creation,in:2019 4th International Conference on Computing, Communications and Security, ICCCS), 2019, pp. 1-8.

[122] A. Sivanathan, H.H. Gharakheili, F. Loi, A. Radford, C. Wijenayake, A. Vishwanath, V. Sivaraman, Classifying iot devices in smart environments using network traffic characteristics, IEEE Trans. Mobile Comput. 18 (8) (2019) 1745-1759.

[123] R. Wang, Z. Liu, Y. Cai, D. Tang, J. Yang, Z. Yang, Benchmark data for mobile app traffic research, in: Proceedings of the 15th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services, MobiQuitous ’18, Association for Computing Machinery, New York, NY, USA, 2018, pp. 402-411.

[124] M. Usama, A. Qayyum, J. Qadir, A. Al-Fuqaha, Black-box adversarial machine learning attack on network traffic classification, in: 2019 15th International Wireless Communications Mobile Computing Conference, IWCMC), 2019, pp. 84-89.

[125] K. Ren, T. Zheng, Z. Qin, X. Liu, Adversarial attacks and defenses in deep learning, Engineering 6 (3) (2020) 346-360, https://doi.org/10.1016/j.eng.2019.12.012.

[126] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, R. Fergus, Intriguing properties of neural networks, in: 2nd International Conference on Learning Representations, ICLR, 2014.

[127] I. Goodfellow, J. Shlens,C. Szegedy, Explaining and Harnessing Adversarial Examples, arXiv 1412.6572.

[128] N. Carlini, D. Wagner, Towards evaluating the robustness of neural networks, in: 2017 IEEE Symposium on Security and Privacy, SP, 2017, pp. 39-57.

[129] N. Papernot, P. McDaniel, S. Jha, M. Fredrikson, Z.B. Celik, A. Swami, The limitations of deep learning in adversarial settings, in: 2016 IEEE European Symposium on Security and Privacy, EuroS P, 2016, pp. 372-387.