A structured distributed learning framework for irregular cellular spatial-temporal traffic prediction^✩

Abstract

Spatial-temporal traffic prediction technology is crucial for network planning, resource allocation optimizing, and user experience improving. With the development of virtual network operators, multi-operator collaborations, and edge computing, spatial-temporal traffic data has taken on a distributed nature. Consequently, non- centralized spatial-temporal traffic prediction solutions have emerged as a recent research focus. Currently, the majority of research typically adopts federated learning methods to train traffic prediction models distributed on each base station. This method reduces additional burden on communication systems. However, this method has a drawback: it cannot handle irregular traffic data. Due to unstable wireless network environments, device failures, insufficient storage resources, etc., data missing inevitably occurs during the process of collecting traffic data. This results in the irregular nature of distributed traffic data. Yet, commonly used traffic prediction models such as Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) typically assume that the data is complete and regular. To address the challenge of handling irregular traffic data, this paper transforms irregular traffic prediction into problems of estimating latent variables and generating future traffic. To solve the aforementioned problems, this paper introduces split learning to design a structured distributed learning framework. The framework comprises a Global-level Spatial structure mining Model (GSM) and several Node- level Generative Models (NGMs). NGM and GSM represent Seq2Seq models deployed on the base station and graph neural network models deployed on the cloud or central controller. Firstly, the time embedding layer in NGM establishes the mapping relationship between irregular traffic data and regular latent temporal feature variables. Secondly, GSM collects statistical feature parameters of latent temporal feature variables from various nodes and executes graph embedding for spatial-temporal traffic data. Finally, NGM generates future traffic based on latent temporal and spatial feature variables. The introduction of the time attention mechanism enhances the framework’s capability to handle irregular traffic data. Graph attention network introduces spatially correlated base station traffic feature information into local traffic prediction, which compensates for missing information in local irregular traffic data. The proposed framework effectively addresses the distributed prediction issues of irregular traffic data. By testing on real world datasets, the proposed framework improves traffic prediction accuracy by 35% compared to other commonly used distributed traffic prediction methods.

Keywords

Network measurement and analysis / Distributed learning / Irregular time series / Cellular spatial-temporal traffic / Traffic prediction

References

[1] X. Liu, G. Chuai, X. Wang, Z. Xu, W. Gao, Qoe assessment model based on continu- ous deep learning for video in wireless networks, IEEE Trans. Mob. Comput. 22 (6) (2023) 3619-3633.

[2] Y. Hou, K. Zhang, X. Liu, G. Chuai, W. Gao, X. Chen, An inter-slice RB leasing and association adjustment scheme in O-RAN, IEEE Trans. Netw. Serv. Manag. 21 (1) (2024) 402-417.

[3] X. Chen, J. Wu, Y. Cai, H. Zhang, T. Chen, Energy-efficiency oriented traffic offload- ing in wireless networks: a brief survey and a learning approach for heterogeneous cellular networks, IEEE J. Sel. Areas Commun. 33 (4) (2015) 627-640.

[4] H. Song, J. Bai, Y. Yi, J. Wu, L. Liu, Artificial intelligence enabled Internet of things: network architecture and spectrum access, IEEE Comput. Intell. Mag. 15 (1) (2020) 44-51.

[5] J.L. Schafer, Analysis of Incomplete Multivariate Data, CRC Press, 1997.

[6] Y. Xu, F. Yin, W. Xu, J. Lin, S. Cui, Wireless traffic prediction with scalable Gaussian process: framework, algorithms, and verification, IEEE J. Sel. Areas Commun. 37 (6) (2019) 1291-1306.

[7] Y. Yao, B. Gu, Z. Su, M. Xu, W. Guizani, Mvstgn: A multi-view spatial-temporal graph network for cellular traffic prediction, IEEE Trans. Mob. Comput. 22 (5) (2023) 2837-2849.

[8] İ. Kök, S. Özdemir, Deepmdp: a novel deep-learning-based missing data prediction protocol for iot, IEEE Internet Things J. 8 (1) (2020) 232-243.

[9] P.B. Weerakody, K.W. Wong, G. Wang, W. Ela, A review of irregular time series data handling with gated recurrent neural networks, Neurocomputing 441 (2021) 161-178.

[10] S. De, M. Bermudez-Edo, H. Xu, Z. Cai, Deep generative models in the industrial Internet of things: a survey, IEEE Trans. Ind. Inform. 18 (9) (2022) 5728-5737.

[11] X. Chen, G. Chuai, K. Zhang, W. Gao, Spatial-temporal cellular traffic prediction: a novel method based on causality and graph attention network, in: 2023 IEEE Wire- less Communications and Networking Conference (WCNC), IEEE, 2023, pp. 1-6.

[12] S.N. Shukla, B.M. Marlin, A survey on principles, models and methods for learning from irregularly sampled time series, preprint, arXiv:2012.00168.

[13] O. Nassef, W. Sun, H. Purmehdi, M. Tatipamula, T. Mahmoodi, A survey: distributed machine learning for 5g and beyond, Comput. Netw. 207 (2022) 108820.

[14] B.K. Mathew, J.C. Ng, J.L. Zerbe, Using proxies to enable on-device machine learn- ing, Dec. 28 2021.

[15] A. Pacheco, E. Flores, R. Sánchez, S. Almanza-García, Smart classrooms aided by deep neural networks inference on mobile devices, in: 2018 IEEE International Con- ference on Electro/Information Technology (EIT), IEEE, 2018, pp. 0605-0609.

[16] Y. Kim, J. Kim, D. Chae, D. Kim, J.Kim, 𝜇layer: low latency on-device inference using cooperative single-layer acceleration and processor-friendly quantization,in:Proceedings of the Fourteenth EuroSys Conference 2019, 2019, pp. 1-15.

[17] R. Mayer, H.-A. Jacobsen, Scalable deep learning on distributed infrastructures: chal- lenges, techniques, and tools, ACM Comput. Surv. 53 (1) (2020) 1-37.

[18] B. McMahan, E. Moore, D. Ramage, S. Hampson, B.A. y Arcas, Communication- efficient learning of deep networks from decentralized data, in: Artificial Intelligence and Statistics, PMLR, 2017, pp. 1273-1282.

[19] V. Perifanis, N. Pavlidis, R.-A. Koutsiamanis, P.S. Efraimidis, Federated learning for 5g base station traffic forecasting, Comput. Netw. 235 (2023) 109950.

[20] D. Kim, S. Shin, J. Jeong, J. Lee, Joint edge server selection and dataset manage- ment for federated learning-enabled mobile traffic prediction, IEEE Internet Things J. 11 (3) (2024) 4971-4986.

[21] J. Nan, M. Ai, A. Liu, X. Duan, Regional-union based federated learning for wireless traffic prediction in 5g-advanced/6g network, in: 2022 IEEE/CIC International Con- ference on Communications in China (ICCC Workshops), IEEE, 2022, pp. 423-427.

[22] L. Zhang, C. Zhang, B. Shihada, Efficient wireless traffic prediction at the edge: a federated meta-learning approach, IEEE Commun. Lett. 26 (7) (2022) 1573-1577.

[23] Y. Liu, J. James, J. Kang, D. Niyato, S. Zhang, Privacy-preserving traffic flow predic- tion: a federated learning approach, IEEE Internet Things J. 7 (8) (2020) 7751-7763.

[24] X. Wang, Z. Zhou, F. Xiao, K. Xing, Z. Yang, Y. Liu, C. Peng, Spatio-temporal analysis and prediction of cellular traffic in Metropolis, IEEE Trans. Mob. Comput. 18 (9) (2018) 2190-2202.

[25] C. Zhang, S. Dang, B. Shihada, M.-S. Alouini, Dual attention-based federated learning for wireless traffic prediction, in: IEEE INFOCOM 2021-IEEE Conference on Com- puter Communications, IEEE, 2021, pp. 1-10.

[26] J. Zhang, X. Hu, Z. Cai, L. Zhu, Y. Feng, Federated learning based on mutual infor- mation clustering for wireless traffic prediction, Electronics 12 (21) (2023) 4476.

[27] L. Li, Y. Zhao, J. Wang, C. Zhang, Wireless traffic prediction based on a gradient similarity federated aggregation algorithm, Appl. Sci. 13 (6) (2023) 4036.

[28] Q. Liu, S. Sun, M. Liu, Y. Wang, B. Cao, Online spatio-temporal correlation-based fed- erated learning for traffic flow forecasting, IEEE Trans. Intell. Transp. Syst. 25 (10) (2024) 13027-13039.

[29] J. Feng, C. Du, Q. Mu, Traffic flow prediction based on federated learning and spatio- temporal graph neural networks, ISPRS Int. J. Geo-Inf. 13 (6) (2024) 210.

[30] C. Thapa, P.C.M. Arachchige, S. Camtepe, L. Sun, Splitfed: when federated learning meets split learning,in: Proceedings of the AAAI Conference on Artificial Intelli- gence, vol. 36, 2022, pp. 8485-8493.

[31] C. Meng, S. Rambhatla, Y. Liu,Cross-node federated graph neural network for spatio-temporal data modeling, in:Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021, pp. 1202-1211.

[32] Q. Zeng, Q. Sun, G. Chen, H. Duan, C. Li, G. Song, Traffic prediction of wireless cel- lular networks based on deep transfer learning and cross-domain data, IEEE Access 8 (2020) 172387-172397.

[33] M. Schirmer, M. Eltayeb, S. Lessmann, M. Rudolph, Modeling irregular time series with continuous recurrent units, in: International Conference on Machine Learning, PMLR, 2022, pp. 19388-19405.

[34] V.J. Aski, R.S. Chavan, V.S. Dhaka, G. Rani, E. Zumpano, E. Vocaturo, Forecasting of mobile network traffic and spatio-temporal analysis using modLSTM, Mach. Learn. 113 (2024) 2277-2300.

[35] S. Jaffry, S.F. Hasan, Cellular traffic prediction using recurrent neural networks, in: 2020 IEEE 5th International Symposium on Telecommunication Technologies (ISTT), IEEE, 2020, pp. 94-98.

[36] B. Gu, J. Zhan, S. Gong, W. Liu, Z. Su, M. Guizani, A spatial-temporal transformer network for city-level cellular traffic analysis and prediction, IEEE Trans. Wirel. Com- mun. 22 (12) (2023) 9412-9423.

[37] F. Li, Z. Zhang, X. Chu, J. Zhang, S. Qiu, J. Zhang, A meta-learning based framework for cell-level mobile network traffic prediction, IEEE Trans. Wirel. Commun. 22 (6) (2023) 4264-4280.

[38] S. Zhao, X. Jiang, G. Jacobson, R. Jana, W.-L. Hsu, R. Rustamov, M. Talasila, S.A. Aftab, Y. Chen, C. Borcea, Cellular network traffic prediction incorporating han- dover: a graph convolutional approach, in: 2020 17th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON), IEEE, 2020, pp. 1-9.

[39] S. Saha, A. Haque, G. Sidebottom, Analyzing the impact of outlier data points on multi-step Internet traffic prediction using deep sequence models, IEEE Trans. Netw. Serv. Manag. 20 (2) (2023) 1345-1362.

[40] Q. Zeng, Y. Liang, G. Chen, H. Duan, Q. Wu, A novel long-term noise prediction sys-tem based on 𝛼dtw-dcrnn using periodically unaligned spatiotemporal distribution sequences, IEEE Syst. J. 17 (3) (2023) 4591-4602.

[41] S.T. Nabi, M.R. Islam, M.G.R. Alam, M.M. Hassan, S.A. AlQahtani, G. Aloi, G. Fortino, Deep learning based fusion model for multivariate lte traffic forecasting and opti- mized radio parameter estimation, IEEE Access 11 (2023) 14533-14549.

[42] W. Zheng, Y. Li, M. Hong, G. Zhao, X. Fan, Network traffic matrix prediction with incomplete data via masked matrix modeling, Inf. Sci. 657 (2024) 119835.

[43] E. Candes, B. Recht, Exact matrix completion via convex optimization, Commun. ACM 55 (6) (2012) 111-119.

[44] A. Sacco, F. Esposito, G. Marchetto, Completing and predicting Internet traffic ma- trices using adversarial autoencoders and hidden Markov models, IEEE Trans. Netw. Serv. Manag. 20 (3) (2023) 2244-2258.

[45] T. Deng, M. Wan, K. Shi, L. Zhu, X. Wang, X. Jiang, Short term prediction of wireless traffic based on tensor decomposition and recurrent neural network, SN Appl. Sci. 3 (9) (2021) 779.

[46] W. Jiang, Cellular traffic prediction with machine learning: a survey, Expert Syst. Appl. 201 (2022) 117163.

[47] H. Bahuleyan, L. Mou, O. Vechtomova, P. Poupart, Variational attention for sequence-to-sequence models, preprint, arXiv:1712.08207.

[48] K. Zhang, G. Chuai, J. Zhang, X. Chen, Z. Si, S. Maimaiti, Dic-st: a hybrid prediction framework based on causal structure learning for cellular traffic and its application in urban computing, Remote Sens. 14 (6) (2022) 1439.

[49] Z.C. Lipton, D.C. Kale, R. Wetzel, et al., Modeling missing data in clinical time series with rnns, Mach. Learn. Healthc. 56 (56) (2016) 253-270.

[50] Y. Tian, K. Zhang, J. Li, X. Lin, B. Yang, Lstm-based traffic flow prediction with missing data, Neurocomputing 318 (2018) 297-305.

[51] S. Tipirneni, C.K. Reddy, Self-supervised transformer for sparse and irregularly sam- pled multivariate clinical time-series, ACM Trans. Knowl. Discov. Data 16 (6) (2022) 1-17.

[52] A. Taïk,S. Cherkaoui, Electrical load forecasting using edge computing and feder-ated learning, in: ICC 2020-2020 IEEE International Conference on Communications (ICC), IEEE, 2020, pp. 1-6.