AFSTGCN: Prediction for multivariate time series using an adaptive fused spatial-temporal graph convolutional network

Abstract

The prediction for Multivariate Time Series (MTS) explores the interrelationships among variables at historical moments, extracts their relevant characteristics, and is widely used in finance, weather, complex industries and other fields. Furthermore, it is important to construct a digital twin system. However, existing methods do not take full advantage of the potential properties of variables, which results in poor predicted accuracy. In this paper, we propose the Adaptive Fused Spatial-Temporal Graph Convolutional Network (AFSTGCN). First, to address the problem of the unknown spatial-temporal structure, we construct the Adaptive Fused Spatial-Temporal Graph (AFSTG) layer. Specifically, we fuse the spatial-temporal graph based on the interrelationship of spatial graphs. Simultaneously, we construct the adaptive adjacency matrix of the spatial-temporal graph using node embedding methods. Subsequently, to overcome the insufficient extraction of disordered correlation features, we construct the Adaptive Fused Spatial-Temporal Graph Convolutional (AFSTGC) module. The module forces the reordering of disordered temporal, spatial and spatial-temporal dependencies into rule-like data. AFSTGCN dynamically and synchronously acquires potential temporal, spatial and spatial-temporal correlations, thereby fully extracting rich hierarchical feature information to enhance the predicted accuracy. Experiments on different types of MTS datasets demonstrate that the model achieves state-of-the-art single-step and multi-step performance compared with eight other deep learning models.

Keywords

Adaptive adjacency matrix / Digital twin / Graph convolutional network / Multivariate time series prediction / Spatial-temporal graph

References

[1] M. Längkvist, L. Karlsson, A. Loutfi, A review of unsupervised feature learning and deep learning for time-series modeling, Pattern Recogn. Lett. 42 (1) (2014) 11-24.

[2] H. Yin, H. Qi, J. Xu, W.N. Hung, X. Song, Generalized framework for similarity measure of time series, Math. Probl Eng. 2014 (16) (2014) 572124.

[3] S.Y. Shih, F.K. Sun, H.Y. Lee, Temporal pattern attention for multivariate time series forecasting, Mach. Learn. 108 (8-9) (2019) 1421-1441.

[4] Y. Qin, D. Song, H. Cheng, W. Cheng, G. Jiang, G.W. Cottrell,A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction, IJCAI’17: Proceedings of the 26th International Joint Conference on Artificial Intelligence, 2017, pp. 2627-2633.

[5] Y. Xiao, H. Yin, T. Duan, H. Qi, Y. Zhang, A. Jolfaei, K. Xia, An Intelligent prediction model for UCG state based on dual-source LSTM, Int. J. Mach. Learn. & Cyber. 12 (2021) 3169-3178.

[6] Y. Liu, C. Gong, L. Yang, Y. Chen, DSTP-RNN: A dual-stage two-phase attention-based recurrent neural network for long-term and multivariate time series prediction, Expert Syst. Appl. 143 (2020) 113082.

[7] H. Liu, G. Hua, H. Yin, Y. Xu, An intelligent Grey Wolf optimizer algorithm for distributed compressed sensing, Comput. Intell. Neurosci. 143 (2018) 1723191.

[8] Y. Xiao, J. Yin, Y. Hu, J. Wang, H. Yin, H. Qi, Monitoring and control in underground coal gasification: current research status and future perspective, Sustainability 11 (1) (2019) 217.

[9] J. Daihong, Z. Sai, D. Lei, D. Yueming, Multi-scale generative adversarial network for image super-resolution, Soft Comput. 26 (8) (2022) 3631-3641.

[10] F. Tao, J. Cheng, Q. Qi, M. Zhang, H. Zhang, F. Sui, Digital twin-driven product design, manufacturing and service with big data, Int. J. Adv. Manuf. Technol. 94 (9)(2018) 3563-3576.

[11] J. Guo, Z. Lv, Application of digital twins in multiple fields, Multimed. Tool. Appl. 81 (19) (2022) 26941-26967.

[12] W. Hu, Y. He, Z. Liu, J. Tan, M. Yang, J. Chen, Toward a digital twin: time series prediction based on a hybrid ensemble empirical mode decomposition and bo-lstm neural networks, J. Mech. Des. 143 (5) (2021) 051705.

[13] Y. Li, Z. Zhu, D. Kong, H. Han, Y. Zhao, EA-LSTM: evolutionary attention-based LSTM for time series prediction, Knowl. Base Syst. 181 (2019) 104785.

[14] X. Wu, B. Shi, Y. Dong, C. Huang, L. Faust, N.V. Chawla, Restful: resolution-aware forecasting of behavioral time series data, international conference on information and knowledge management, Proceedings of the 27th ACM International Conference on Information and Knowledge Management, ACM (2018) 1073-1082.

[15] P. Esling, C. Agon, Time-series data mining, ACM Comput. Surv. 45 (1) (2012) 1-34.

[16] T. Lin, B.G. Horne, C.L. Giles, How embedded memory in recurrent neural network architectures helps learning long-term temporal dependencies, Neural Network. 11 (5) (1998) 861-868.

[17] Y. Xiao, H. Yin, Y. Zhang, H. Qi, Y. Zhang, Z. Liu, A dual-stage attention-based Conv-LSTM network for spatio-temporal correlation and multivariate time series prediction, Int. J. Intell. Syst. 36 (5) (2021) 2036-2057.

[18] G. Lai, W.C. Chang, Y. Yang, H. Liu, Modeling long- and short-term temporal patterns with deep neural networks, 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018 (2018) 95-104.

[19] H. Dai, E.B. Khalil, Y. Zhang, B. Dilkina, L. Song, Learning combinatorial optimization algorithms over graphs, in: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS, 2017, pp. 6351-6361.

[20] T.N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, 5th international conference on learning representations, ICLR 2017 -Conf. Track Proc. (2017) 1-14.

[21] N.K. Ahmed, R.A. Rossi, R. Zhou, J.B. Lee, X. Kong, T. Lwillke, H. Eldardiry, Inductive Representation Learning in Large Attributed Graphs, arXiv: Machine Learning, 2017, pp. 1-11.

[22] Z. Wu, S. Pan, G. Long, J. Jiang, X. Chang, C. Zhang, Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM, 2020, pp. 753-763.

[23] Y. Li, R. Yu, C. Shahabi, Y. Liu, Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting, International Conference on Learning Representations, 2017, pp. 1-16.

[24] J. Zhou, G. Cui, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, M. Sun,Graph Neural Networks: A Review of Methods and Applications 1, AI Open, 2018, pp. 51-87.

[25] M. Xu, W. Dai, C. Liu, X. Gao, W. Lin, G.J. Qi, H. Xiong, Spatial-temporal transformer networks for traffic flow forecasting, CoRR abs/2001. 02908 (2020).

[26] B. Yu, H. Yin, Z. Zhu, Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting,in:Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI, 2018, pp. 3634-3640.

[27] L. Zhao, Y. Song, C. Zhang, Y. Liu, P. Wang, T. Lin, M. Deng, H. Li, T-GCN: A Temporal Graph Convolutional Network for Traffic Prediction, 21 (9), IEEE Transactions on Intelligent Transportation Systems, 2020, pp. 3848-3858.

[28] X. Wang, Y. Ma, Y. Wang, W. Jin, X. Wang, J. Tang, C. Jia, J. Yu,Traffic flow prediction via spatial temporal graph neural network, in:The Web Conference 2020 - Proceedings of the World Wide Web Conference, WWW 2020, 2020, pp. 1082-1092.

[29] Z. Wu, S. Pan, G. Long, J. Jiang, C. Zhang, Graph Wavenet for Deep Spatialoral Graph Modeling. Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI, 2019, pp. 1907-1913.

[30] L. Bai, L. Yao, C. Li, X. Wang, C. Wang, Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting, Advances in neural information processing systems 33 (2020) 17804-17815.

[31] K. Guo, Y. Hu, Z. Qian, Y. Sun, J. Gao, B. Yin, Dynamic graph convolution network for traffic forecasting based on latent network of Laplace matrix estimation, IEEE Trans. Intell. Transport. Syst. 23 (2) (2022) 1009-1018.

[32] C. Song, Y. Lin, S. Guo, H. Wan, Spatial-temporal synchronous graph convolutional networks: a new framework for spatial-temporal network data forecasting, Proc. AAAI Conf. Artif. Intell. 34 (01) (2020) 914-921.

[33] M. Li, Z. Zhu, Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting. AAAI Conference on Artificial Intelligence 5th, 35, AAAI, 2021, pp. 4189-4196.

[34] Y. Hechtlinger, P. Chakravarti, J. Qin, A Generalization of Convolutional Neural Networks to Graph-Structured Data, arXiv: Machine Learning, 2017, pp. 1-14.

[35] F. Zamora-Martínez, P. Romeu, P. Botella-Rocamora, J. Pardo, On-line learning of indoor temperature forecasting models towards energy efficiency, Energy Build. 83 (2014) 162-172.

[36] S. Bai, J.Z. Kolter, V. Koltun, An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling, 2018 arXiv preprint arXiv: 1803.01271.

[37] J. Becerra-Rico, M.A. Aceves-Fernández, K. Esquivel-Escalante, J.C. Pedraza-Ortega, Airborne particle pollution predictive model using Gated Recurrent Unit (GRU) deep neural networks, Earth Sci. Inf. 13 (3) (2020) 821-834.

[38] A. Gu, C. Gulcehre, T. Paine, M. Hoffman, R. Pascanu, Improving the gating mechanism of recurrent neural networks, in: International Conference on Machine Learning, PMLR, 2020, pp. 3800-3809.