Multi-scale persistent spatiotemporal transformer for long-term urban traffic flow prediction

Abstract

Long-term urban traffic flow prediction is an important task in the field of intelligent transportation, as it can help optimize traffic management and improve travel efficiency. To improve prediction accuracy, a crucial issue is how to model spatiotemporal dependency in urban traffic data. In recent years, many studies have adopted spatiotemporal neural networks to extract key information from traffic data. However, most models ignore the semantic spatial similarity between long-distance areas when mining spatial dependency. They also ignore the impact of predicted time steps on the next unpredicted time step for making long-term predictions. Moreover, these models lack a comprehensive data embedding process to represent complex spatiotemporal dependency. This paper proposes a multi-scale persistent spatiotemporal transformer (MSPSTT) model to perform accurate long-term traffic flow prediction in cities. MSPSTT adopts an encoder-decoder structure and incorporates temporal, periodic, and spatial features to fully embed urban traffic data to address these issues. The model consists of a spatiotemporal encoder and a spatiotemporal decoder, which rely on temporal, geospatial, and semantic space multi-head attention modules to dynamically extract temporal, geospatial, and semantic characteristics. The spatiotemporal decoder combines the context information provided by the encoder, integrates the predicted time step information, and is iteratively updated to learn the correlation between different time steps in the broader time range to improve the model's accuracy for long-term prediction. Experiments on four public transportation datasets demonstrate that MSPSTT outperforms the existing models by up to 9.5% on three common metrics.

Keywords

Graph neural network / Multi-head attention mechanism / Spatio-temporal dependency / Traffic flow prediction / Graph neural network / Multi-head attention mechanism / Spatio-temporal dependency / Traffic flow prediction

References

[1]	J.-W. Jiang, C.-K. Han, W.X. Zhao, J.-Y. Wang. PDFormer: Propagation delay-aware dynamic long-range transformer for traffic flow prediction. Proc. of 37th AAAI Conf. on Artificial Intelligence, Washington (2023), pp. 4365-4373.
[2]	S.-N. Guo, Y.-F. Lin, H.-Y. Wan, X.-C. Li, G. Cong. Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting. IEEE T. Knowl. Data Eng., 34 (11) (Nov.2021), pp. 5415-5428.
[3]	J.-W. Zhu, X. Han, H.-H. Deng, et al. KST-GCN: A knowledge-driven spatial-temporal graph convolutional network for traffic forecasting. IEEE T. Intell. Transp., 23 (9) (Sept.2020), pp. 15055-15065.
[4]	M.-Z. Li, Z.-X. Zhu. Spatial-temporal fusion graph neural networks for traffic flow forecasting. Proc. of the 35th AAAI Conf. on Artificial Intelligence (2020), pp. 4189-4196.
[5]	L. Yu, B.-W. Du, X. Hu, L.-L. Sun, L.-Z. Han, W.-F. Lv. Deep spatio-temporal graph convolutional network for traffic accident prediction. Neurocomputing, 423 (Jan.2021), pp. 135-147. View articleGoogle Scholar.
[6]	J.-C. Ye, L.-L. Sun, B.-W. Du, Y.-J. Fu, H. Xiong. Coupled layer-wise graph convolution for transportation demand prediction. Proc. of the 35th AAAI Conf. on Artificial Intelligence (2020), pp. 4617-4625.
[7]	B. Pu, J.-S. Liu, Y. Kang, J.-G. Chen, P.S. Yu. MVSTT: A multiview spatial-temporal transformer network for traffic-flow forecasting. IEEE T. Cybern., 54 (3) (Mar.2024), pp. 1582-1595.
[8]	C. Song, Y.-F. Lin, S.-N. Guo, H.-Y. Wan. Spatial-temporal synchronous graph convolutional networks: a new framework for spatial-temporal network data forecasting. Proc. of the 34th AAAI Conf. on Artificial Intelligence, New York (2020), pp. 914-921.
[9]	M.-X. Xu, W.-R. Dai, C.-M. Liu, et al. Spatial-temporal transformer networks for traffic flow forecasting. (2021).
	[Online]. Available. https://arxiv.org/abs/2001.02908.Online]. Available. https://arxiv.org/abs/2001.02908. March 2021.
[10]	X.-Y. Wang, Y. Ma, Y.-Q. Wang, et al. Traffic flow prediction via spatial temporal graph neural network. Proc. of the Web Conf.2020, Taipei, China (2020), pp. 1082-1092.
[11]	B.M. Williams, L.A. Hoel. Modeling and forecasting vehicular traffic flow as a seasonal ARIMA process: theoretical basis and empirical results. J. Transport. Eng., 129 (6) (Nov.2003), pp. 664-672.
[12]	L. Zheng, C. Zhou, J. Wu, H. Jiang, S.-Y. Cui. Integrating granger causality and vector auto-regression for traffic prediction of large-scale WLANs. KSII T. Internet Inf., 10 (1) (Jan.2016), pp. 136-151.
[13]	C.H. Wu, J.M. Ho, D.T. Lee. Travel-time prediction with support vector regression. IEEE T. Intell. Transp., 5 (4) (Dec.2004), pp. 276-281.
[14]	H.V. Lint, C.V. Hinsbergen.Short-term traffic and travel time prediction models. Transportation Research Circular E-C168(October 2012), pp. 22-41.
[15]	J.-B. Zhang, Y. Zheng, D.-K. Qi. Deep spatio-temporal residual networks for citywide crowd flows prediction. Proc. of the 31st AAAI Conf. on Artificial Intelligence, San Francisco (2017), pp. 1655-1661.
[16]	H.-X. Yao, F. Wu, J.-T. Ke, et al. Deep multi-view spatial-temporal network for taxi demand prediction. Proc. of the AAAI Conf. on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conf. and Eighth AAAI Symp. on Educational Advances in Artificial, New Orleans (2018), pp. 316:1-8.
[17]	W.-W. Jiang. Graph-based deep learning for communication networks: A survey. Comput. Commun., 185 (March2022), pp. 40-54. View articleGoogle Scholar.
[18]	Z.-H. Wu, S.-R. Pan, F.-W. Chen, et al. A comprehensive survey on graph neural networks. IEEE T. Neural Networks and Learning Systems, 32 (1) (Jan.2021), pp. 4-24.
[19]	Y.-G.Li, R Yu, S. Cyrus. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting.
	[Online]. Available. https://arxiv.org/abs/1707.01926 (Feb.2018).
[20]	B. Yu, H.-T. Yin, Z.-X. Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. Proc. of the 27th Int. Joint Conf. on Artificial Intelligence, Stockholm (2018), pp. 3634-3640.
[21]	S.-N. Guo, Y.-F. Lin, N. Feng, C. Song, H.-Y. Wan. Wan, Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. Proc. of the 33th AAAI Conf. on Artificial Intelligence, Honolulu (2019), pp. 922-929.
[22]	Z.-H. Lin, M.-M. Li, Z.-B. Zheng, Y.-Y. Cheng, C. Yuan. Self-attention ConvLSTM for spatiotemporal prediction. Proc. of the 34th AAAI Conf. on Artificial Intelligence, the 32nd Innovative Applications of Artificial Intelligence Conf., the 10th AAAI Symp. on Educational Advances in Artificial Intelligence, New York (2020), pp. 11531-11538.
[23]	X. Ye, S. Fang, F. Sun, C.-X. Zhang, S.-M. Xiang. Meta graph transformer: a novel framework for spatial-temporal traffic prediction. Neurocomputing, 491 (Jun.2022), pp. 544-563. View articleGoogle Scholar.
[24]	A. Vaswani, N.M. Shazeer, N. Parmar, et al. Attention is all you need. Proc. of the 31st Int. Conf. on Neural Information Processing Systems, Long Beach (2017), pp. 6000-6010.
[25]	L. Cai, K. Janowicz, G.-C. Mai, B. Yan, R. Zhu. Traffic transformer: Capturing the continuity and periodicity of time series for traffic forecasting. T. GIS, 24 (3) (June2020), pp. 736-755.
[26]	G.-Y. Jin, Z.-X. Xi, H.-Y. Sha, Y.-H. Feng, J.-C. Huang. Deep multi-view graph-based network for citywide ride-hailing demand prediction. Neurocomputing, 510 (Oct.2022), pp. 79-94. View articleGoogle Scholar.
[27]	Y. Li, José M.F.Moura. Forecaster: a graph transformer for forecasting spatial and time-dependent data.
	[Online].Available. https://arxiv.org/abs/1909.04019 (February2020).
[28]	M. Belkin, P. Niyogi.Laplacian eigenmaps and spectral techniques for embedding and clustering. Proc. of the 14th Int. Conf. on Neural Information Processing Systems: Natural and Synthetic, Vancouver(2001), pp. 585-591.
[29]	D.J. Berndt, J. Clifford.Using dynamic time warping to find patterns in time series. Proc. of the 3rd Int. Conf. on Knowledge Discovery and Data Mining, Seattle(1994), pp. 359-370.
[30]	L.-B. Liu, J.-J. Zhen, G.-B. Li, et al. Dynamic spatial-temporal representation learning for traffic flow prediction. IEEE T. Intell. Transp., 22 (11) (Nov.2021), pp. 7169-7183.
[31]	J.-Y. Wang, J.-W. Jiang, W.-J. Jiang, C. Li, W.X. Zhao. LibCity: an open library for traffic prediction. Proc. of the 29th Int. Conf. on Advances in Geographic Information Systems, Beijing (2021), pp. 145-148.
[32]	S. Hochreiter, J. Schmidhuber. Long short-term memory. Neural Comput., 9 (8) (Nov.1997), pp. 1735-1780.
[33]	Z.-H. Wu, S.-R. Pan, G.-D. Long, J. Jiang. Graph WaveNet for deep spatial-temporal graph modeling. Proc. of the 28th Int. Joint Conf. on Artificial Intelligence, Macao, China(2019), pp. 1907-1913.