Attention based simplified deep residual network for citywide crowd flows prediction

Abstract

Crowd flows prediction is an important problem of urban computing whose goal is to predict the number of incoming and outgoing people of regions in the future. In practice, emergency applications often require less training time. However, there is a little work on how to obtain good prediction performance with less training time. In this paper, we propose a simplified deep residual network for our problem. By using the simplified deep residual network, we can obtain not only less training time but also competitive prediction performance compared with the existing similar method. Moreover, we adopt the spatio-temporal attention mechanism to further improve the simplified deep residual network with reasonable additional time cost. Based on the real datasets, we construct a series of experiments compared with the existing methods. The experimental results confirm the efficiency of our proposed methods.

Keywords

crowd flows prediction / spatio-temporal data mining / attention

References

[1] Zheng Y, Capra L, Wolfson O, Yang H. Urban computing: concepts, methodologies, and applications. ACM Transactions on Intelligent Systems and Technology, 2014, 5(3): 38

[2] Zhang J, Zheng Y, Qi D. Deep spatio-temporal residual networks for citywide crowd flows prediction. In: Proceedings of AAAI Conference on Artificial Intelligence. 2017, 1655–1661

[3] Wang L, Geng X, Ma X, Liu F, Yang Q. Crowd flow prediction by deep spatio-temporal transfer learning. 2018, arXiv preprint arXiv:1802.00386

[4] Wu C, Yin T, Ge S, Yu K. Ensemble learning for crowd flows prediction on campus. In: Proceedings of International Conference on Smart Computing and Communication. 2017, 103–113

[5] Zhang J, Zheng Y, Qi D, Li R, Yi X. DNN-based prediction model for spatio-temporal data. In: Proceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 2016

[6] Song X, Zhang Q, Sekimoto Y, Shibasaki R. Prediction of human emergency behavior and their mobility following large-scale disaster. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2014, 5–14

[7] Fan Z, Song X, Shibasaki R, Adachi R. Citymomentum: an online approach for crowd behavior prediction at a citywide level. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 2015, 559–569

[8] Silva R, Kang S M, Airoldi E M. Predicting traffic volumes and estimating the effects of shocks in massive transportation systems. Proceedings of the National Academy of Sciences, 2015, 112(18): 5643–5648

[9] Xu Y, Kong Q J, Klette R, Liu Y. Accurate and interpretable bayesian mars for traffic flow prediction. IEEE Transactions on Intelligent Transportation Systems, 2014, 15(6): 2457–2469

[10] Bao J, He T, Ruan S, Li Y, Zheng Y. Planning bike lanes based on sharingbikes’ trajectories. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2017, 1377–1386

[11] Li Y, Zheng Y, Zhang H, Chen L. Traffic prediction in a bike-sharing system. In: Proceedings of the 23rd SIGSPATIAL International Conference on Advances in Geographic Information Systems. 2015

[12] Kong X, Xu Z, Shen G, Wang J, Yang Q, Zhang B. Urban traffic congestion estimation and prediction based on floating car trajectory data. Future Generation Computer Systems, 2016, 61: 97–107

[13] Zheng Y, Yi X, Li M, Li R, Shan Z, Chang E, Li T. Forecasting finegrained air quality based on big data. In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2015, 2267–2276

[14] Hamed M M, Al-Masaeid H R, Said Z M B. Short-term prediction of traffic volume in urban arterials. Journal of Transportation Engineering, 1995, 121(3): 249–254

[15] Ding Q Y, Wang X F, Zhang X Y, Sun Z Q. Forecasting traffic volume with space-time arima model. Advanced Materials Research, 2011, 156: 979–983

[16] Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 1994, 5(2): 157–166

[17] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 1997, 9(8): 1735–1780

[18] Yu B, Yin H, Zhu Z. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In: Proceedings of International Joint Conferences on Artificial Intelligence. 2018, 3634–3640

[19] LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11): 2278–2324

[20] Shi X, Chen Z, Wang H, Yeung D Y, Wong W K, Woo W C. Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Proceedings of Advances in Neural Information Processing Systems. 2015, 802–810

[21] Xiong F, Shi X, Yeung D Y. Spatiotemporal modeling for crowd counting in videos. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 5151–5159

[22] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770–778

[23] Chen L, Zhang H, Xiao J, Nie L, Shao J, Liu W, Chua T S. SCA-CNN: spatial and channel-wise attention in convolutional networks for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 5659–5667

[24] Lu J, Xiong C, Parikh D, Socher R. Knowing when to look: adaptive attention via a visual sentinel for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 375–383

[25] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proceedings of Advances in Neural Information Processing Systems. 2017, 5998–6008

[26] Bahdanau D, Chorowski J, Serdyuk D, Brakel P, Bengio Y. End-to-end attention-based large vocabulary speech recognition. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2016, 4945–4949

[27] Chorowski J K, Bahdanau D, Serdyuk D, Cho K, Bengio Y. Attentionbased models for speech recognition. In: Proceedings of Advances in Neural Information Processing Systems. 2015, 577–585

[28] Zhou X, Shen Y, Zhu Y, Huang L. Predicting multi-step citywide passenger demands using attention-based neural networks. In: Proceedings of the 11th ACM International Conference onWeb Search and Data Mining. 2018, 736–744

[29] Geng X, Li Y, Wang L, Zhang L, Yang Q, Ye J, Liu Y. Spatiotemporal multi-graph convolution network for ride-hailing demand forecasting. In: Proceedings of AAAI Conference on Artificial Intelligence. 2019

[30] Veliˇckovíc P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y. Graph attention networks. In: Proceedings of International Conference on Learning Representations. 2018

[31] Liang Y, Ke S, Zhang J, Yi X, Zheng Y. Geoman: multi-level attention networks for geo-sensory time series prediction. In: Proceedings of International Joint Conferences on Artificial Intelligence. 2018, 3428–3434

[32] Liu L, Zhang R, Peng J, Li G, Du B, Lin L. Attentive crowd flow machines. In: Proceedings of the 26th ACM International Conference on Multimedia. 2018, 1553–1561

[33] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks. In: Proceedings of Advances in Neural Information Processing Systems. 2012, 1097–1105

[34] Hu X, Dai G, Ge Y, Ning Z, Liu Y. A simplified deep residual network for citywide crowd flows prediction. In: Proceedings of the International Conference on Semantics, Knowledge and Grids. 2019, 60–67

[35] Kingma D P, Ba J. Adam: a method for stochastic optimization. 2014, arXiv preprint arXiv:1412.6980

[36] Wu S, Tang Y, Zhu Y, Wang L, Xie X, Tan T. Session-based recommendation with graph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2019, 346–353

[37] Box G E P, Jenkins G M, Reinsel G C, Ljung G M. Time series analysis: forecasting and control. Journal of the Operational Research Society, 2015, 22(2): 199–201

[38] Williams B M, Durvasula P K, Brown D E. Urban freeway traffic flow prediction: application of seasonal autoregressive integrated moving average and exponential smoothing models. Transportation Research Record, 1998, 1644(1): 132–141

[39] Lütkepohl H. Vector Autoregressive Models. Cheltenham: Edward Elgar Publishing, 2013

[40] Hoang M X, Zheng Y, Singh A K. FCCF: forecasting citywide crowd flows based on big data. In: Proceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 2016

RIGHTS & PERMISSIONS

Higher Education Press