CNN-LSTM based incremental attention mechanism enabled phase-space reconstruction for chaotic time series prediction

Xiao-Qian Lu; Jun Tian; Qiang Liao; Zheng-Wu Xu; Lu Gan

doi:10.1016/j.jnlest.2024.100256

Journal of Electronic Science and Technology ›› 2024, Vol. 22 ›› Issue (2) : 100256 DOI: 10.1016/j.jnlest.2024.100256

research-article

CNN-LSTM based incremental attention mechanism enabled phase-space reconstruction for chaotic time series prediction

Author information +

^a Bidding and Purchasing Center, University of Electronic Science and Technology of China, Chengdu 611731, China

^b Yibin Research Institute, University of Electronic Science and Technology of China, Yibin, 644000, China

^c School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China

^* E-mail addresses: xqlu@uestc.edu.cn (X.-Q. Lu),

uestctj@gmail.com (J. Tian),

ybinst@uestc.edu.cn (Q. Liao),

zwxu@uestc.edu.cn (Z.-W. Xu),

ganlu@uestc.edu.cn (L. Gan).

Xiao-Qian Lu received the M.S. degree in signal and information processing from University of Electronic Science and Technology of China (UESTC), Chengdu, China in 2007. She is working at the Bidding and Purchasing Center, UESTC. Her research interest is signal and information processing.

Jun Tian received the M.S. degree in signal and information processing from UESTC in 2022. He is an artificial intelligence algorithm engineer with ZTE's Mobile Device Division. His research interest is machine learning, especially in terminal device-side machine learning applications.

Qiang Liao received the M.S. degree in signal and information processing from Chongqing University of Posts and Telecommunications, Chongqing, China in 2022. He is working with the Yibin Research Institute of UESTC. His research interest is machine learning and deep learning.

Zheng-Wu Xu received the Ph.D. in signal and information processing from UESTC in 2014. He is a professor with the School of Information and Communication Engineering, UESTC. His research interest is signal and information processing.

Lu Gan received the M.S. and Ph.D. degrees in signal and information processing from UESTC in 2002 and 2006, respectively. He is a professor with the School of Information and Communication Engineering, UESTC. His research interests include signal classification, array signal processing, compressive sensing, and passive radar.

Show less

History +

PDF (1474KB)

Abstract

To improve the prediction accuracy of chaotic time series and reconstruct a more reasonable phase space structure of the prediction network, we propose a convolutional neural network (CNN)-long short-term memory (LSTM) prediction model based on the incremental attention mechanism. Firstly, a traversal search is conducted through the traversal layer for finite parameters in the phase space. Then, an incremental attention layer is utilized for parameter judgment based on the dimension weight criteria (DWC). The phase space parameters that best meet DWC are selected and fed into the input layer. Finally, the constructed CNN-LSTM network extracts spatiotemporal features and provides the final prediction results. The model is verified using Logistic, Lorenz, and sunspot chaotic time series, and the performance is compared from the two dimensions of prediction accuracy and network phase space structure. Additionally, the CNN-LSTM network based on incremental attention is compared with LSTM, CNN, recurrent neural network (RNN), and support vector regression (SVR) for prediction accuracy. The experiment results indicate that the proposed composite network model possesses enhanced capability in extracting temporal features and achieves higher prediction accuracy. Also, the algorithm to estimate the phase space parameter is compared with the traditional CAO, false nearest neighbor, and C–C, three typical methods for determining the chaotic phase space parameters. The experiments reveal that the phase space parameter estimation algorithm based on the incremental attention mechanism is superior in prediction accuracy compared with the traditional phase space reconstruction method in five networks, including CNN-LSTM, LSTM, CNN, RNN, and SVR.

Keywords

Chaotic time series / Incremental attention mechanism / Phase-space reconstruction

Cite this article

Download citation ▾

Xiao-Qian Lu, Jun Tian, Qiang Liao, Zheng-Wu Xu, Lu Gan. CNN-LSTM based incremental attention mechanism enabled phase-space reconstruction for chaotic time series prediction. Journal of Electronic Science and Technology, 2024, 22(2): 100256 DOI:10.1016/j.jnlest.2024.100256

登录浏览全文

4963

注册一个新账户忘记密码

Author contribution

X.-Q. Lu, J. Tian and L. Gan contributed to the conception of the study; J. Tian and Q. Liao performed the experiment; J. Tian. and X.-Q. Lu contributed significantly to analysis and manuscript preparation and wrote the manuscript; L. Gan. and Z.-W. Xu helped perform the analysis with constructive discussions. All authors have read and agreed to the published version of the manuscript.

Declaration of competing interest

The authors declared no conflicts of interest.

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	M. Han, S.-B. Feng, C.L.P. Chen, M.-L. Xu, T. Qiu, Structured manifold broad learning system: A manifold perspective for large-scale chaotic time series analysis and prediction, IEEE T. Knowl. Data En. 31 (9) (2019) 1809-1821.

[2]	Y.-N. Wang, Automatic Classification of Sunspots Using Convolutional Neural Networks, Harbin Institute of Technology, Harbin, China, (2020). M.S. Thesis.

[3]	X.-J. Li, Nonlinear Characteristics Identification and Analysis for River Runoff Time Series, Ph.D, Dissertation, Wuhan University, Wuhan, China, (2013).

[4]	J. Gao, L.-L. Jiang, Z.-Y. Xu, Chaos game representation walk model for the protein sequences, Chin. Phys. B 18 (10) (2009) 4571.

[5]	G.E.P. Box, G.M. Jenkins, G.C. Reinsel, G.M. Ljung, Time Series Analysis: Forecasting and Control (fifth ed.), John Wiley & Sons, Hoboken, USA, (2015).

[6]	W.-D. Yang, J.-Z. Wang, T. Niu, P. Du, A novel system for multi-step electricity price forecasting for electricity market management, Appl. Soft Comput. 88 (2020) 106029.

[7]	D.E. Rumelhart, G.E. Hinton, R.J. Williams, Learning representations by back-propagating errors, Nature 323 (6088) (1986) 533-536.

[8]	M. Han, J.-H. Xi, S.-G. Xu, F.-L. Yin, Prediction of chaotic time series based on the recurrent predictor neural network, IEEE T. Signal Processing 52 (12) (2004) 3409-3416.

[9]	L.J. Herrera, H. Pomares, I. Rojas, A. Guillén, A. Prieto, O. Valenzuela, Recursive prediction for long term time series forecasting using advanced models, Neurocomputing 70 (16/18) (2007) 2870-2880.

[10]	W.-J. Wang, C.-Q. Men, W.-Z. Lu, Online prediction model based on support vector machine, Neurocomputing 71 (4/6) (2008) 550-558.

[11]	M. Sangiorgio, F. Dercole, Robustness of LSTM neural networks for multi-step forecasting of chaotic time series, Chaos Soliton, Fractals 139 (2020) 110045.

[12]	S.-X. Li, X.-K. Sun, L. Yin, S.-S. Zhang, A GPS height time series prediction method based on chaos theory and LSTM, J. Navigation Positioning 8 (1) (2020) 65-73.

[13]	Y.-X. Zhang, X.-Y. Zhao, Y.-F. Yang, Chaotic characteristic analysis of daily irrigation return flow time series, in: Proc. of 8th Intl. Conf. on Intelligent Computing and Signal Processing, Xi’an, China, (2023), pp. 258-261.

[14]	J.-T. Zeng, L. Jin, Y.-J. Zhou, Sea-surface target detection based on neural network prediction, in: Proc. of IEEE 3rd Intl. Conf. on Software Engineering and Artificial Intelligence, Xiamen, China, (2023), pp. 199-203.

[15]	L.-Y. Cao, Practical method for determining the minimum embedding dimension of a scalar time series, Physica D 110 (1/2) (1997) 43-50.

[16]	M.B. Kennel, R. Brown, H.D.I. Abarbanel, Determining embedding dimension for phase-space reconstruction using a geometrical construction, Phys. Rev. A 45 (6) (1992) 3403-3411.

[17]	H.S. Kim, R. Eykholt, J.D. Salas, Nonlinear dynamics, delay times, and embedding windows, Physical D 127 (1/2) (1999) 48-60.

[18]	H.-H. Li, K. Li, J.-H. Guo, Y. Yang, Y. Zhou, NPSR: Neural network enabled phase-space reconstruction for wireless channel prediction, in: Proc. of IEEE Globecom Workshops, Rio de Janeiro, Brazil, (2022), pp. 1729-1735.

[19]	N.H. Packard, J.P. Crutchfield, J.D. Farmer, R.S. Shaw, Geometry from a time series, Phys. Rev. Lett. 45 (9) (Sept. 1980) 712-716.

[20]	F. Takens, Detecting strange attractors in turbulence, in: Proc. of Dynamical Systems and Turbulence, Warwick, UK, (1981), pp. 366-381.

[21]	S.-M. Wang, H. Shuai, Q.-S. Liu, Phase space reconstruction driven spatio-temporal feature learning for dynamic facial expression recognition, IEEE T. Affect, Computing 13 (3) (2022) 1466-1476.

[22]	V. Mnih, N. Heess, A. Graves, K. Kavukcuoglu, Recurrent models of visual attention, in: Proc. of 27th Intl. Conf. on Neural Information Processing Systems, Montreal, Canada, (2014), pp. 2204-2212.

[23]	Y. LeCun, B. Boser, J.S. Denker, et al., Backpropagation applied to handwritten zip code recognition, Neural Comput. 1 (4) (1989) 541-551.

[24]	W.-J. Huang, Y.-T. Li, Y. Huang, Prediction of chaotic time series using hybrid neural network and attention mechanism, Acta Phys. Sin. 70 (1) (2021) 1-9. 010501.

[25]	X.-L. Ma, Z.-M. Tao, Y.-H. Wang, H.-Y. Yu, Y.-P. Wang, Long short-term memory neural network for traffic speed prediction using remote microwave sensor data, Transport, Res. C: Emer. 54 (2015) 187-197.

[26]	SILSO, World data center-sunspot number and long-term solar observations, royal observatory of Belgium, online sunspot number catalogue [Online]. Available, https://www.sidc.be/sunspot-data/SIDCpub.php, May 2024.