Effective ensemble learning approach for SST field prediction using attention-based PredRNN

Abstract

Accurate prediction of sea surface temperature (SST) is extremely important for forecasting oceanic environmental events and for ocean studies. However, the existing SST prediction methods do not consider the seasonal periodicity and abnormal fluctuation characteristics of SST or the importance of historical SST data from different times; thus, these methods suffer from low prediction accuracy. To solve this problem, we comprehensively consider the effects of seasonal periodicity and abnormal fluctuation characteristics of SST data, as well as the influence of historical data in different periods, on prediction accuracy. We propose a novel ensemble learning approach that combines the Predictive Recurrent Neural Network(PredRNN) network and an attention mechanism for effective SST field prediction. In this approach, the XGBoost model is used to learn the long-period fluctuation law of SST and to extract seasonal periodic features from SST data. The exponential smoothing method is used to mitigate the impact of severely abnormal SST fluctuations and extract the a priori features of SST data. The outputs of the two aforementioned models and the original SST data are stacked and used as inputs for the next model, the PredRNN network. PredRNN is the most recently developed spatiotemporal deep learning network, which simulates both spatial and temporal representations and is capable of transferring memory across layers and time steps. Therefore, we used it to extract the spatiotemporal correlations of SST data and predict future SSTs. Finally, an attention mechanism is added to capture the importance of different historical SST data, weigh the output of each step of the PredRNN network, and improve the prediction accuracy. The experimental results on two ocean datasets confirm that the proposed approach achieves higher training efficiency and prediction accuracy than the existing SST field prediction approaches do.

Graphical abstract

Keywords

SST prediction / ensemble learning / XGBoost / PredRNN / attention mechanism

References

[1] Wentz F J , Gentemann C , Smith D , Chelton D . Satellite measurements of sea surface temperature through clouds. Science, 2000, 288( 5467): 847– 850

[2] Solanki H U , Bhatpuria D , Chauhan P . Integrative analysis of AltiKa-SSHa, MODIS-SST, and OCM-chlorophyll signatures for fisheries applications. Marine Geodesy, 2015, 38( S1): 672– 683

[3] Funk C C , Hoell A . The leading mode of observed and CMIP5 ENSO-residual sea surface temperatures and associated changes in Indo-Pacific climate. Journal of Climate, 2015, 28( 11): 4309– 4329

[4] Aparna S G , D’souza S , Arjun N B . Prediction of daily sea surface temperature using artificial neural networks. International Journal of Remote Sensing, 2018, 39( 12): 4214– 4231

[5] Liu Y , Fu W . Assimilating high-resolution sea surface temperature data improves the ocean forecast potential in the Baltic Sea. Ocean Science, 2018, 14( 3): 525– 541

[6] Stockdale T N , Balmaseda M A , Vidard A . Tropical Atlantic SST prediction with coupled ocean–atmosphere GCMs. Journal of Climate, 2006, 19( 23): 6047– 6061

[7] Xue Y , Leetmaa A . Forecasts of tropical Pacific SST and sea level using a Markov model. Geophysical Research Letters, 2000, 27( 17): 2701– 2704

[8] Lins I D , Araujo M , das Chagas Moura M . Prediction of sea surface temperature in the tropical Atlantic by support vector machines. Computational Statistics & Data Analysis, 2013, 61 : 187– 198

[9] Patil K , Deo M C , Ravichandran M . Prediction of sea surface temperature by combining numerical and neural techniques. Journal of Atmospheric and Oceanic Technology, 2016, 33( 8): 1715– 1726

[10] He Q , Zha C , Sun M , Jiang X Y , Qi F M , Huang D M , Song W . Surface temperature parallel prediction algorithm under Spark platform. Marine Science Bulletin, 2019, 38( 3): 280– 289

[11] LeCun Y , Bengio Y , Hinton G . Deep learning. Nature, 2015, 521( 7553): 436– 444

[12] Zhang Q , Wang H , Dong J , Zhong G , Sun X . Prediction of sea surface temperature using long short-term memory. IEEE Geoscience and Remote Sensing Letters, 2017, 14( 10): 1745– 1749

[13] Yang Y , Dong J , Sun X , Lima E , Mu Q , Wang X . A CFCC-LSTM model for sea surface temperature prediction. IEEE Geoscience and Remote Sensing Letters, 2018, 15( 2): 207– 211

[14] Xiao C , Chen N , Hu C , Wang K , Xu Z , Cai Y P , Xu L , Chen Z , Gong J . A spatiotemporal deep learning model for sea surface temperature field prediction using time-series satellite data. Environmental Modelling & Software, 2019, 120 : 104502–

[15] Wei L , Guan L , Qu L . Prediction of sea surface temperature in the South China Sea by artificial neural networks. IEEE Geoscience and Remote Sensing Letters, 2020, 17( 4): 558– 562

[16] Wang Y, Long M, Wang J. PredRNN: recurrent neural networks for predictive learning using spatiotemporal LSTMs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 879– 888

[17] Shi X, Chen Z, Wang H, Yeung D Y, Wong W K, Woo W C. Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Proceedings of the 28th International Conference on Neural Information Processing Systems. 2015, 802– 810

[18] Chen T, Guestrin C. XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016, 785– 794

[19] Friedman J H . Greedy function approximation: a gradient boosting machine. The Annals of Statistics, 2001, 29( 5): 1189– 1232

[20] Feng J, Yu Y, Zhou Z-H. Multi-layered gradient boosting decision trees. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 3555−3565

[21] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000−6010

[22] Xie J , Zhang J , Yu J , Xu L . An adaptive scale sea surface temperature predicting method based on deep learning with attention mechanism. IEEE Geoscience and Remote Sensing Letters, 2020, 17( 5): 740– 744

[23] Shi X, Yeung D-Y. Machine learning for spatiotemporal sequence forecasting: a survey. 2018, arXiv preprint arXiv: 1808.06865

RIGHTS & PERMISSIONS

Higher Education Press 2021