An Ensemble Model of Arima and Ann with Restricted Boltzmann Machine Based on Decomposition of Discrete Wavelet Transform for Time Series Forecasting

Abstract

Time series forecasting research area mainly focuses on developing effective forecasting models to improve prediction accuracy. An ensemble model composed of autoregressive integrated moving average (ARIMA), artificial neural network (ANN), restricted Boltzmann machines (RBM), and discrete wavelet transform (DWT) is presented in this paper. In the proposed model, DWT first decomposes time series into approximation and detail. Then Khashei and Bijari’s model, which is an ensemble model of ARIMA and ANN, is applied to the approximation and detail to extract their both linear and nonlinear components and fit the relationship between the components as a function instead of additive relationship. Furthermore, RBM is used to perform pre-training for generating initial weights and biases based on inputs feature for ANN. Finally, the forecasted approximation and detail are combined to obtain final forecasting. The forecasting capability of the proposed model is tested with three well-known time series: sunspot, Canadian lynx, exchange rate time series. The prediction performance is compared to the other six forecasting models. The results indicate that the proposed model gives the best performance in all three data sets and all three measures (i.e. MSE, MAE and MAPE).

Keywords

Time series forecasting / autoregressive integrated moving average (ARIMA) / artificial neural network (ANN) / discrete wavelet transform (DWT) / restricted Boltzmann machine (RBM)

References

[1] Ackley D., E H., Sejnowski T.. A learning algorithm for Boltzmann machines. Cognitive Science, 1985, 9: 147-169.

[2] Adamowski J., Chan H. F.. A wavelet neural network conjunction model for groundwater level forecasting. Journal of Hydrology, 2011, 407(1): 28-40.

[3] Adamowski J., Sun K.. Development of a coupled wavelet transform and neural network method for flow forecasting of non-perennial rivers in semi-arid watersheds. Journal of Hydrology, 2010, 390(1): 85-91.

[4] Campbell M., Walker A.. A survey of statistical work on the Mackenzie River series of annual Canadian lynx trappings for the years 1821–1934 and a new analysis. Journal of the Royal Statistical Society. Series A (general), 1977, 140(4): 411-431.

[5] Chen J., Tang X.. Ensemble of multiple kNN classifiers for societal risk classification. Journal of Systems Science and Systems Engineering, 2017, 26(4): 433-447.

[6] Cogranne R., Fridrich J.. Modeling and extending the ensemble classifier for steganalysis of digital images using hypothesis testing theory. IEEE Transactions on Information Forensics and Security, 2015, 10(12): 2627-2642.

[7] Conejo A.J., Plazas M.A., Espinola R., Molina A.B.. Day-ahead electricity price forecasting using the wavelet transform and ARIMA models. IEEE Transactions on Power Systems, 2005, 20(2): 1035-1042.

[8] Dayhoff J.A.. Neural Network Architectures: An Introduction, 1995, Cambridge: MIT press

[9] De Gooijer J.G., Hyndman R.J.. 25 Years of time series forecasting. International Journal of Forecasting, 2006, 22(3): 443-473.

[10] Fard A.K., Akbari-Zadeh M.R.. A hybrid method based on wavelet, ANN and ARIMA model for short-term load forecasting. Journal of Experimental & Theoretical Articial Intelligence, 2014, 26(2): 167-182.

[11] Fliege N. J.. Multirate Digital Signal Processing: Multirate Systems, Filter Banks, Wavelets, 1994, Chichester: John Willy & Sons, Inc..

[12] Gao W., Tian Z.. Learning Granger causality graphs for multivariate nonlinear time series. Journal of Systems Science and Systems Engineering, 2009, 18(1): 38-52.

[13] Hecht-Nielsen R.. Theory of the backpropagation neural network. International Joint Conference on Neural Networks, 1989 593-605.

[14] Hinton G.E., Sejnowski T.J.. Rumelhart D.E., McClelland J.L.. Learning and relearning in Boltzmann machines. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, 1986, Cambridge, MA: MIT Press 282-317.

[15] Huang N.E., Shen Z., Long S.R., Wu M.C., Shih H.H., Zheng Q., Yen N.-C., Tung C.C., Liu H.H.. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. In: Proceedings of the Royal Society of London A: Mathematical, Physical And Engineering Sciences, 1998, 454(1971): 903-995.

[16] Kennedy J., Eberhart R.C.. Particle swarm optimization. Proceedings of the IEEE International Conference on Neural Networks, 1995 1942-1948.

[17] Khandelwal I., Adhikari R., Verma G.. Time series forecasting using hybrid ARIMA and ANN models based on DWT decomposition. Procedia Computer Science, 2015, 48: 173-179.

[18] Khashei M., Bijari M.. A novel hybridization of articial neural networks and ARIMA models for time series forecasting. Applied Soft Computing, 2011, 11(2): 2664-2675.

[19] Lin T., Pourahmadi M.. Nonparametric and non-linear models and data mining in time series: a case-study on the Canadian lynx data. Journal of the Royal Statistical Society: Series C (Applied Statistics), 1998, 47(2): 187-201.

[20] MacKay D.J.. A practical bayesian framework for backpropagation networks. Neural Computation, 1992, 4(3): 448-472.

[21] Meese R., Rogoff K.. Empirical exchange rate models of the seventies: Do they fit out of sample. Journal of International Economics, 1983, 14(1-2): 3-24.

[22] Nourani V., Komasi M., Mano A.. A multivariate ANN-Wavelet approach for rainfall-runoff modeling. Water Resources Management, 2009, 23(14): 2877-2894.

[23] Pannakkong W., Huynh V. N.. A new hybrid linear-nonlinear model based on decomposition of discrete wavelet transform for time series forecasting. International Symposium on Knowledge and Systems Sciences, 2017 186-196.

[24] Partal T. Ö. Wavelet and neuro-fuzzy conjunction model for precipitation forecasting. Journal of Hydrology, 2007, 342(1): 199-212.

[25] Rai B., Singh N.. Forecasting automobile warranty performance in presence of ‘maturing data’ phenomena using multilayer perceptron neural network. Journal of Systems Science and Systems Engineering, 2005, 14(2): 159-176.

[26] Takashi K., Shinsuke K., Kunikazu K., Masanao O.. Time series forecasting using a deep belief network with restricted Boltzmann machines. Neurocomputing, 2014, 137: 47-56.

[27] Tan Z., Zhang J., Wang J., Xu J.. Day-ahead electricity price forecasting using wavelet transform combined with ARIMA and GARCH models. Applied Energy, 2010, 87(11): 3606-3610.

[28] Thury G.. Industrial production in Germany and Austria: A case study in structural time series modeling. Journal of Systems Science and Systems Engineering, 2003, 12(2): 159-170.

[29] Thury G. Mi.. Calendar effects in monthly time series models. Journal of Systems Science and Systems Engineering, 2005, 14(2): 218-230.

[30] Tiwari M. K., Chatterjee C.. Development of an accurate and reliable hourly flood forecasting model using WaveletBootstrapANN (WBANN) hybrid approach. Journal of Hydrology, 2010, 394(3): 458-470.

[31] Twala B.. Combining classifiers for credit risk prediction. Journal of Systems Science and Systems Engineering, 2009, 18(3): 292-311.

[32] Wei S., Zuo D., Song J.. Improving prediction accuracy of river discharge time series using a wavelet-NAR articial neural network. Journal of Hydroinformatics, 2012, 14(4): 974-991.

[33] Wong C., Li W.. On a mixture autoregressive model. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2000, 62(1): 95-115.

[34] Wu W., Liu M., Liu Q., Shen W.. A quantum multi-agent based neural network model for failure prediction. Journal of Systems Science and Systems Engineering, 2016, 25(2): 210-228.

[35] Xiao J., Xie L., He C., Jiang X.. Dynamic classifier ensemble model for customer classification with imbalanced class distribution. Expert Systems with Applications, 2012, 39(3): 3668-3675.

[36] Qiu X., Ren Y., Suganthan P.N., Amaratunga G.A.. Empirical Mode Decomposition based ensemble deep learning for load demand time series forecasting. Applied Soft Computing, 2017, 54: 246-255.

[37] Zhang G.P.. Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing, 2003, 50: 159-175.

[38] Zhang G., Patuwo B.E., Hu M.Y.. Forecasting with artificial neural networks: the state of the art. International Journal of Forecasting, 1998, 14(1): 35-62.

[39] Zhou H.C., Peng Y., Liang G.H.. The research of monthly discharge predictor-corrector model based on wavelet decomposition. Water Resources Management, 2008, 22(2): 217-227.