Deep mixture network for uncertainty quantification in time series forecasting

Abstract

A novel uncertainty-aware forecasting model, termed Monte Carlo (MC) dropout-convolutional neural network-Kolmogorov-Arnold network-long short-term memory (MCKLNet), is proposed for high-volatility time series forecasting. The model combines a convolutional neural network (CNN) for local feature extraction, a convolutional Kolmogorov-Arnold network (KAN) for feature enhancement, long short-term memory (LSTM) networks for temporal dependency modeling, and MC dropout for Bayesian uncertainty quantification. It aims to address three major challenges in volatile time series prediction: insufficient local feature extraction, inadequate long-term dependency modeling, and limited reliability of probabilistic uncertainty estimation. To evaluate its effectiveness, pork price forecasting is employed as a representative case study due to its high volatility and sensitivity to multiple influencing factors. Experimental results show that MCKLNet consistently outperforms conventional baseline models in both point prediction accuracy and uncertainty interval calibration, demonstrating strong robustness, interpretability, and potential for broader application in economic forecasting, energy demand prediction, financial market analysis, and other complex sequential prediction tasks.

Keywords

Monte Carlo (MC) dropout / uncertainty quantification / time series forecasting / convolutional neural network-Kolmogorov-Arnold network-long short-term memory (MCKLNet) / probabilistic forecasting

References

[1] BAI S J, ZICO KOLTER J, KOLTUN V . An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv Preprint, arXiv:1803.01271, 2018.

[2] DAI Z H, YANG Z L, YANG Y M, et al. Transformer-XL: attentive language models beyond a fixed-length context. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, Jul 28 - Aug 2, Florence, Italy. Stroudsburg, PA, USA: Association for Computational Linguistics, 2019: 2978-2988.

[3] KENDALL A, GAL Y . What uncertainties do we need in Bayesian deep learning for computer vision? Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS'17), 2017, Dec 4 - 9, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates Inc, 2017: 5575-5585.

[4] HOCHREITER S, SCHMIDHUBER J . Long short-term memory. Neural Computation, 1997, 9(8): 1735-1780.

[5] CHO K, VAN MERRIËNBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP'14), 2014, Oct 25 - 29, Doha, Qatar. Stroudsburg, PA, USA: Association for Computational Linguistics, 2014: 1724-1734.

[6] ZHU D, ZHAO Q Y, CHOI H, et al. A scalable multi-photon coincidence detector based on superconducting nanowires. Nature Nanotechnology, 2018, 13(7): 596-601.

[7] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17), 2017, Dec 4 - 9, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates Inc, 2017, 6000-6010.

[8] ZHOU H Y, ZHANG S H, PENG J Q, et al. Informer: beyond efficient transformer for long sequence time-series forecasting. Proceedings of the 35th AAAI Conference on Artificial Intelligence, the 33rd Conference on Innovative Applications of Artificial Intelligence, the 11th Symposium on Educational Advances in Artificial Intelligence (AAAI'21/IAAI'21/EAAI'21), 2021, Feb 2 - 9, held virtually. Technical Tracks 12. Palo Alto, CA, USA. AAAI Press, 2021: 11106-11115.

[9] GAL Y, GHAHRAMANI Z. Dropout as a Bayesian approximation: representing model uncertainty in deep learning. Proceedings of the 33rd International Conference on Machine Learning (ICML'16), 2016, Jun 19 - 24, New York, NY, USA. Journal of Machine Learning Research (JMLR), 2016: 1050-1059.

[10] LAKSHMINARAYANAN B, PRITZEL A, BLUNDELL C. Simple and scalable predictive uncertainty estimation using deep ensembles. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17), 2017, Dec 4 - 9, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates Inc, 2017: 6405-6416.

[11] FORTUNATO M, BLUNDELL C, VINYALS O . Bayesian recurrent neural networks. arXiv Preprint, arXiv:1704.02798, 2019.

[12] ZHOU H J, ZHOU F T, ZHAO C Y, et al. Multimodal data integration for precision oncology: challenges and future directions. arXiv Preprint, arXiv:2406.19611, 2024.

[13] WU Z H, PAN S R, CHEN F W, et al. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32(1): 4-24.

[14] SUN F H, MENG X Y, ZHANG Y, et al. Agricultural product price forecasting methods: a review. Agriculture, 2023, 13(9): Article 1671.

[15] WANG Y, WANG Z, WANG X Y, et al. Multi-step-ahead and interval carbon price forecasting using transformer-based hybrid model. Environmental Science and Pollution Research, 2023, 30(42): 95692-95719.

[16] YANG X D, TANG G Q . Deep study on marketing decision under the price fluctuation of pig market based on Bayesian method. Proceedings of the 16th Dahe Fortune China Forum and Chinese High-educational Management Annual Academic Conference (DFHMC'20), 2020, Dec 4 - 6, Zhengzhou, China. Piscataway, NJ, USA: IEEE, 2020: 73-76.

[17] CHO K, VAN MERRIENBOER B, BAHDANAU D, et al. On the properties of neural machine translation: encoder-decoder approaches. Proceedings of the 8th Workshop on Syntax, Semantics and Structure in Statistical Translation (SSST'14), 2014, Oct 25, Doha, Qatar. Stroudsburg, PA, USA: Association for Computational Linguistics, 2014: 103-111.

[18] SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks. arXiv Preprint, arXiv:1409.3215, 2014.

[19] ZAREMBA W, SUTSKEVER I, VINYALS O . Recurrent neural network regularization. arXiv Preprint, arXiv:1409.2329, 2015.

[20] LECUN Y, BOSER B, DENKER J S, et al. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1989, 1(4): 541-551.

[21] LIU B, ZHANG Y, HE D J, et al. Identification of apple leaf diseases based on deep convolutional neural networks. Symmetry, 2018, 10(1): Article 11.

[22] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks. Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS'12), 2012, Dec 3 - 6, Lake Tahoe, NV, USA. Red Hook, NY, USA: Curran Associates Inc, 2012: 1097-1105.

[23] BOUREAU Y L, PONCE J, LECUN Y . A theoretical analysis of feature pooling in visual recognition. Proceedings of the 27th International Conference on International Conference on Machine Learning (ICML'10), 2010, Jun 21 - 24, Haifa Israel. Madison, WI, USA: Omnipress, 2010: 111-118.

[24] ZEILER M D, FERGUS R . Visualizing and understanding convolutional networks. Proceedings of the 13th European Conference on Computer Vision (ECCV'14): Part Ⅰ, 2014, Sep 6 - 12, Zurich, Switzerland. LNCS 8689. Berlin, Germany: Springer, 2014: 818-833.

[25] HECHT-NIELSEN R. Kolmogorov's mapping neural network existence theorem. Proceedings of the IEEE 1st Annual International Conference on Neural Networks, 1987, Jun 21 - 24, San Diego, CA, USA. Piscataway, NJ, USA: IEEE, 1987: 11-14.

[26] HORNIK K, STINCHCOMBE M, WHITE H. Multilayer feedforward networks are universal approximators. Neural Networks, 1989, 2(5): 359-366.

[27] DAS S, CHIANG Y K, MATHUR S . Thermal Sunyaev-Zel'dovich effect in the circumgalactic medium.I. detection and a surprising pattern in self-similarity and baryon sufficiency. The Astrophysical Journal, 2023, 951(2): Article 125.

[28] XU K P, CHEN L F, WANG S R . Kolmogorov-Arnold networks for time series: bridging predictive power and interpretability. arXiv Preprint, arXiv:2406.02496, 2024.

[29] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 2014, 15(1): 1929-1958.

[30] HINTON G E, SRIVASTAVA N, KRIZHEVSKY A, et al. Improving neural networks by preventing co-adaptation of feature detectors. arXiv Preprint, arXiv:1207.0580, 2012.

[31] DAMIANOU A C, LAWRENCE N D . Deep Gaussian processes. Proceedings of the 16th International Conference on Artificial Intelligence and Statistics (AISTATS'13), 2013, Apr 29 - May 1, Scottsdale, AZ, USA. Journal of Machine Learning Research (JMLR), 2013: 207-215.

[32] HENNIGHAUSEN L G, SIPPEL A E . Characterization and cloning of mRNAs specific for mouse mammary gland during lactation. European Journal of Biochemistry, 1982, 125(1): 131-141.

[33] GHYSELS E, SINKO A, VALKANOV R . MIDAS regressions: further results and new directions. Econometric Reviews, 2007, 26(1): 53-90.

[34] MARCELLINO M, SCHUMACHER C. Factor-MIDAS for now-and forecasting with ragged-edge data: a model comparison for German GDP. Discussion Paper, Series 1: Economic Studies, No. 34/2007.

[35] SAX C, STEINER P . Temporal disaggregation of time series. The R Journal, 2013, 5(2): 80-87.