A Temporally Disentangled Contrastive Diffusion Model for Spatiotemporal Imputation

Abstract

The analysis of spatiotemporal data is essential across many fields, such as transportation, meteorology and healthcare. Data gathered in practical applications often suffer from incompleteness due to device failures and network disruptions. Spatiotemporal imputation targets the estimation of missing observations by exploiting intrinsic spatial–temporal dependencies. Although traditional statistical and machine-learning methods depend on restrictive distributional assumptions, graph- or recurrent-based models accumulate errors through iterative propagation. Diffusion probabilistic models mitigate these issues by sampling directly from a learnt data prior instead of recycling past imputations. However, existing conditional diffusion variants still converge towards overly similar reconstructions, obscuring the genuine uncertainty and heterogeneity of real-world traffic, environmental or clinical streams. Preserving-and faithfully quantifying-this intrinsic diversity is crucial for reliable forecasting and downstream decision-making. We propose C²TSD, a conditional diffusion framework that integrates disentangled temporal representations and contrastive learning to improve generalisability in spatiotemporal imputation. Specifically, the approach uses disentangled temporal representations as conditional information to guide the reverse process. We also enhance the final loss using a contrastive learning strategy to improve representation quality, mitigating the impact of data missing completely at random (MCAR) and noise on learnt features. Through comprehensive experiments using three distinct real-world datasets, C²TSD has competitive results compared to leading-edge baselines.

Keywords

data mining / time series / transportation

Funding

This study was supported by the Australian Research Council (Grants DP220103717 and LE220100078).

Conflicts of Interest

The authors declare no conflicts of interest.

Data Availability Statement

The data that support the findings of this study are available in the following links: https://mega.nz/folder/Ei4SBRYD#ZjOinn0CzFPkiE_V9yVhJw https://github.com/LMZZML/PriSTI.

References

[1] Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang, “Graph Wavenet for Deep Spatial-Temporal Graph Modeling,” in 28th International Joint Conference on Artificial Intelligence (IJCAI, 2019), 1907-1913.

[2] Q. Li, J. Tan, J. Wang, and H. Chen, “A Multimodal Event-Driven Lstm Model for Stock Prediction Using Online News,” IEEE Transactions on Knowledge and Data Engineering 33, no. 10 (2020): 3323-3337, https://doi.org/10.1109/tkde.2020.2968894.

[3] D. A. Tedjopurnomo, Z. Bao, B. Zheng, F. M. Choudhury, and A. K. Qin, “A Survey on Modern Deep Neural Network for Traffic Prediction: Trends, Methods and Challenges,” IEEE Transactions on Knowledge and Data Engineering 34, no. 4 (2020): 1544-1561, https://doi.org/10.1109/tkde.2020.3001195.

[4] F. Di Martino and F. Delmastro, “Explainable AI for Clinical and Remote Health Applications: A Survey on Tabular and Time Series Data,” Artificial Intelligence Review 56, no. 6 (2023): 5261-5315, https://doi.org/10.1007/s10462-022-10304-3.

[5] P. Bauer, A. Thorpe, and G. Brunet, “The Quiet Revolution of Numerical Weather Prediction,” Nature 525, no. 7567 (2015): 47-55, https://doi.org/10.1038/nature14956.

[6] H. Du, S. Du, and W. Li, “Probabilistic Time Series Forecasting With Deep Non-Linear State Space Models,” CAAI Transactions on Intelligence Technology 8, no. 1 (2023): 3-13, https://doi.org/10.1049/cit2.12085.

[7] F. H. Ismail, G. Forestier, J. Weber, L. Idoumghar, and P. A. Muller, “Deep Learning for Time Series Classification: A Review,” Data Mining and Knowledge Discovery 33, no. 4 (2019): 917-963, https://doi.org/10.1007/s10618-019-00619-1.

[8] A. Blázquez-García, A. Conde, U. Mori, and J. A. Lozano, “A Review on Outlier/Anomaly Detection in Time Series Data,” ACM Computing Surveys 54, no. 3 (2021): 1-33, https://doi.org/10.1145/3444690.

[9] Z. Che, S. Purushotham, K. Cho, D. Sontag, and Y. Liu, “Recurrent Neural Networks for Multivariate Time Series With Missing Values,” Scientific Reports 8, no. 1 (2018): 1-12, https://doi.org/10.1038/s41598-018-24271-9.

[10] A. Cini, I. Marisca, and C. Alippi, “Filling the Gaps: Multivariate Time Series Imputation by Graph Neural Networks,” in International Conference on Learning Representations (ICLR, 2022), 1-15.

[11] W. Cao, D. Wang, J. Li, H. Zhou, L. Li, and Y. Li, “Brits: Bidirectional Recurrent Imputation for Time Series,” Advances in Neural Information Processing Systems 31 (2018): 1-10.

[12] J. Wang, W. Du, W. Cao, et al., Deep Learning for Multivariate Time Series Imputation: A Survey,” preprint, arXiv:2402.04059 (2024): 10696-10704, https://doi.org/10.24963/ijcai.2024/1187.

[13] Y. Liu, R. Yu, S. Zheng, E. Zhan, and Y. Yue, “Naomi: Non-Autoregressive Multiresolution Sequence Imputation,” Advances in Neural Information Processing Systems 32 (2019): 1-10.

[14] J. Ho, A. Jain, and P. Abbeel, “Denoising Diffusion Probabilistic Models,” Advances in Neural Information Processing Systems 33 (2020): 6840-6851.

[15] Y. Tashiro, J. Song, Y. Song, and S. Ermon , “Csdi: Conditional Score-Based Diffusion Models for Probabilistic Time Series Imputation,” Advances in Neural Information Processing Systems 34 (2021): 24804-24816.

[16] M. Liu, H. Huang, H. Feng, L. Sun, B. Du, and Y. Fu, “PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputation,” in 39th International Conference on Data Engineering (IEEE, 2023), 1927-1939.

[17] P. Dhariwal and A. Nichol, “Diffusion Models Beat Gans on Image Synthesis,” Advances in Neural Information Processing Systems 34 (2021): 8780-8794.

[18] A. Q. Nichol and P. Dhariwal, “Improved Denoising Diffusion Probabilistic Models,” in International Conference on Machine Learning (PMLR, 2021), 8162-8171.

[19] S. Li, Y. Chen, and H. Xiong, “Channel-Aware Contrastive Conditional Diffusion for Multivariate Probabilistic Time Series Forecasting,” preprint, arXiv:2410.02168 (2024).

[20] R. B. Cleveland, W. S. Cleveland, J. E. McRae, and I. Terpenning, “STL: A Seasonal-Trend Decomposition Procedure Based on Loess,” Journal of Official Statistics 6, no. 1 (1990): 3-33.

[21] R. J. Hyndman and G. Athanasopoulos , Forecasting: Principles and Practice (OTexts, 2018).

[22] K. Bandara, C. Bergmeir, and H. Hewamalage, “LSTM-MSNet: Leveraging Forecasts on Sets of Related Time Series With Multiple Seasonal Patterns,” IEEE Transactions on Neural Networks and Learning Systems 32, no. 4 (2020): 1586-1599, https://doi.org/10.1109/tnnls.2020.2985720.

[23] H. Wu, J. Xu, J. Wang, and M. Long, “Autoformer: Decomposition Transformers With Auto-Correlation for Long-Term Series Forecasting,” in Advances in Neural Information Processing Systems 34 (2021): 22419-22430.

[24] R. Sen, H. F. Yu, and I. S. Dhillon, “Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting,” in Advances in Neural Information Processing Systems 32 (2019): 4837-4846.

[25] G. Woo, C. Liu, D. Sahoo, A. Kumar, and S. Hoi, “CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting,” in International Conference on Learning Representations (ICLR, 2022), 1-15.

[26] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A Simple Framework for Contrastive Learning of Visual Representations,” in International Conference on Machine Learning (PMLR, 2020), 1597-1607.

[27] M. Kang and J. Park , “Contragan: Contrastive Learning for Conditional Image Generation,” Advances in Neural Information Processing Systems 33 (2020): 21357-21369.

[28] M. Kang, S. Zhu, and J. Park, “Contrastive Generative Adversarial Networks,” in AAAI Conference on Artificial Intelligence (AAAI, 2021), 8116-8124.

[29] J. Zhou, J. Li, G. Zheng, X. Wang, and C. Zhou, “Mtsci: A Conditional Diffusion Model for Multivariate Time Series Consistent Imputation,” in Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (CIKM, 2024), 3474-3483.

[30] Y. Sun, J. Li, Y. Xu, T. Zhang, and X. Wang, “Deep Learning Versus Conventional Methods for Missing Data Imputation: A Review and Comparative Study,” Expert Systems with Applications 227 (2023): 120201, https://doi.org/10.1016/j.eswa.2023.120201.

[31] K. Zhang, Q. Wen, C. Zhang, et al., Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects,” IEEE Transactions on Pattern Analysis and Machine Intelligence 46, no. 10 (2024): 6775-6794, https://doi.org/10.1109/tpami.2024.3387317.

[32] Y. Zheng, L. Capra, O. Wolfson, and H. Yang, “Urban Computing: Concepts, Methodologies, and Applications,” ACM Transactions on Intelligent Systems and Technology 5, no. 3 (2014): 1-55, https://doi.org/10.1145/2629592.

[33] Y. Li, R. Yu, C. Shahabi, and Y. Liu, “Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting,” in International Conference on Learning Representations. ICLR (2018), 1-15.

[34] H. Hersbach, “Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction Systems,” Weather and Forecasting 15, no. 5 (2000): 559-570, https://doi.org/10.1175/1520-0434(2000)015<0559:dotcrp>2.0.co;2.

[35] I. R. White, P. Royston, and A. M. Wood, “Multiple Imputation Using Chained Equations: Issues and Guidance for Practice,” Statistics in Medicine 30, no. 4 (2011): 377-399, https://doi.org/10.1002/sim.4067.

[36] H. F. Yu, N. Rao, and I. S. Dhillon, “Temporal Regularized Matrix Factorization for High-Dimensional Time Series Prediction,” Advances in Neural Information Processing Systems 29 (2016): 1-9.

[37] X. Chen, Z. He, Y. Chen, Y. Lu, and J. Wang, “Missing Traffic Data Imputation and Pattern Discovery With a Bayesian Augmented Tensor Factorization Model,” Transportation Research Part C: Emerging Technologies 104 (2019): 66-77, https://doi.org/10.1016/j.trc.2019.03.003.

[38] J. Yoon, J. Jordon, and M. Schaar, “Gain: Missing Data Imputation Using Generative Adversarial Nets,” in International Conference on Machine Learning (PMLR, 2018), 5689-5698.

[39] A. W. Mulyadi, E. Jun, and H. I. Suk, “Uncertainty-Aware Variational-Recurrent Imputation Network for Clinical Time Series,” IEEE Transactions on Cybernetics 52, no. 9 (2021): 9684-9694, https://doi.org/10.1109/tcyb.2021.3053599.

[40] V. Fortuin, D. Baranchuk, G. Rätsch, and S. Mandt, “Gp-vae: Deep Probabilistic Time Series Imputation,” in International Conference on Artificial Intelligence and Statistics (PMLR, 2020), 1651-1661.

[41] J. L. Alcaraz and N. Strodthoff, “Diffusion-Based Time Series Imputation and Forecasting With Structured State Space Models,” Transactions on Machine Learning Research (2023), https://openreview.net/forum?id=hHiIbk7ApW.

[42] S. Zhang, S. Wang, H. Miao, H. Chen, C. Fan, and J. Zhang, “Score-CDM: Score-Weighted Convolutional Diffusion Model for Multivariate Time Series Imputation,” preprint, arXiv:2405.13075 (2024).

[43] T. Nie, G. Qin, W. Ma, Y. Mei, and J. Sun, “ImputeFormer: Low rankness-induced Transformers for Generalizable Spatiotemporal Imputation,” in 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (ACM, 2024), 2260-2271.

[44] X. Yi, Y. Zheng, J. Zhang, and T. Li, “ST-MVL: Filling Missing Values in Geo-Sensory Time Series Data,” in 25th International Joint Conference on Artificial Intelligence (IJCAI, 2016), 1-7.

[45] J. Bracher, E. L. Ray, R. J. Tibshirani, and N. G. Reich, “Evaluating Epidemic Forecasts in an Interval Format,” PLoS Computational Biology 17, no. 2 (2021): e1008618, https://doi.org/10.1371/journal.pcbi.1008618.

[46] K. Rasul, C. Seward, I. Schuster, and R. Vollgraf, “Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting,” in Proceedings of the 38th International Conference on Machine Learning (2021), 8857-8868.

[47] T. Gneiting and A. E. Raftery, “Strictly Proper Scoring Rules, Prediction, and Estimation,” Journal of the American Statistical Association 102, no. 477 (2007): 359-378, https://doi.org/10.1198/016214506000001437.

[48] C. F. Ansley and R. Kohn, “On the Estimation of ARIMA Models With Missing Values,” in Time Series Analysis of Irregularly Observed Data (Springer, 1984), 9-37.

[49] R. H. Shumway and D. S. Stoffer, “An Approach to Time Series Smoothing and Forecasting Using the EM Algorithm,” Journal of Time Series Analysis 3, no. 4 (1982): 253-264, https://doi.org/10.1111/j.1467-9892.1982.tb00349.x.

[50] T. Hastie, R. Tibshirani, J. H. Friedman, and J. H. Friedman , The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Vol. 2 (Springer, 2009).

[51] S. Liu, X. Li, G. Cong, Y. Chen, and Y. Jiang, “Multivariate Time-Series Imputation With Disentangled Temporal Representations,” in International Conference on Learning Representations (ICLR, 2022), 1-15.

[52] Y. Luo, Y. Zhang, X. Cai, and X. Yuan, “E2gan: End-to-end Generative Adversarial Network for Multivariate Time Series Imputation,” in 28th International Joint Conference on Artificial Intelligence (AAAI Press, 2019), 3094-3100.

[53] Y. Chen, Z. Li, C. Yang, X. Wang, G. Long, and G. Xu, “Adaptive Graph Recurrent Network for Multivariate Time Series Imputation,” in International Conference on Neural Information Processing (Springer Nature, 2022), 64-73.

[54] Y. Chen, K. Shi, X. Wang, and G. Xu, “MTSTI: A Multi-Task Learning Framework for Spatiotemporal Imputation,” in International Conference on Advanced Data Mining and Applications (Springer Nature, 2023), 180-194.

[55] Y. Chen, R. Hu, Z. Li, et al., Exploring Explicit and Implicit Graph Learning for Multivariate Time Series Imputation,” Engineering Applications of Artificial Intelligence 127 (2024): 107217, https://doi.org/10.1016/j.engappai.2023.107217.

[56] S. R. Kuppannagari, Y. Fu, C. M. Chueng, and V. K. Prasanna, “Spatio-Temporal Missing Data Imputation for Smart Power Grids,” in 12th ACM International Conference on Future Energy Systems (ACM, 2021), 458-465.

[57] Z. Wu, L. Cao, and L. Qi, “eVAE: Evolutionary Variational Autoencoder,” IEEE Transactions on Neural Networks and Learning Systems 36, no. 2 (2024): 3288-3299, https://doi.org/10.1109/tnnls.2024.3359275.

[58] Z. Wu, Q. Zhang, J. Zhou, H. Chen, and Y. Liu, “WAVAE: A Weakly Augmented Variational Autoencoder for Time Series Anomaly Detection,” Information Fusion 125 (2025): 103462, https://doi.org/10.1016/j.inffus.2025.103462.

[59] Z. Wu, X. Fan, and L. Cao, “ProgDiffusion: Progressively Self-encoding Diffusion Models,” in Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Vol. 1 (2025), 1633-1644, https://doi.org/10.1145/3690624.3709222.

[60] K. Rasul, C. Seward, I. Schuster, and R. Vollgraf, “Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting,” in International Conference on Machine Learning (PMLR, 2021), 8857-8868.

[61] T. Yan, H. Zhang, T. Zhou, Y. Zhan, and Y. Xia, “ScoreGrad: Multivariate Probabilistic Time Series Forecasting With Continuous Energy-Based Generative Models,” preprint, arXiv:2106.10121 (2021).

[62] Y. Li, X. Lu, Y. Wang, and D. Dou, “Generative Time Series Forecasting With Diffusion, Denoise, and Disentanglement,” Advances in Neural Information Processing Systems 35 (2022): 23009-23022.

[63] H. Wen, Y. Lin, Y. Xia, et al., Diffstg: Probabilistic Spatio-Temporal Graph Forecasting With Denoising Diffusion Models,” in 31st ACM International Conference on Advances in Geographic Information Systems (ACM, 2023), 1-12.

[64] R. Li, X. Li, S. Gao, S. B. Choy, and J. Gao, “Graph Convolution Recurrent Denoising Diffusion Model for Multivariate Probabilistic Temporal Forecasting,” in International Conference on Advanced Data Mining and Applications (Springer, 2023), 661-676.

[65] J. Li, R. Zheng, H. Feng, M. Li, and X. Zhuang, “Permutation Equivariant Graph Framelets for Heterophilous Graph Learning,” IEEE Transactions on Neural Networks and Learning Systems 35, no. 9 (2024): 11634-11648, https://doi.org/10.1109/tnnls.2024.3370918.

[66] M. Li, A. Micheli, Y. G. Wang, et al., Guest Editorial: Deep Neural Networks for Graphs: Theory, Models, Algorithms, and Applications,” IEEE Transactions on Neural Networks and Learning Systems 35, no. 4 (2024): 4367-4372, https://doi.org/10.1109/tnnls.2024.3371592.

[67] L. Bai, L. Cui, Y. Wang, et al., Haqjsk: Hierarchical-Aligned Quantum Jensen-Shannon Kernels for Graph Classification,” IEEE Transactions on Knowledge and Data Engineering 36, no. 11 (2024): 6370-6384, https://doi.org/10.1109/tkde.2024.3389966.