A Temporal Correlation Networks Based on Interactive Modelling for Remote Sensing Images Change Detection

Abstract

Change detection identifies dynamic changes in surface cover and feature status by comparing remote sensing images at different points in time, which is of wide application value in the fields of disaster early warning, urban management and ecological monitoring. Mainstream datasets are dominated by long-term datasets; to support short-term change detection, we collected a new dataset, HNU-CD, which contains some small and hard-to-identify change regions. A time correlation network (TCNet) is also proposed to address these challenges. First, foreground information is enhanced by interactively modelling foreground relations, while background noise is smoothed. Secondly, the temporal correlation between bit-time images is utilised to refine the feature representation and minimise false alarms due to irrelevant changes. Finally, a U-Net inspired architecture is adapted for dense upsampling to preserve details. TCNet demonstrates excellent performance on both the HNU-CD (Hainan University change detection dataset) dataset and three widely used public datasets, indicating that its generalisation capabilities have been enhanced. The ablation experiments provide a good demonstration of the ability to reduce the impact caused by pseudo-variation through temporal correlation modelling.

Keywords

change detection / neural nets / small-scale dataset / temporal correlation

References

[1] H. Zhang, M. Lin, G. Yang, and L. Zhang, “ESCNet: An End-to-End Superpixel-Enhanced Change Detection Network for Very-High-resolution Remote Sensing Images,” IEEE Transactions on Neural Net-works and Learning Systems 34, no. 1 (January 2023): 28-42, https://doi.org/10.1109/TNNLS.2021.3089332.

[2] H. Chen and Z. Shi, “A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection,” Remote Sensing 12, no. 10 (May 2020): 1662, https://doi.org/10.3390/rs12101662.

[3] K. Sakurada and T. Okatani, “Change Detection From a Street Image Pair Using CNN Features and Superpixel Segmentation,” Proceedings of British Machine Vision Conference 61 (2015): 61.1-61.12, https://doi.org/10.5244/c.29.61.

[4] X. Zheng, D. Guan, B. Li, Z. Chen, and L. Pan, “Global and Local Graph-Based Difference Image Enhancement for Change Detection,” Remote Sensing 15, no. 5 (2023): 1194, https://doi.org/10.3390/rs15051194.

[5] X. Feng and L. Lang, “Cgtracker: A Center Graph Based Multi-Pedestrian-Object Detection and Tracking,” Journal of Computer Sci-ence and Technology 14, no. 2 (2022): 200-215, https://doi.org/10.1007/s11390-022-2204-8.

[6] G. Tang, X. Zhu, J. Guo, and S. Dietze, “Time Enhanced Graph Neural Networks for Session-based Recommendation,” Knowledge-Based Systems 251 (2022): 109204, https://doi.org/10.1016/j.knosys.2022.109204.

[7] Q. Xu, K. Chen, X. Sun, Y. Zhang,H. Li, and G. Xu, “Pseudo-Siamese Capsule Network for Aerial Remote Sensing Images Change Detection,” IEEE Geoscience and Remote Sensing Letters 19 (2022): 1-5: Art no. 6000405, https://doi.org/10.1109/LGRS.2020.3022512.

[8] M. Zhang, G. Xu, K. Chen, M. Yan, and X. Sun, “Triplet-Based Se-mantic Relation Learning for Aerial Remote Sensing Image Change Detection,” IEEE Geoscience and Remote Sensing Letters 16, no. 2 (February 2019): 266-270, https://doi.org/10.1109/lgrs.2018.2869608.

[9] X. Zhang, S. Cheng,L. Wang, and H. Li, “Asymmetric Cross-Attention Hierarchical Network Based on CNN and Transformer for Bitemporal Remote Sensing Images Change Detection,” IEEE Trans-actions on Geoscience and Remote Sensing 61 (2023): 1-15: Art no. 2000415, https://doi.org/10.1109/TGRS.2023.3245674.

[10] C. Han, C. Wu, and B. Du, “HCGMNet: A Hierarchical Change Guiding Map Network for Change Detection,” in IGARSS 2023—2023 IEEE International Geoscience and Remote Sensing Symposium 2023), 5511-5514, https://doi.org/10.1109/IGARSS52108.2023.10283341.

[11] R. G. Babukarthik, D. Chandramohan, D. Tripathi, M. Kumar, and G. Sambasivam, “COVID-19 Identification in Chest X-ray Images Using Intelligent Multi-Level Classification Scenario,” Computers & Electrical Engineering 104 (2022/12/01/2022): 108405, https://doi.org/10.1016/j.compeleceng.2022.108405.

[12] X. Zheng, X. Chen, X. Lu, and B. Sun, “Unsupervised Change Detection by Cross-Resolution Difference Learning,” IEEE Transactions on Geoscience and Remote Sensing 60 (January 2022): 1-16, https://doi.org/10.1109/tgrs.2021.3079907.

[13] Z. Zheng, Y. Zhong,J. Wang, and A. Ma, “Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020), 4095-4104, https://doi.org/10.1109/CVPR42600.2020.00415.

[14] Z. Zhu and C. E. Woodcock, “Continuous Change Detection and Classification of Land Cover Using all Available Landsat Data,” Remote Sensing of Environment 144 (March 2014): 152-171, https://doi.org/10.1016/j.rse.2014.01.011.

[15] R. E. Kennedy, Z. Yang, and W. B. Cohen, “Detecting Trends in Forest Disturbance and Recovery Using Yearly Landsat Time Series: 1. LandTrendr — Temporal Segmentation Algorithms,” Remote Sensing of Environment 114, no. 12 (December 2010): 2897-2910, https://doi.org/10.1016/j.rse.2010.07.008.

[16] S. P. Patil, R. S. Apare, R. H. Borhade, and P. N. Mahalle, “Auto-mated Dyslexia Screening Using Children's Handwriting in English Language With Convolutional Neural Network and Bidirectional Long Short-Term Memory Model,” Engineered Science 32 (2024): 1345, https://doi.org/10.30919/es1345.

[17] R. Caye Daudt, B. Le Saux, and A. Boulch, “Fully Convolutional Siamese Networks for Change Detection,” in 2018 25th IEEE Interna-tional Conference on Image Processing ICIP, 2018), 4063-4067, https://doi.org/10.1109/ICIP.2018.8451652.

[18] S. Sabour, N. Frosst, and G. E. Hinton, “Dynamic Routing Between Capsules ” Proceedings of Advances in Neural Information Processing Systems (2017): 3859-3869, https://dlnext.acm.org/doi/10.5555/3294996.3295142.

[19] T. Lei, J. Wang, H. Ning, et al., “Difference Enhancement and Spatial-Spectral Nonlocal Network for Change Detection in VHR Remote Sensing Images,” IEEE Transactions on Geoscience and Remote Sensing 60 (2022): 1-13: Art no. 4507013, https://doi.org/10.1109/TGRS.2021.3134691.

[20] H. Chen and Z. Shi, “A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection,” Remote Sensing 12, no. 10 (2020): 1662, https://doi.org/10.3390/rs12101662.

[21] C.-P. Chen J.-W. Hsieh P.-Y. Chen Y.-K. Hsieh and B.-S. Wang, “SARAS-Net: Scale and Relation Aware Siamese Network for Change Detection,” Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 12 (2023): 14187-14195, https://doi.org/10.1609/aaai.v37i12.26660.

[22] H. Lyu, H. Lu, and L. Mou, “Learning a Transferable Change Rule From a Recurrent Neural Network for Land Cover Change Detection,” Remote Sensing 8, no. 6 (2016): 506, https://doi.org/10.3390/rs8060506.

[23] Y. Liu, C. Pang, Z. Zhan, X. Zhang, and X. Yang, “Building Change Detection for Remote Sensing Images Using a Dual-Task Constrained Deep Siamese Convolutional Network Model,” IEEE Geoscience and Remote Sensing Letters 18, no. 5 (May 2021): 811-815, https://doi.org/10.1109/LGRS.2020.2988032.

[24] J. Zheng, Y. Tian, C. Yuan, et al., “MDESNet: Multitask Difference-Enhanced Siamese Network for Building Change Detection in High-Resolution Remote Sensing Images,” Remote Sensing 14, no. 15 (2022): 3775, https://doi.org/10.3390/rs14153775.

[25] Y. Zhao, P. Chen, Z. Chen, Y. Bai,Z. Zhao, and X. Yang, “A Triple-Stream Network With Cross-Stage Feature Fusion for High-Resolution Image Change Detection,” IEEE Transactions on Geoscience and Remote Sensing 61 (2023): 1-17: Art no. 5600417, https://doi.org/10.1109/TGRS.2022.3233849.

[26] C. Zhou, Z. He, J. Dong, Y. Li,J. Ren, and A. Plaza, “Low-Rank and Sparse Representation Meet Deep Unfolding: A New Interpretable Network for Hyperspectral Change Detection,” IEEE Transactions on Geoscience and Remote Sensing 63 (2025): 1-16: Art no. 5513516, https://doi.org/10.1109/tgrs.2025.3564996.

[27] W. G. C. Bandara and V. M. Patel, “A Transformer-Based Siamese Network for Change Detection,” in IGARSS 2022—2022 IEEE Interna-tional Geoscience and Remote Sensing Symposium 2022), 207-210, https://doi.org/10.1109/IGARSS46834.2022.9883686.

[28] Y. Deng, Y. Meng, J. Chen, A. Yue, D. Liu, and J. Chen, “TChange: A Hybrid Transformer-CNN Change Detection Network,” Remote Sensing 15, no. 5 (2023): 1219, https://doi.org/10.3390/rs15051219.

[29] H. Chen,Z. Qi, and Z. Shi, “Remote Sensing Image Change Detection With Transformers,” IEEE Transactions on Geoscience and Remote Sensing 60 (2022): 1-14: Art no. 5607514, https://doi.org/10.1109/TGRS.2021.3095166.

[30] Y. Feng, H. Xu, J. Jiang,H. Liu, and J. Zheng, “ICIF-Net: Intra-Scale Cross-Interaction and Inter-Scale Feature Fusion Network for Bitem-poral Remote Sensing Images Change Detection,” IEEE Transactions on Geoscience and Remote Sensing 60 (2022): 1-13: Art no. 4410213, https://doi.org/10.1109/TGRS.2022.3168331.

[31] Y. Feng, J. Jiang,H. Xu, and J. Zheng, “Change Detection on Remote Sensing Images Using Dual-Branch Multilevel Intertemporal Network,” IEEE Transactions on Geoscience and Remote Sensing 61 (2023): 1-15: Art no. 4401015, https://doi.org/10.1109/TGRS.2023.3241257.

[32] C. Zhang, L. Wang,S. Cheng, and Y. Li, “SwinSUNet: Pure Trans-former Network for Remote Sensing Image Change Detection,” IEEE Transactions on Geoscience and Remote Sensing 60 (2022): 1-13: Art no. 5224713, https://doi.org/10.1109/TGRS.2022.3160007.

[33] A. Mohammadian and F. Ghaderi, “SiamixFormer: A Siamese Transformer Network for Building Detection and Change Detection From Bi-Temporal Remote Sensing Images,” arXiv e-prints (2022), https://doi.org/10.48550/arXiv.2208.00657.

[34] C. Zhou, Q. Shi, D. He, B. Tu, H. Li, and A. Plaza, “Spectral-Spatial Sequence Characteristics-Based Convolutional Transformer for Hyper-spectral Change Detection,” CAAI Transactions on Intelligence Tech-nology 8, no. 4 (2023): 1237-1257, https://doi.org/10.1049/cit2.12226.

[35] J. Zhang, J. Lei, W. Xie, Z. Fang, Y. Li, and Q. Du, “SuperYOLO: Super Resolution Assisted Object Detection in Multimodal Remote Sensing Imagery,” IEEE Transactions on Geoscience and Remote Sensing 61 (2023): 1-15, https://doi.org/10.1109/TGRS.2023.3258666.

[36] Q. Shi, M. Liu, S. Li, X. Liu,F. Wang, and L. Zhang, “A Deeply Supervised Attention Metric-Based Network and an Open Aerial Image Dataset for Remote Sensing Change Detection,” IEEE Transactions on Geoscience and Remote Sensing 60 (2022): 1-16: Art no. 5604816, https://doi.org/10.1109/TGRS.2021.3085870.

[37] C. Zhang, P. Yue, D. Tapete, et al., “A Deeply Supervised Image Fusion Network for Change Detection in High Resolution bi-Temporal Remote Sensing Images,” ISPRS Journal of Photogrammetry and Remote Sensing 166 (2020): 183-200, https://doi.org/10.1016/j.isprsjprs.2020.06.003.

[38] S. Fang, K. Li, J. Shao, and Z. Li, “SNUNet-CD: A Densely Con-nected Siamese Network for Change Detection of VHR Images,” IEEE Geoscience and Remote Sensing Letters 19 (2022): 1-5: Art no. 8007805, https://doi.org/10.1109/LGRS.2021.3056416.

[39] B. Jiang, Z. Wang, X. Wang, et al., “VcT: Visual Change Transformer for Remote Sensing Image Change Detection,” IEEE Transactions on Geoscience and Remote Sensing 61 (2023): 1-14: Art no. 2005214, https://doi.org/10.1109/TGRS.2023.3327139.

[40] C. Han, C. Wu, H. Guo, M. Hu, and H. Chen, “HANet: A Hierar-chical Attention Network for Change Detection With Bitemporal Very-High-Resolution Remote Sensing Images,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 16 (2023): 3867-3878, https://doi.org/10.1109/JSTARS.2023.3264802.