UAV object tracking algorithm based on spatial saliency-aware correlation filter

i

This article has been updated. A Correction to this article was published on .

i

The Original Article was published on

i

The Original Article was published on

i

This article has been updated

i

This article was retracted on

i

This article has been updated

Abstract

Recently, correlation filter-based tracking methods have been widely adopted in UAV target tracking due to their outstanding performance and excellent tracking efficiency. However, existing correlation filter-based tracking methods still face issues such as redundant visual features with weak discriminative ability, inadequate spatio-temporal information mining, and filter degradation. In order to overcome these challenges, this paper proposes a spatial saliency-aware strategy that reduces redundant information in spatial and channel dimensions, thus improving the discriminative ability between the target and background. Also, this paper proposes a position estimation mechanism under spatio-temporal joint constraints to fully mine spatio-temporal information and enhance the robustness of the model in complex scenarios. Furthermore, this paper establishes a positive expert group using historical positive samples to assess the reliability of candidate samples, thereby effectively mitigating the filter degradation issue. Ultimately, the effectiveness of the proposed method is demonstrated through the evaluation of multiple public datasets. The experimental results reveal that this method outperforms others in tracking performance under various challenging conditions.

Keywords

UAV object tracking / spatial saliency perception / positive expert group / spatio-temporal joint constraints

CHANGE HISTORY

A Correction to this paper has been published:
https://doi.org

The Retraction Note was updated to reflect reception of statements of disagreement from all authors.

This article has been retracted.Please see the Retraction Notice for more detail:
https://doi.org

References

[1] H. Zhang, P. He, M. Zhang, C. Daqing, E. Neretin, B. Li, UAV target tracking method based on deep reinforcement learning, in International Conference on Cyber-Physical Social Intelligence (ICCSI), IEEE, (2022), 274-277. https://doi.org/10.1109/ICCSI55536.2022.9970588

[2] R. Wu, Y. Liu, X. Wang, P. Yang, Visual tracking based on spatiotemporal transformer and fusion sequences, Image Vision Comput., 148 (2024), 105107. https://doi.org/10.1016/j.imavis.2024.105107

[3] J. McGee, S. J. Mathew, F. Gonzalez, Unmanned aerial vehicle and artificial intelligence for thermal target detection in search and rescue applications, in International Conference on Unmanned Aircraft Systems, IEEE, (2020), 883-891. https://doi.org/10.1109/ICUAS48674.2020.9213849

[4] P. Byukusenge, Y. Zhang, Life detection based on uavs - thermal images in search and rescue operation, in IEEE 22nd International Conference on Communication Technology, IEEE, (2022), 1728-1731. https://doi.org/10.1109/ICCT56141.2022.10073136

[5] K. Chen, L. Wang, H. Wu, C. Wu, Y. Liao, Y. Chen, et al., Background-aware correlation filter for object tracking with deep cnn features, Eng. Lett., 32 (2024), 1351-1363.

[6] J. Wen, H. Chu, Z. Lai, T. Xu, L. Shen, Enhanced robust spatial feature selection and correlation filter learning for uav tracking, Neural Networks, 161 (2023), 39-54. https://doi.org/10.1016/j.neunet.2023.01.003

[7] J. Lin, J. Peng, J. Chai, Real-time UAV correlation filter based on response-weighted background residual and spatio-temporal regularization, IEEE Geosci. Remote Sens. Lett., 20 (2023), 1-5. https://doi.org/10.1109/LGRS.2023.3272522

[8] T. Xu, Z. H. Feng, X. J. Wu, J. Kittler, Joint group feature selection and discriminative filter learning for robust visual object tracking, in IEEE/CVF International Conference on Computer Vision , IEEE, (2019), 7949-7959. https://doi.org/10.1109/ICCV.2019.00804

[9] A. Lukezic, T. Vojir, L. C. Zajc, J. Matas, M. Kristan, Discriminative correlation filter with channel and spatial reliability, in IEEE Conference on Computer Vision and Pattern Recognition, IEEE, (2017), 6309-6318. https://doi.org/10.1109/CVPR.2017.515

[10] Y. Chen, K. Chen, Four mathematical modeling forms for a correlation filter object tracking algorithm and the fast calculation for the filter, Electron. Res. Arch., 32 (2024), 4684-4714. https://doi.org/10.3934/era.2024213

[11] L. Bertinetto, J. Valmadre, J. Henriques, A. Vedaldi, P. Torr, Fully-convolutional siamese networks for object tracking, in Computer Vision-ECCV 2016 Workshops, Springer, 9914 (2016), 850-865. https://doi.org/10.1007/978-3-319-48881-356

[12] P. Voigtlaender, J. Luiten, P. H. S. Torr, B. Leibe, Siam r-cnn: Visual tracking by re-detection, in IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, (2020), 6578-6588. https://doi.org/10.1109/CVPR42600.2020.00661

[13] X. Chen, B. Yan, J. Zhu, D. Wang, X. Yang, H. Lu, Transformer tracking, in IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, (2021), 8126-8135. https://doi.org/10.1109/CVPR46437.2021.00803

[14] B. Yan, H. Peng, J. Fu, D. Wang, H. Lu, Learning spatio-temporal transformer for visual tracking, in IEEE/CVF International Conference on Computer Vision , IEEE, (2021), 10448-10457. https://doi.org/10.1109/ICCV48922.2021.01028

[15] Y. Cui, C. Jiang, L. Wang, G. Wu, Mixformer: End-to-end tracking with iterative mixed attention, in IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, (2022), 13608-13618. https://doi.org/10.1109/CVPR52688.2022.01324

[16] S. Xuan, S. Li, M. Han, X. Wan, G. Xia, Object tracking in satellite videos by improved correlation filters with motion estimations, IEEE Trans. Geosci. Remote Sens., 58 (2020), 1074-1086. https://doi.org/10.1109/TGRS.2019.2943366

[17] S. Ma, B. Zhao, Z. Hou, W. Yu, L. Pu, X. Yang, Socf: A correlation filter for real-time uav tracking based on spatial disturbance suppression and object saliency-aware, Expert Syst. Appl., 238 (2024), 122131. https://doi.org/10.1016/j.eswa.2023.122131

[18] J. van de Weijer, C. Schmid, J. Verbeek, D. Larlus, Learning color names for real-world applications, IEEE Trans. Image Process., 18 (2009), 1512-1523. https://doi.org/10.1109/TIP.2009.2019809

[19] L. Bertinetto, J. Valmadre, S. Golodetz, O. Miksik, P. Torr, Staple: Complementary learners for real-time tracking, in IEEE Conference on Computer Vision and Pattern Recognition, IEEE, (2016), 1401-1409. https://doi.org/10.1109/CVPR.2016.156

[20] H. K. Galoogahi, A. Fagg, S. Lucey, Learning background-aware correlation filters for visual tracking, in IEEE International Conference on Computer Vision, IEEE, (2017), 1135-1143. https://doi.org/10.1109/ICCV.2017.129

[21] M. Mueller, N. Smith, B. Ghanem, Context-aware correlation filter tracking, in IEEE Conference on Computer Vision and Pattern Recognition , IEEE, (2017), 1396-1404. https://doi.org/10.1109/CVPR.2017.152

[22] M. Danelljan, G. Bhat, F. S. Khan, M. Felsberg, Eco: Efficient convolution operators for tracking, in IEEE Conference on Computer Vision and Pattern Recognition, IEEE, (2017), 6638-6646. https://doi.org/10.1109/CVPR.2017.733

[23] C. Ma, J. B. Huang, X. Yang, M. H. Yang, Hierarchical convolutional features for visual tracking, in IEEE International Conference on Computer Vision , IEEE, (2015), 3074-3082. https://doi.org/10.1109/ICCV.2015.352

[24] M. Danelljan, G. Hager, F. S. Khan, M. Felsberg, Convolutional features for correlation filter based visual tracking, in IEEE International Conference on Computer Vision Workshops, IEEE, (2015), 58-66. https://doi.org/10.1109/ICCVW.2015.84

[25] K. Dai, D. Wang, H. Lu, C. Sun, J. Li, Visual tracking via adaptive spatially-regularized correlation filters, in IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, (2019), 4670-4679. https://doi.org/10.1109/CVPR.2019.00480

[26] J. Hu, L. Shen, G. Sun, Squeeze-and-excitation networks, in IEEE Conference on Computer Vision and Pattern Recognition, IEEE, (2018), 7132-7141. https://doi.org/10.1109/CVPR.2018.00745

[27] F. Du, P. Liu, W. Zhao, X. Tang, Joint channel reliability and correlation filters learning for visual tracking, IEEE Trans. Circuits Syst. Video Technol., 30 (2019), 1625-1638. https://doi.org/10.1109/TCSVT.2019.2909654

[28] T. Xu, Z. H. Feng, X. J. Wu, J. Kittler, Learning adaptive discriminative correlation filters via temporal consistency preserving spatial feature selection for robust visual object tracking, IEEE Trans. Image Process., 28 (2019), 5596-5609. https://doi.org/10.1109/TIP.2019.2919201

[29] W. Feng, R. Han, Q. Guo, J. Zhu, S. Wang, Dynamic saliency-aware regularization for correlation filter-based object tracking, IEEE Trans. Image Process., 38 (2019), 3232-3245. https://doi.org/10.1109/TIP.2019.2895411

[30] D. Zhao, L. Xiao, H. Fu, T. Wu, X. Xu, B. Dai, Augmenting cascaded correlation filters with spatial-temporal saliency for visual tracking, Inf. Sci., 470 (2019), 78-93. https://doi.org/10.1016/j.ins.2018.08.053

[31] P. Yang, Q. Wang, J. Dou, L. Dou, Sdcs-cf: Saliency-driven localization and cascade scale estimation for visual tracking, J. Visual Commun. Image Represent., 98 (2024), 104040. https://doi.org/10.1016/j.jvcir.2023.104040

[32] C. Fu, J. Xu, F. Lin, F. Guo, T. Liu, Z. Zhang, Object saliency-aware dual regularized correlation filter for real-time aerial tracking, IEEE Trans. Geosci. Remote Sens., 58 (2020), 8940-8951. https://doi.org/10.1109/TGRS.2020.2992301

[33] X. Yang, S. Li, J. Ma, J. Yang, J. Yan, Co-saliency-regularized correlation filter for object tracking, Signal Process. Image Commun., 103 (2022), 116655. https://doi.org/10.1016/j.image.2022.116655

[34] P. Zhang, W. Liu, D. Wang, Y. Lei, H. Wang, H. Lu, Non-rigid object tracking via deep multi-scale spatial-temporal discriminative saliency maps, Pattern Recognit., 100 (2020), 107130. https://doi.org/10.1016/j.patcog.2019.107130

[35] L. Gao, B. Liu, P. Fu, M. Xu, J. Li, Visual tracking via dynamic saliency discriminative correlation filter, Appl. Intell., 52 (2022), 5897-5911. https://doi.org/10.1007/s10489-021-02260-2

[36] F. Li, C. Tian, W. Zuo, L. Zhang, M. H. Yang, Learning spatial-temporal regularized correlation filters for visual tracking, in IEEE Conference on Computer Vision and Pattern Recognition, IEEE, (2018), 4904-4913. https://doi.org/10.1109/CVPR.2018.00515

[37] Y. Li, C. Fu, F. Ding, Z. Huang, G. Lu, Autotrack: Towards high-performance visual tracking for uav with automatic spatio-temporal regularization, in IEEE/CVF Conference on Computer Vision and Pattern Recognition , IEEE, (2020), 11920-11929. https://doi.org/10.1109/CVPR42600.2020.01194

[38] Y. Chen, H. Wu, Z. Deng, J. Zhang, H. Wang, L. Wang, et al., Deep-feature-based asymmetrical background-aware correlation filter for object tracking, Digital Signal Process., 148 (2024), 104446. https://doi.org/10.1016/j.dsp.2024.104446

[39] Z. Song, J. Yu, Y. P. P. Chen, W. Yang, Transformer tracking with cyclic shifting window attention, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2022), 8791-8800. https://doi.org/10.1109/CVPR52688.2022.00859

[40] M. Danelljan, G. Hager, F. S. Khan, M. Felsberg, Adaptive decontamination of the training set: A unified formulation for discriminative visual tracking, in IEEE Conference on Computer Vision and Pattern Recognition, IEEE, (2016), 1430-1438. https://doi.org/10.1109/CVPR.2016.159

[41] M. Danelljan, A. Robinson, F. S. Khan, M. Felsberg, Beyond correlation filters: Learning continuous convolution operators for visual tracking, in European Conference on Computer Vision, Springer, (2016), 472-488. https://doi.org/10.1007/978-3-319-46454-129 https://doi.org/10.1007/978-3-319-46454-129

[42] Q. Hu, H. Wu, J. Wu, J. Shen, H. Hu, Y. Chen, et al., Spatio-temporal self-learning object tracking model based on anti-occlusion mechanism, Eng. Lett., 31 (2023), 1141-1150.

[43] Y. Huang, Y. Chen, C. Lin, Q. Hu, J. Song, Visual attention learning and antiocclusion-based correlation filter for visual object tracking, J. Electron. Imaging, 32 (2023), 013023. https://doi.org/10.1117/1.JEI.32.1.013023

[44] Y. Wu, J. Lim, M. H. Yang, Online object tracking: A benchmark, in IEEE Conference on Computer Vision and Pattern Recognition , IEEE, (2013), 2411-2418. https://doi.org/10.1109/CVPR.2013.312

[45] S. Li, D. Y. Yeung, Visual object tracking for unmanned aerial vehicles: A benchmark and new motion models, in AAAI Conference on Artificial Intelligence, AAAI Press, (2017), 4140-4146.

[46] M. Mueller, N. G. Smith, B. Ghanem, A benchmark and simulator for uav tracking, in European Conference on Computer Vision, Springer, (2016), 445-461. https://doi.org/10.1007/978-3-319-46448-027

[47] S. Ma, Z. Zhao, L. Pu, Z. Hou, L. Zhang, X. Zhao, Learning discriminative correlation filters via saliency-aware channel selection for robust visual object tracking, J. Real-Time Image Process., 20 (2023), 51. https://doi.org/10.1007/s11554-023-01306-7

[48] M. Danelljan, G. Hager, F. S. Khan, M. Felsberg, Learning spatially regularized correlation filters for visual tracking, in IEEE International Conference on Computer Vision, IEEE, (2015), 4310-4318. https://doi.org/10.1109/ICCV.2015.490

[49] M. Danelljan, G. Hager, F. S. Khan, M. Felsberg, Discriminative scale space tracking, IEEE Trans. Pattern Anal. Mach. Intell., 39 (2016), 1561-1575. https://doi.org/10.1109/TPAMI.2016.2609928

[50] N. Wang, W. Zhou, Q. Tian, R. Hong, M. Wang, H. Li, Multi-cue correlation filters for robust visual tracking, in IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, (2018), 4844-4853. https://doi.org/10.1109/CVPR.2018.00509

[51] Z. Huang, C. Fu, Y. Li, F. Lin, P. Lu, Learning aberrance repressed correlation filters for real-time UAV tracking, in IEEE/CVF International Conference on Computer Vision, IEEE, (2019), 2891-2900. https://doi.org/10.1109/ICCV.2019.00298

[52] C. Fu, J. Ye, J. Xu, Y. He, J. Xu, Y. He, Disruptor-aware interval-based response inconsistency for correlation filters in real-time aerial tracking, IEEE Trans. Geosci. Remote Sens., 59 (2021), 6301-6313. https://doi.org/10.1109/TGRS.2020.3030265

[53] X. F. Zhu, X. J. Wu, T. Xu, Z. H. Feng, J. Kittler, Robust visual object tracking via adaptive attribute-aware discriminative correlation filters, IEEE Trans. Multimedia, 24 (2022), 301-312. https://doi.org/10.1109/TMM.2021.3050073

[54] C. Ma, X. Yang, C. Zhang, M. H. Yang, Long-term correlation tracking, in IEEE Conference on Computer Vision and Pattern Recognition, (2015), 5388-5396. https://doi.org/10.1109/CVPR.2015.7299177

[55] Y. Li, J. Zhu, A scale adaptive kernel correlation filter tracker with feature integration, in Computer Vision-ECCV 2014 Workshops , Springer, 8926 (2015), 254-265. https://doi.org/10.1007/978-3-319-16181-518

[56] M. Danelljan, G. Bhat, F. S. Khan, M. Felsberg, Atom: Accurate tracking by overlap maximization, in IEEE Conference on Computer Vision and Pattern Recognition, IEEE, (2019), 4660-4669. https://doi.org/10.1109/CVPR.2019.00479

[57] G. Bhat, M. Danelljan, L. V. Gool, R. Timofte, Learning discriminative model prediction for tracking, in IEEE/CVF International Conference on Computer Vision , IEEE, (2019), 6182-6191. https://doi.org/10.1109/ICCV.2019.00628