Spatial-temporal regularized correlation filtering algorithm with adaptive aspect ratio

Abstract

In object tracking, the traditional correlation filtering algorithm is unable to perceive the change of scale aspect ratio for moving targets, and it is easily affected by a complex environment, resulting in tracking failure. Therefore, a spatial-temporal regularized correlation filtering algorithm with adaptive aspect ratio(AAR-SRCF) was proposed. Firstly, the average peak-to-correlation energy(APCE) and peak score were used as references to weigh and fuse each feature response map to achieve accurate results. Additionally, a set of novel one-dimensional boundary filters were presented, integrating near-orthogonality and spatial regularization. These filters can adaptively detect changes in the target scale and aspect ratio by precisely locating the boundaries of the target's bounding box. Moreover, spatial regularization effectively mitigated the negative impact of the boundary effect for boundary filters. Finally, the learning rate of each boundary filter was adjusted separately according to the peak-to-sidelobe ratio (PSR) to prevent the model from degradation. Through extensive experiments on OTB datasets, the proposed algorithm shows excellent tracking performance, achieving better results than other excellent algorithms in each challenge attribute.

Keywords

object tracking / correlation filter / spatial regularization / adaptive aspect ratio / template update / response fusion

References

[1] LI Y, LI M M, ZHENG Q B, et al. Survey on video object tracking algorithms.Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1504-1515.

[2] BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 13-18, 2010, San Francisco, CA, USA. Piscataway, N.J.:IEEE, 2010: 2544-2550.

[3] HENRIQUES J F, CASEIRO R, MARTINS P, et al. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583-596.

[4] DANELLJAN M, HGER G, SHAHNAZ K F, et al. Accurate scale estimation for robust visual tracking//The British Machine Vision Conference, September 1-5, 2014, Nottingham, UK. Durham: BMVA Press, 2014: 1-11.

[5] LI F, YAO Y, LI P, et al. Integrating boundary and center correlation filters for visual tracking with aspect ratio variation//IEEE International Conference on Computer Vision, October 24-27, 2017, Venice, Italy. Piscataway, N.J.: IEEE, 2017: 2001-2009.

[6] DANELLJAN M, ROBINSON A, KHAN F S, et al. Beyond correlation filters: learning continuous convolution operators for visual tracking//European Conference on Computer Vision, October 8-16, 2016, Amsterdam, The Netherlands. Heidelberg: Springer, 2016: 472-488.

[7] LI B, YAN J, WU W, et al. High performance visual tracking with Siamese region proposal network//IEEE Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, USA. Piscataway, N.J.: IEEE, 2018: 8971-8980.

[8] DANELLJAN M, HGER G, SHAHNAZ K F, et al. Learning spatially regularized correlation filters for visual tracking//2015 IEEE International Conference on Computer Vision, June 5-12, 2015, Boston, MA, USA. Piscataway, N.J.: IEEE, 2015: 4310-4318.

[9] LI F, TIAN C, ZUO W, et al. Learning spatial-temporal regularized correlation filters for visual tracking//IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, USA. Piscataway, N.J.: IEEE, 2018: 4904-4913.

[10] LI Y, FU C, DING F, et al. AutoTrack: Towards high-performance visual tracking for UAV with automatic spatio-temporal regularization//IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 13-19, 2020, Seattle, WA, USA. Piscataway, N.J.:IEEE, 2020: 11923-11932.

[11] HUANG W, YUAN L, ZHANG J J, et al. An anti-occlusion and scale-adaptive long-time target tracking algorithm. Electronics Optics and Control, 2021, 28(10): 44-48.

[12] LIU S, YANG W, SHAO X L. A background-aware target tracking algorithm based on scale adaptation. Electronics Optics and Control, 2020, 27(9): 19-23.

[13] WU Y, LIM J, YANG M H. Online object tracking: a benchmark//IEEE Conference on Computer Vision and Pattern Recognition, June 23-28, 2013, Portland, OR, USA. Piscataway, N.J.: IEEE, 2013: 2411-2418.

[14] WU Y, LIM J, YANG M H. Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834-1848.

[15] LIU S, WANG S, LIU X, et al. Fuzzy detection aided real-time and robust visual tracking under complex environments. IEEE Transactions on Fuzzy Systems, 2021, 29(1): 90-102.

[16] MA H, ACTON S T, LIN Z. SITUP: Scale invariant tracking using average peak-to-correlation energy. IEEE Transactions on Image Processing, 2020, 29: 3546-3557.

[17] WANG M M, LIU Y, HUANG Z Y. Large margin object tracking with circulant feature maps//IEEE Conference on Computer Vision and Pattern Recognition, October 24-27, 2017, Venice, Italy. Piscataway, N.J.: IEEE, 2017: 4800-4808.

[18] GALOOGAHI H K, FAGG A, LUCEY S. Learning background-aware correlation filters for visual tracking//IEEE International Conference on Computer Vision and Pattern Recognition, October 24-27, 2017, Venice, Italy. Piscataway, N.J.: IEEE, 2017: 1144-1152.

[19] DANELLJAN M, HGER G, SHAHNAZ K F, et al. Discriminative scale space tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(8): 1561-1575.

[20] LI Y, ZHU J. A scale adaptive kernel correlation filter tracker with feature integration//European Conference on Computer Vision, September 6-12, Zurich, Switzerland. Heidelberg: Springer, 2014: 254-265.

[21] ZHANG Z, PENG H, FU J, et al. Ocean: Object-aware anchor-free tracking//The 16th European Conference Computer Vision, August 23-28, 2020, Glasgow, UK. Heidelberg: Springer, 2020: 771-787.

[22] WANG N, SONG Y, MA C, et al. Unsupervised deep tracking//IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 15- 21, 2019, Los Angeles CA, USA.Piscataway, N.J.: IEEE, 2019: 1308-1317.

[23] VALMADRE J, BERTINETTO L, HENRIQUES J, et al. End-to-end representation learning for correlation filter based tracking/IEEE Conference on Computer Vision and Pattern Recognition,October 24-27, 2017, Venice, Italy. Piscataway, N.J.: IEEE, 2017: 2805-2813.

[24] BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional Siamese networks for object tracking//European Conference on Computer Vision, October 8-16, 2016, Amsterdam, The Netherlands. Heidelberg: Springer, 2016: 850-865.