Real-time manifold regularized context-aware correlation tracking

Abstract

Despite the demonstrated success of numerous correlation filter (CF) based tracking approaches, their assumption of circulant structure of samples introduces significant redundancy to learn an effective classifier. In this paper, we develop a fast manifold regularized context-aware correlation tracking algorithm that mines the local manifold structure information of different types of samples. First, different from the traditional CF based tracking that only uses one base sample, we employ a set of contextual samples near to the base sample, and impose a manifold structure assumption on them. Afterwards, to take into account the manifold structure among these samples, we introduce a linear graph Laplacian regularized term into the objective of CF learning. Fortunately, the optimization can be efficiently solved in a closed form with fast Fourier transforms (FFTs), which contributes to a highly efficient implementation. Extensive evaluations on the OTB100 and VOT2016 datasets demonstrate that the proposed tracker performs favorably against several state-of-the-art algorithms in terms of accuracy and robustness. Especially, our tracker is able to run in real-time with 28 fps on a single CPU.

Keywords

visual tracking / manifold regularization / correlation filter / graph Laplacian

References

[1] Li X, Hu W M, Shen C H, Zhang Z F, Dick A, Hengel A V D. A survey of appearance models in visual object tracking. ACM Transactions on Intelligent Systems and Technology, 2013, 4(4): 1–13

[2] Wang H J, Ge H J. Visual tracking using discriminative representation with l2 regularization. Frontiers of Computer Science, 2018, 12(1): 1–13

[3] Ali A, Jalil A, Niu J W, Zhao X K, Rathore S, Ahmed J, Iftikhar M A. Visual object tracking−classical and contemporary approaches. Frontiers of Computer Science, 2016, 10(1): 167–188

[4] Zhang K H, Liu Q S, Ynag J, Yang M H. Visual tracking via boolean map representations. Pattern Recognition, 2018, 81: 147–160

[5] Zhang K H, Li X J, Song H H, Liu Q S, Lian W. Visual tracking using spatio-temporally nonlocally regularized correlation filter. Pattern Recognition, 2018, 83: 185–195

[6] Bolme D S, Beveridge J R, Draper B A, Lui Y M. Visual object tracking using adaptive correlation filters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2010, 2544–2550

[7] Henriques J F, Caseiro R, Martins P, Batista J. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583–596

[8] Zhang K H, Zhang L, Liu Q S, Zhang D, Yang M H. Fast visual tracking via dense spatio-temporal context learning. In: Proceedings of European Conference on Computer Vision. 2014, 127–141

[9] Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr P H S. Staple: complementary learners for real-time tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 1401–1409

[10] Zhang K H, Liu Q S, Wu Y, Yang M H. Robust visual tracking via convolutional networks without training. IEEE Transactions on Image Processing, 2016, 25(4): 1779–1792

[11] Ma C, Xu Y, Ni B B, Yang X K. When correlation filters meet convolutional neural networks for visual tracking. IEEE Signal Processing Letters, 2016, 23(10): 1454–1458

[12] Danelljan M, Häger G, Khan F, Felsberg M. Accurate scale estimation for robust visual tracking. In: Proceedings of the British Machine Vision Conference. 2014

[13] Kristan M, Leonardis A, Matas J. The visual object tracking VOT2017 challenge results. In: Proceedings of the IEEE International Conference on Computer Vision Workshop. 2017, 1949–1972

[14] Mueller M, Smith N, Ghanem B. Context-aware correlation filter tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 1396–1404

[15] Galoogahi K H, Fagg A, Lucey S. Learning background-aware correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 1135–1143

[16] Yan Y, Nie F, Li W, Gao C Q, Yang Y, Xu D. Image classification by cross-media active learning with privileged information. IEEE Transactions on Multimedia, 2016, 18(12): 2494–2502

[17] Yang Y, Ma Z G, Nie F P, Chang X J, Hauptmann A G. Multi-class active learning by uncertainty sampling with diversity maximization. International Journal of Computer Vision, 2015, 113(2): 113–127

[18] Yang Y, Nie F P, Xu D, Luo J B, Zhuang Y T, Pan Y H. A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(4): 723–742

[19] Wu Y, Lim J W, Yang M H. Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834–1848

[20] Danelljan M, Shahbaz K F, Felsberg M, Joost V W. Adaptive color attributes for real-time visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 1090–1097

[21] Ma C, Huang J B, Yang X K, Yang M H. Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision. 2015, 3074–3082

[22] Danelljan M, Hager G, Khan F S, Felsberg M. Convolutional features for correlation filter based visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision Workshops. 2015, 58–66

[23] Danelljan M, Robinson A, Khan F S, Felsberg M. Beyond correlation filters: learning continuous convolution operators for visual tracking. In: Proceedings of European Conference on Computer Vision. 2016, 472–488

[24] Danelljan M, Bhat G, Khan F S, Felsberg M. ECO: rfficient convolution operators for tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 21–26

[25] Liu S, Zhang T Z, Cao X C, Xu C S. Structural correlation filter for robust visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4312–4320

[26] Lukezic A, Vojír T, Zajc L C, Matas J, Kristan M. Discriminative correlation filter with channel and spatial reliability. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 6309–6318

[27] Danelljan M, Hager G, Shahbaz K F, Felsberg M. Learning spatially regularized correlation filters for visual tracking. In: Proceedings of European Conference on Computer Vision. 2015, 4310–4318

[28] Belkin M, Niyogi P, Sindhwani V. Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 2006, 7(Nov): 2399–2434

[29] Chang X J, Yang Y. Semisupervised feature analysis by mining correlations among multiple tasks. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(10): 2294–2305

[30] Yu S, Yang Y, Hauptmann A. Harry potter’s marauder’s map: localizing and tracking multiple persons-of-interest by nonnegative discretization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 3714–3720

[31] Bai Y C, Tang M. Robust tracking via weakly supervised ranking SVM. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 1854–1861

[32] Hu H W, Ma B, Shen J B, Shao L. Manifold regularized correlation object tracking. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(5): 1786–1795

[33] Zhuang B, Lu H C, Xiao Z Y, Wang D. Visual tracking via discriminative sparse similarity map. IEEE Transactions on Image Processing, 2014, 23(4): 1872–1881

[34] Belkin M, Niyogi P. Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in Neural Processing Systems, 2001, 585–591

[35] Ma C, Yang X K, Zhang C Y, Yang M H. Long-term correlation tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 5388–5396

[36] Zhang J M, Ma S G, Sclaroff S. MEEM: robust tracking via multiple experts using entropy minimization. In: Proceedings of European Conference on Computer Vision. 2014, 188–203

[37] Hare S, Golodetz S, Saffari A, Vineet V, Cheng M M, Hicks S L, Torr P H S. Struck: structured output tracking with kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(10): 2096–2109

[38] Wu Y, Lim J W, Yang M H. Online object tracking: a benchmark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 2411–2418

[39] Song Y B, Ma C, Gong L J, Zhang J W, Lau R W H, Yang M H. Crest: convolutional residual learning for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 2574–2583

[40] Zhu G, Porikli F, Li H D. Beyond local search: tracking objects everywhere with instance-specific proposals. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 943–951

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag GmbH Germany, part of Springer Nature