Correlation filter tracking based on superpixel and multifeature fusion

Abstract

For the correlation filtering (CF) tracking algorithm is not robust enough and cannot adapt to scale changes, target occlusion (OCC) and other complex interferences. We introduce a CF tracking algorithm based on superpixel and multifeature fusion (CFSMF). First, superpixel segmentation and clustering are performed for the target and its surrounding environment in the initial frame. Then, a target appearance is reconstructed through block segmentation-based overlapping analysis to remove redundant information. On this basis, the histogram of gradient (HOG) and HSI color features of the target sub-block are extracted to interact with their respective position filters. Accordingly, the target position is determined by the weighted fusion of the response values. In the scale prediction stage, we independently train a scale filter with a multiscale pyramid constructed at the estimated target location. The object scale is estimated in terms of the filter response, thereby enabling the tracking algorithm to adapt to the object scale change. Lastly, we introduce an OCC criterion for determining whether to update the model or not. Compared with the classical tracking algorithm kernelized correlation filters (KCF), the proposed algorithm boosts the tracking success rate by 20% and tracking accuracy by 15.9%. Our algorithm in this paper could track the target stably even when the target is occluded and its scale changes.

References

[1] RossD A, LimJ, LinR-S, YangM H. International Journal of Computer Vision, 2008, 11: 125

[2] Jia Xu, H. Lu and M. H. Yang. Visual Tracking via Adaptive Structural Local Sparse Appearance Model, in Proc. IEEE Conf. Computer. Vis. Pattern Recognition, 1822 (2012).

[3] N Y Wang and D Y Yeung, Learning a Deep Compact Image Representation for Visual Tracking, International Conference on Neural Information Processing Systems Curran Associates Inc., 809 (2013).

[4] Adam Amit, E. Rivlin and I. Shimshoni, Robust Fragments-based Tracking using the Integral Histogram, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 798 (2006).

[5] J Shen and S Zafeiriouand GChrysos, The First Facial Landmark Tracking in-the-Wild Challenge: Benchmark and Results, IEEE International Conference on Computer Vision Workshops, 1003 (2015).

[6] M Matthias, N. Smith and B. Ghanem, Context-Aware Correlation Filter Tracking, IEEE Conference on Computer Vision and Pattern Recognition, 1387 (2017).

[7] H Sheng, K Lv, J H Chen and W li, Robust Visual Tracking Using Correlation Response Map, IEEE International Conference on Image Processing, 2381 (2016).

[8] LiY, ZhuJ. A Scale Adaptive Kernel Correlation Filter Tracker with Feature Integration, European Conference on Computer Vision, 2014, Cham, Springer

[9] LiuJ M, GuoJ W, ShiS. Optics and Precision Engineering, 2018, 26: 2100

[10] Bolme D S, Beveridge J R, Draper B A and Y M Lui, Visual Object Tracking Using Adaptive Correlation Filters, IEEE Conference on Computer Vision and Pattern Recognition, 2544 (2010).

[11] J. F. Henriques, R. Caseiro, P. Martins and J. Batisita, Exploiting the Circulant Structure of Tracking-by-detection with Kernels, the 12th European Conference on Computer Vision, 702 (2012).

[12] HenriquesJ F, CaseiroR, MartinsP, BatisitaJ. PAMI, 2015, 37: 583

[13] D. Martin, F. Khan, M. Felsberg and J. Weijer, Adaptive Color Attributes for Real-Time Visual Tracking, IEEE Conference on Computer Vision and Pattern Recognition, 1090 (2014).

[14] B. Luca, J. Valmadre, S. Golodetz, O. Miksik and P. H. Torr, Staple: Complementary Learners for Real-Time Tracking, IEEE Conference on Computer Vision and Pattern Recognition, 1401 (2016).

[15] YangX H, ZhangH, YangL, LiuC S, PeterX. A Joint Multi-Feature and Scale-Adaptive Correlation Filter Tracker. IEEE Access, 2018, PP: 1

[16] WangS, LuH, YangF, YangM-H. Superpixel Tracking, 2011, Barcelona, IEEE International Conference on Computer Vision: 1323

[17] DanelljanM, HägerG, KhanF S, FelsbergM. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2016, 39: 1561

[18] C Ma, X K Yang, C Y Zhang and M S Yang, Long-term Correlation Tracking, IEEE Conference on Computer Vision and Pattern Recognition, 5388 (2015).

[19] Li Y and Zhu J, A Scale Adaptive Kernel Correlation Filter Tracker with Feature Integration, European Conference on Computer Vision, 254 (2016).

[20] Y K Qi, S P Zhang, L Qin, H Yao, Q Huang, J Lim and M H Yang, Hedged Deep Tracking, IEEE Conference on Computer Vision and Pattern Recognition, 4303 (2016).

[21] M. Danelljan, A. Robinson, F. S. Khan and M. Felsberg, Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking, European Conference on Computer Vision, 2016.

[22] M. Danelljan, G. Bhat, F. S. Khan and M Felsberg, ECO: Efficient Convolution Operators for Tracking, IEEE Conference on Computer Vision and Pattern Recognition, 21 (2017).

[23] AchantaR, ShajiA, SmithK, LucchiA, FuaP, SusstrunkS. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34: 2274

[24] YiW, LimJ, YangM H. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37: 1834

[25] M. Danelljan, G. Hager, F. Shahbaz Khan and M. Felsberg, Learning Spatially Regularized Correlation Filters for Visual Tracking, International Conference on Computer Vision, 4310 (2015).

[26] Danelljan M, Hager G, Khan F and M Felsberg, Convolutional Features for Correlation Filter Based Visual Tracking, IEEE International Conference on Computer Vision Workshop, 621 (2015).

[27] L. Bertinetto, J Valmadre, S Golodetz, O Miksik and P Torr, Staple: Complementary Learners for Real-Time Tracking, IEEE Conference on Computer Vision and Pattern Recognition, 2016.

[28] C Ma, J Huang, X Yang and M Yang, Hierarchical Convolutional Features for Visual Tracking, International Conference on Computer Vision, 2015.