Robust object tracking with RGBD-based sparse learning

Abstract

Robust object tracking has been an important and challenging research area in the field of computer vision for decades. With the increasing popularity of affordable depth sensors, range data is widely used in visual tracking for its ability to provide robustness to varying illumination and occlusions. In this paper, a novel RGBD and sparse learning based tracker is proposed. The range data is integrated into the sparse learning framework in three respects. First, an extra depth view is added to the color image based visual features as an independent view for robust appearance modeling. Then, a special occlusion template set is designed to replenish the existing dictionary for handling various occlusion conditions. Finally, a depth-based occlusion detection method is proposed to efficiently determine an accurate time for the template update. Extensive experiments on both KITTI and Princeton data sets demonstrate that the proposed tracker outperforms the state-of-the-art tracking algorithms, including both sparse learning and RGBD based methods.

Keywords

Object tracking / Sparse learning / Depth view / Occlusion templates / Occlusion detection

References

[1] Avidan, S., 2007. Ensemble tracking.IEEE Trans. Patt. Anal. Mach. Intell., 29(2):261–271.https://doi.org/10.1109/TPAMI.2007.35

[2] Babenko, B., Yang, M.H., Belongie,S. , 2009. Visual tracking with online multiple instance learning.IEEE Conf. on Computer Vision and Pattern Recognition, p.983–990. https://doi.org/10.1109/CVPR.2009.5206737

[3] Bao, C.L., Wu, Y., Ling, H.B., , 2012. Real time robust L1 tracker using accelerated proximal gradient approach.IEEE Conf. on Computer Vision and Pattern Recognition, p.1830–1837. https://doi.org/10.1109/CVPR.2012.6247881

[4] Black, M.J., Jepson, A.D., 1998. EigenTracking: robust matching and tracking of articulated objects using a view-based representation. Int. J. Comput. Vis., 26(1): 63–84. https://doi.org/10.1023/A:1007939232436

[5] Candes, E.J., Romberg , J.K., Tao, T. , 2006. Stable signal recovery from incomplete and inaccurate measurements.Commun. Pure Appl. Math., 59(8):1207–1223. https://doi.org/10.1002/cpa.20124

[6] Chen, X., Pan, W.K., Kwok, J.T. , , 2009. Accelerated gradient method for multi-task sparse learning problem.9th IEEE Int. Conf. on Data Mining, p.746–751. https://doi.org/10.1109/ICDM.2009.128

[7] Choi, W., Pantofaru , C., Savarese, S. , 2011. Detecting and tracking people using an RGB-D camera via multiple detector fusion.IEEE Int. Conf. on Computer Vision Workshops, p.1076–1083. https://doi.org/10.1109/ICCVW.2011.6130370

[8] Comaniciu, D., Ramesh, V., Meer, P., 2003. Kernel-based object tracking.IEEE Trans. Patt. Anal. Mach. Intell., 25(5):564–577. https://doi.org/10.1109/TPAMI.2003.1195991

[9] Dalal, N., Triggs, B., 2005. Histograms of oriented gradients for human detection.IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, p.886–893. https://doi.org/10.1109/CVPR.2005.177

[10] Donoho, D.L., 2006. Compressed sensing.IEEE Trans. Inform. Theory, 52(4):1289–1306. https://doi.org/10.1109/TIT.2006.871582

[11] Hong, Z.B., Mei, X., Prokhorov, D. , , 2013. Tracking via robust multi-task multi-view joint sparse representation.IEEE Int. Conf. on Computer Vision, p.649–656. https://doi.org/10.1109/ICCV.2013.86

[12] Lan, X.Y., Ma, A., Yuen, P., 2014. Multi-cue visual tracking using robust feature-level fusion based on joint sparse representation.IEEE Int. Conf. on Computer Vision and Pattern Recognition, p.1194–1201. https://doi.org/10.1109/CVPR.2014.156

[13] Ling, H.B., Bai, L., Blasch, E., , 2010. Robust infrared vehicle tracking across target pose change using L1 reg-ularization.IEEE Conf. on Information Fusion, p.1–8. https://doi.org/10.1109/ICIF.2010.5711902

[14] Liu, B.Y., Yang, L., Huang, J.Z. , , 2010. Robust and fast collaborative tracking with two stage sparse optimization.European Conf. on Computer Vision, p.624–637. https://doi.org/10.1007/978-3-642-15561-1_45

[15] Luber, M., Spinello , L., Arras, K.O. , 2011. People tracking in RGB-D data with on-line boosted target models.IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, p.3844–3849. https://doi.org/10.1109/IROS.2011.6095075

[16] Ma, Z.A., Xiang, Z.Y., 2015. Robust visual tracking via bin-ocular multi-task multi-view joint sparse representation.SAI Intelligent Systems Conf., p.714–722. https://doi.org/10.1109/IntelliSys.2015.7361219

[17] Mei, X., Ling, H.B., 2009. Robust visual tracking using ℓ1 minimization.IEEE 12th Int. Conf. on Computer Vision, p.1436–1443. https://doi.org/10.1109/ICCV.2009.5459292

[18] Mei, X., Ling, H.B., 2011. Robust visual tracking and vehicle classification via sparse representation.IEEE Trans. Patt.Anal. Mach. Intell., 33(11):2259–2272. https://doi.org/10.1109/TPAMI.2011.66

[19] Mei, X., Ling, H.B., Wu, Y. , , 2011. Minimum error bounded efficient ℓ1 tracker with occlusion detection.IEEE Conf. on Computer Vision and Pattern Recognition, p.1257–1264. https://doi.org/10.1109/CVPR.2011.5995421

[20] Ojala, T., Pietikäinen , M., Mäenpää, T., 2002. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns.IEEE Trans. Patt. Anal. Mach. Intell., 24(7):971–987. https://doi.org/10.1109/TPAMI.2002.1017623

[21] Pei, S.C., Lin, C.N., 1995. Image normalization for pattern recognition.Image Vis. Comput., 13(10):711–723. https://doi.org/10.1016/0262-8856(95)98753-G

[22] Porikli, F., Tuzel, O., Meer, P., 2006. Covariance tracking using model update based on Lie algebra.IEEE Computer Society Conf. on Computer Vision and Pattern Recogni-tion, p.728–735. https://doi.org/10.1109/CVPR.2006.94

[23] Ross, D.A., Lim, J., Lin, R.S., , 2008. Incremental learning for robust visual tracking.Int. J. Comput. Vis., 77(1-3):125–141. https://doi.org/10.1007/s11263-007-0075-7

[24] Song, S.R., Xiao, J.X., 2013. Tracking revisited using RGBD camera: unified benchmark and baselines.IEEE Int. Conf. on Computer Vision, p.233–240. https://doi.org/10.1109/ICCV.2013.36

[25] Williams, O., Blake, A., Cipolla, R. , 2005. Sparse Bayesian learning for efficient visual tracking.IEEE Trans. Patt. Anal. Mach. Intell., 27(8):1292–1304. https://doi.org/10.1109/TPAMI.2005.167

[26] Wright, J., Yang, A.Y., Ganesh, A. , , 2009. Robust face recognition via sparse representation.IEEE Trans. Patt. Anal. Mach. Intell., 31(2):210–227. https://doi.org/10.1109/TPAMI.2008.79

[27] Wu, Y., Lim, J., Yang, M.H., 2013. Online object tracking: a benchmark.IEEE Conf. on Computer Vision and Pattern Recognition, p.2411–2418.https://doi.org/10.1109/CVPR.2013.312

[28] Yang, M., Zhang, L., 2010. Gabor feature-based sparse rep-resentation for face recognition with Gabor occlusion dictionary.European Conf. on Computer Vision, p.448–461. https://doi.org/10.1007/978-3-642-15567-3_33

[29] Yilmaz, A., Javed, O., Shah, M., 2006. Object tracking: a survey. ACM Comput.Surv., 38(4):43–56.https://doi.org/10.1145/1177352.1177355

[30] Yin, Z.Z., Collins , R.T., 2008. Object tracking and detection after occlusion via numerical hybrid local and global mode-seeking.IEEE Conf. on Computer Vision and Pattern Recognition, p.1–8. https://doi.org/10.1109/CVPR.2008.4587542

[31] Zhang, K., Zhang, L., Yang, M.H., 2012. Real-time compres-sive tracking.European Conf. on Computer Vision, p.864–877. https://doi.org/10.1007/978-3-642-33712-3_62

[32] Zhang, T.Z., Ghanem, B., Liu, S., , 2012a. Low-rank sparse learning for robust visual tracking.European Conf. on Computer Vision, p.470–484. https://doi.org/10.1007/978-3-642-33783-3_34

[33] Zhang, T.Z., Ghanem, B., Liu, S., , 2012b. Robust visual tracking via multi-task sparse learning.IEEE Conf. on Computer Vision and Pattern Recognition, p.2042–2049. https://doi.org/10.1109/CVPR.2012.6247908

[34] Zhang, T.Z., Ghanem, B., Liu, S., , 2013. Robust visual tracking via structured multi-task sparse learning.Int. J. Comput. Vis., 101(2):367–383. https://doi.org/10.1007/s11263-012-0582-z

[35] Zhang, T.Z., Liu, S., Ahuja, N.,, 2015a. Robust visual tracking via consistent low-rank sparse learning. Int. J. Comput. Vis., 111(2):171–190. https://doi.org/10.1007/s11263-014-0738-0

[36] Zhang, T.Z., Liu, S., Xu, C.S., , 2015b. Structural sparse tracking.IEEE Conf. on Computer Vision and Pattern Recognition, p.150–158. https://doi.org/10.1109/CVPR.2015.7298610

[37] Zhang, Z.Y., 1994. Iterative point matching for registration of free-form curves and surfaces.Int. J. Comput. Vis., 13(2): 119–152.https://doi.org/10.1007/BF01427149

RIGHTS & PERMISSIONS

2017 Zhejiang University and Springer-Verlag Berlin Heidelberg