Multi-person vision tracking approach based on human body localization features

Abstract

This paper presents a multi-person vision tracking approach based on human body localization features to address the problem of interactive object localization and tracking in a home monitoring scenario. Firstly, the human body localization model is used to obtain the 3D position of the human body, which is then used to construct the human body motion model based on the Kalman filter method. At the same time, the human appearance model is constructed by fusing human color features and features of the histogram of oriented gradient to better characterize the human body. Secondly, the human body observation model is constructed based on the human body motion model and appearance model to measure the similarities between the human body state sequence in the historical frame and the human body observation result in the current frame, and the cost matrix is then obtained. Thirdly, the Hungarian maximum matching algorithm is employed to match each human body in the current and historical frames, and the exception detection mechanism is simultaneously constructed to further reduce the probability of human tracking and matching failure. Finally, a multi-person vision tracking verification platform was constructed, and the achieved average accuracy was 96.6% in the case of human body overlapping, occlusion, disappearance, and appearance; this verifies the feasibility and effectiveness of the proposed method.

Keywords

Multi-person vision tracking / Human body positioning / Motion model / Body observation model

References

[1] Li J, Zhan W, Hu Y, et al. Generic tracking and probabilistic prediction framework and its application in autonomous driving. IEEE Trans Intell Transp Syst, 2020, 21(9): 3634-3649.

[2] Fukunaga K, Hostetler LD. The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans Inf Theory, 1975, 21(1): 32-40.

[3] Liu B, Huang J, Kulikowski C, et al. Robust visual tracking using local sparse appearance model and K-selection. IEEE Trans Pattern Anal Mach Intell, 2013, 35(12): 2968-2981.

[4] Elhoseny M. Multi-object detection and tracking (MODT) machine learning model for real-time video surveillance systems. Circuits Syst Signal Process, 2019, 39(2): 611-630.

[5] Aftab W, Mihaylova L. Learning Gaussian process approach for maneuvering target tracking and smoothing. IEEE Trans Aerosp Electron Syst, 2021, 57(1): 278-292.

[6] Zhang L, Li Y, Nevatia R (2008) Global data association for multi-object tracking using network flows. In: IEEE conference on computer vision and pattern recognition, Anchorage, USA, pp 23−28. https://doi.org/10.1109/CVPR.2008.4587584

[7] Shitrit HB, Berclaz J, Fleuret F, et al. Multi-commodity network flow for tracking multiple people. IEEE Trans Pattern Anal Mach Intell, 2014, 36(8): 1614-1627.

[8] Zamir AR, Dehghan A, Shah M, et al. Fitzgibbon A, Lazebnik S, Perona P, et al. GMCP-tracker: global multi-object tracking using generalized minimum clique graphs. Computer vision–ECCV 2012: Lecture Notes in Computer Science, 2012, Berlin: Springer.

[9] Milan A, Roth S, Schindler K. Continuous energy minimization for multitarget tracking. IEEE Trans Pattern Anal Mach Intell, 2014, 36(1): 58-72.

[10] Dehghan A, Assari SM, Shah M (2015) GMMCP tracker: globally optimal generalized maximum multi clique problem for multiple object tracking. In IEEE conference on computer vision and pattern recognition, Boston, USA, 7−12 June, pp 4091−4099. https://doi.org/10.1109/CVPR.2015.7299036

[11] Zhou H, Ouyang WL, Cheng J, et al. Deep continuous conditional random fields with asymmetric inter-object constraints for online multi-object tracking. IEEE Trans Circuits Syst Video Technol, 2019, 29(4): 1011-1022.

[12] Xiang J, Sang N, Hou J, et al. Multitarget tracking using hough forest random field. IEEE Trans Circuits Syst Video Technol, 2016, 26(11): 2028-2042.

[13] Sun SJ, Akhtar N, Song HS, et al. Deep affinity network for multiple object tracking. IEEE Trans Pattern Anal Mach Intell, 2021, 43(1): 104-119.

[14] Ge Z, Chang F, Liu H (2017) Multi-target tracking based on Kalman filtering and optical flow histogram. In: 2017 Chinese automation congress (CAC), Jinan, China, 20−22 Oct, pp 2540–2545. https://doi.org/10.1109/CAC.2017.8243203

[15] Zhao Z, Yu S, Wu X (2009) A multi-target tracking algorithm using texture for real-time surveillance. In: IEEE international conference on robotics and biomimetics, Bangkok, Thailand, 22–25 Feb, pp 2150−2155. https://doi.org/10.1109/ROBIO.2009.4913335

[16] Sheng H, Zhang Y, Chen J, et al. Heterogeneous association graph fusion for target association in multiple object tracking. IEEE Trans Circuits Syst Video Technol, 2019, 29(11): 3269-3280.

[17] Yang AL, Ren HY, Fei MR, et al. Dynamic body vision localization approach based on multiple regression. Chinese Journal of Scientific Instrument, 2020, 41(7): 252-260.

[18] Xu Y, Yu G, Wang Y, et al. A hybrid vehicle detection method based on Viola-Jones and HOG + SVM from UAV Images. Sensors, 2016, 16(8): 1325-1347.

Funding

Natural Science Foundation of Shanghai http://dx.doi.org/10.13039/100007219(18ZR1415100)