Multi-object tracking based on deep associated features for UAV applications

Abstract

Multi-object tracking (MOT) techniques have been increasingly applied in a diverse range of tasks. Unmanned aerial vehicle (UAV) is one of its typical application scenarios. Due to the scene complexity and the low resolution of moving targets in UAV applications, it is difficult to extract target features and identify them. In order to solve this problem, we propose a new re-identification (re-ID) network to extract association features for tracking in the association stage. Moreover, in order to reduce the complexity of detection model, we perform the lightweight optimization for it. Experimental results show that the proposed re-ID network can effectively reduce the number of identity switches, and surpass current state-of-the-art algorithms. In the meantime, the optimized detector can increase the speed by 27% owing to its lightweight design, which enables it to further meet the requirements of UAV tracking tasks.

References

[1] CiaparroneG, SánchezF L, TabikS, et al.. Deep learning in video multi-object tracking: a survey [J]. Neurocomputing, 2020, 381: 61-88

[2] KamalR, ChemmanamA J, JoseB A, et al.. Construction safely surveillance using machine learning[C], 2020, New York, IEEE: 1-6

[3] XuY, OsepA, BanY, et al.. How to train your deep multi-object tracker[C], 2020, New York, IEEE: 6787-6796

[4] BehrendtK, NovakL, BotrosR. A deep learning approach to traffic lights: detection, tracking, and classification[C], 2017, New York, IEEE: 1370-1377

[5] PereiraR, GarroteL, BarrosT, et al.. A deep learning-based indoor scene classification approach enhanced with inter-object distance semantic features[C], 2021, New York, IEEE: 32-38

[6] WuJ, CaoJ, SongL, et al.. Track to detect and segment: an online multi-object tracker[C], 2021, New York, IEEE: 12352-12361

[7] LiuY, LiX, BaiT, et al.. Multi-object tracking with hard-soft attention network and group-based cost minimization[J]. Neurocomputing, 2021, 447: 80-91

[8] LiuQ, ChuQ, LiuB, et al.. GSM: graph similarity model for multi-object tracking[C], 2020, California, IJCAI: 530-536

[9] ZhouX, KoltunV, KrähenbühlP. Tracking objects as points[C], 2020, Cham, Springer: 474-490Virtual

[10] ZhangY, WangC, WangX, et al.. Fairmot: on the fairness of detection and re-identification in multiple object tracking[J]. International journal of computer vision, 2021, 129(11):3069-3087

[11] BrasóG, Leal-TaixéL. Learning a neural solver for multiple object tracking[C], 2020, New York, IEEE: 6247-6257

[12] HORNAKOVA A, HENSCHEL R, ROSENHAHN B, et al. Lifted disjoint paths with application in multiple object tracking[C]//International Conference on Machine Learning, July 12–18, 2020, Virtual. IMLS, 2020: 4364–4375.

[13] KimC, FuxinL, AlotaibiM, et al.. Discriminative appearance modeling with, multi-track pooling for real-time multi-object tracking[C], 2021, New York, IEEE: 9553-9562

[14] WojkeN, BewleyA, PaulusD. Simple online and realtime tracking with a deep association metric[C], 2017, New York, IEEE: 3645-3649

[15] SandlerM, HowardA, ZhuM, et al.. Mobilenetv2: inverted residuals and linear bottlenecks[C], 2018, New York, IEEE: 4510-4520

[16] HuJ, ShenL, SunG. Squeeze-and-excitation networks[C], 2018, New York, IEEE: 7132-7141

[17] XieS, GirshickR, DollárP, et al.. Aggregated residual transformations for deep neural networks[C], 2017, New York, IEEE: 1492-1500

[18] SzegedyC, LiuW, JiaY, et al.. Going deeper with convolutions [C], 2015, New York, IEEE: 1-9

[19] FRANKLE J, CARBIN M. The lottery ticket hypothesis: finding sparse, trainable neural networks[EB/OL]. (2018-03-09) [2022-03-13]. https://arxiv.org/abs/1803.03635v5.

[20] HINTON G, VINYALS O, DEAN J. Distilling the knowledge in a neural network[EB/OL]. (2015-03-09) [2022-03-13]. https://arxiv.org/abs/1503.02531.

[21] JacobB, KligysS, ChenB, et al.. Quantization and training of neural networks for efficient integer-arithmetic-only inference[C], 2018, New York, IEEE: 2704-2713

[22] HanK, WangY, TianQ, et al.. Ghostnet: more features from cheap operations[C], 2020, New York, IEEE: 1580-1589

[23] LiuX, LiuW, MeiT, et al.. A deep learning-based approach to progressive vehicle re-identification for urban surveillance[C], 2016, Cham, Springer: 869-884

[24] HeK, ZhangX, RenS, et al.. Deep residual learning for image recognition[C], 2016, New York, IEEE: 770-778

[25] HOWARD A G, ZHU M, CHEN B, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications[EB/OL]. (2017-04-17) [2022-03-13]. https://arxiv.org/abs/1704.04861.

[26] ZhangX, ZhouX, LinM, et al.. Shufflenet: an extremely efficient convolutional neural network for mobile devices[C], 2018, New York, IEEE: 6848-6856

[27] DuD, QiY, YuH, et al.. The unmanned aerial vehicle benchmark: object detection and tracking[C], 2018, Cham, Springer: 370-386

[28] DAI J, LI Y, HE K, et al. R-FCN: object detection via region-based fully convolutional networks [J]. Advances in neural information processing systems, 2016, 29.

[29] LiuW, AnguelovD, ErhanD, et al.. SSD: single shot multibox detector[C], 2016, Cham, Springer: 21-37

[30] RenS, HeK, GirshickR, et al.. Faster R-CNN: towards real-time object detection with region proposal networks[J]. Advances in neural information processing systems, 2015, 28: 91-99