Human interaction recognition based on sparse representation of feature covariance matrices

Abstract

A new method for interaction recognition based on sparse representation of feature covariance matrices was presented. Firstly, the dense trajectories (DT) extracted from the video were clustered into different groups to eliminate the irrelevant trajectories, which could greatly reduce the noise influence on feature extraction. Then, the trajectory tunnels were characterized by means of feature covariance matrices. In this way, the discriminative descriptors could be extracted, which was also an effective solution to the problem that the description of the feature second-order statistics is insufficient. After that, an over-complete dictionary was learned with the descriptors and all the descriptors were encoded using sparse coding (SC). Classification was achieved using multiple instance learning (MIL), which was more suitable for complex environments. The proposed method was tested and evaluated on the WEB Interaction dataset and the UT interaction dataset. The experimental results demonstrated the superior efficiency.

Keywords

interaction recognition / dense trajectory / sparse coding / MIL

References

[1] KongY, FuYun. Modeling supporting regions for close human interaction recognition [C]. //Computer Vision-ECCV 2014 Workshops, 2014, Springer International Publishing, Zurich: 2944

[2] KarungaruS, KenjiT, FukumiM. Human action recognition using normalized cone histogram features [C]. //Computational Intelligence for Multimedia, Signal and Vision Processing (CIMSIVP), 2014 IEEE Symposium on, 2014, IEEE, Orkand, FL: 15

[3] HoaiM, ZissermanA. Talking heads: detecting humans and recognizing their interactions [C]. //Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, 2014, IEEE, Columbus, Dhio: 875882

[4] YangL-y, GaoC-q, MengD-y, LuJiang. A novel group-sparsity-optimization-based feature selection model for complex interaction recognition [M]. //Computer Vision–ACCV 2014, 2015, Springer International Publishing, Singapore: 508521

[5] ZhangJ, LinH, NieW Z, ChaisornL, WongY K, KankanhalliM S. Human action recognition bases on local action attributes [J]. Journal of Electrical Engineering & Technology, 2015, 10(3): 1264-1274

[6] NowakE, JurieF, TriggsB. Sampling strategies for bag-of-features image classification [M]. Computer vision–ECCV 2006, 2006490503

[7] WangH, UllahM M, KläserA. Evaluation of local spatio-temporal features for action recognition [C]. //British Machine Vision Conference, 2009, Springer, London: 110

[8] WangH, KläserA, SchmidC. Dense trajectories and motion boundary descriptors for action recognition [J]. International Journal of Computer Vision, 2013, 103(1): 60-79

[9] HaoZ-b, ZhangQ-n, EzquierdoE. Human action recognition by fast dense trajectories [C]. //Proceedings of the 21st ACM international conference on Multimedia, 2013, ACM, Barcelona: 377380

[10] BeaudryC, PeteriR, MascarillaL. Action recognition in videos using frequency analysis of critical point trajectories [C]. //2014 IEEE International Conference on Image Processing (ICIP), 2014, IEEE, Paris: 14451449

[11] SeoJ J, BaddarW J, KimD H. Human action recognition using time-invariant key-trajectories describing spatio-temporal salient motion [C]. //IEEE International Conference on Image Processing, 2015, IEEE, Quebec City: 586590

[12] NiB-b, MoulinP, YangX-k. Motion Part Regularization: Improving action recognition via trajectory group selection [C]. //Computer Vision and Pattern Recognition, 2015, IEEE, Boston: 36983706

[13] ZhangB, RotaP, ConciN. Human interaction recognition in the wild: Analyzing trajectory clustering from multiple-instance-learning perspective [C]. //IEEE International Conference on Multimedia and Expo, 2015, IEEE, Torino: 16

[14] IosifidisA, TefasA, PitasI. Merging linear discriminant analysis with Bag of Words model for human action recognition [C]. //IEEE International Conference on Image Processing, 2015, IEEE, Quebec City: 832836

[15] ElguebalyT, BouguilaN. Improving codebook generation for action recognition using a mixture of Asymmetric Gaussians [C]. //Computational Intelligence for Multimedia, Signal and Vision Processing (CIMSIVP), 2014 IEEE Symposium on, 2014, IEEE, Orbando, FL: 17

[16] WangY-y, LiY-b, JiX-fei. Human action recognition based on global gist feature and local patch coding [J]. Management Review, 2015, 21(11): 38-43

[17] GuoK, IshwarP, KonradJ. Action recognition from video using feature covariance matrices [J]. IEEE Transactions on Image Processing, 2013, 22(6): 2479-2494

[18] BroxT, MalikJ. Object segmentation by long term analysis of point trajectories [C]. //Proc European Conference on Computer Vision, 2010, Springer, Crete, Greece: 282295

[19] SenerF, Ikizler-CinbisN. Two-person interaction recognition via spatial multiple instance embedding [J]. Journal of Visual Communication & Image Representation, 2015, 32: 63-73

[20] ChenY-x, BiJ-b, WangJ Z. MILES: Multiple-instance learning via embedded instance selection [J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2006, 28(12): 1931-1947

[21] GaoC-q, YangL-y, DuY-h. From constrained to unconstrained datasets: An evaluation of local action descriptors and fusion strategies for interaction recognition [J]. World Wide Web-internet & Web Information Systems, 2015, 19(2): 1-12

[22] XiaL-m, ShiX-t, TuH-bin. An approach for complex activity recognition by key frames [J]. Journal of Central South University, 2015, 22(9): 3450-3457