Complex human activities recognition using interval temporal syntactic model

Abstract

A novel method based on interval temporal syntactic model was proposed to recognize human activities in video flow. The method is composed of two parts: feature extract and activities recognition. Trajectory shape descriptor, speeded up robust features (SURF) and histograms of optical flow (HOF) were proposed to represent human activities, which provide more exhaustive information to describe human activities on shape, structure and motion. In the process of recognition, a probabilistic latent semantic analysis model (PLSA) was used to recognize sample activities at the first step. Then, an interval temporal syntactic model, which combines the syntactic model with the interval algebra to model the temporal dependencies of activities explicitly, was introduced to recognize the complex activities with a time relationship. Experiments results show the effectiveness of the proposed method in comparison with other state-of-the-art methods on the public databases for the recognition of complex activities.

Keywords

trajectory shape descriptor / speeded up robust features (SURF) / histograms of optical flow (HOF) / PLSA probabilistic latent semantic analysis / syntactic model

References

[1] KumariS, MitraS K. Human action recognition using DFT[C]. IEEE National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics, 2011239-242

[2] LaptevI. On space-time interest points [J]. International Journal of Computer Vision, 2005, 1(2/3): 432-439

[3] DollarP, RabaudV, CottrellG, BelongieS. Behavior recognition via sparse spatio-temporal features. IEEE International Workshop on Visual Surveillance & Performance Evaluation of Tracking & Surveillance, 200565-72

[4] LaptevI, MarszalekM, SchmidC, RozenfeldB. Learning realistic human actions from movies [C]// Conference on Computer Vision and Pattern Recognition. Anchorage: IEEE, 20081-8

[5] SchuldtC, LaptevI C B. Recognizing human actions: A local SVM approach [C]// International Conference on Pattern Recognition. Cambridge: IEEE, 200432-36

[6] ScovannerP, AliS, ShahM. A 3-dimensional sift descriptor and its application to action recognition [C]// Proceedings of the 15th International Conference on Multimedia. NY: ACM, 2007357-360

[7] KlaserA, MarszalekM, SchmidC. A spatio-temporal descriptor based on 3d-gradients. Proceedings of the 19th British Machine Vision Conference, 2008, 275: 1-10

[8] EricC, VincentR, AndrewZ. Match-time covariance for descriptor. Proceedings of the 24th British Machine Vision Conference, 2013270-281

[9] BregonzioM, LiJ, GongS, TaoX. Discriminative topics modeling for action feature selection and recognition. Proceedings of the 21th British Machine Vision Conference, 20101-11

[10] DaiP, DiH-j, DongL-g, TaoL-mi. Group interaction analysis in dynamic context [J]. IEEE Transactions On Systems, Man and Cybernetics—Part B, 2009, 39(1): 34-42

[11] ZhangY, ZhangY, SwearsE, LariosN, WangZH, JiQ. Modeling temporal interactions with interval temporal bayesian networks for complex activity recognition [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(10): 2468-2483

[12] DamenD, HoggD. Recognizing linked events: Searching the space of feasible explanations. Conference on Computer Vision and Pattern Recognition, 2009927-934

[13] ZhangJ, GongS. Action categorization by structural probabilistic latent semantic analysis [J]. Computer Vision and Image Understanding, 2010, 114(8): 857-864

[14] ArsicD, SchullerB. Real time person tracking and behavior interpretation in multi-camera scenarios applying homography and coupled HMMs. International Conference on Analysis of Verbal and Nonverbal Communication and Enactment, 20111-18

[15] SugimotoM, ZinT T, ToriuT, NakajimaS. Robust rule-based method for human activity recognition [J]. International Journal of Computer Science and Network Security, 2011, 11(4): 37-43

[16] Fernandez-CaballeroA, CastilloJ C R, SanchezJ M. Human activity monitoring by local and global finite state machines [J]. Expert Systems with Applications, 2012, 39(8): 6982-6993

[17] HuangJ-xiaComplex human activity recognition based on SCFG [D], 2013ChangshaCentral South University

[18] XiaL-m, ShiX-t, TuH-bin. An approach for complex activity recognition by key frames [J]. Journal of Central South University, 2015, 22(9): 3450-3457

[19] ShiJ, TomasiC. Good features to track. Conference on Computer Vision and Pattern Recognition, 1994593-600

[20] XiaL-m, YangB-j, TuH-bing. Recognition of suspicious behavior using case-based reasoning [J]. Journal of Central South University, 2015, 22(1): 241-250

[21] AllenJ F. Maintaining knowledge about temporal intervals [J]. Communications of the ACM, 1983, 26(11): 832-843

[22] GrunwaldP. A minimum description length approach to grammar inference. Connectionist, Statistical and Symbolic Approaches to Learning for Natural Language Processing, 1996203-216

[23] ZhangZ, TanT, HnangK. An extended grammar system for learning and recognizing complex visual events [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(2): 240-255

[24] YouTube action dataset [EB/OL]. 2011-11-01. http://www.cs.ucf. edu/liujg/ YouTubeActiondataset.html.

[25] UCF sports action dataset [EB/OL]. 2012-02-01. http://vision.eecs. ucf.edu/datasetsActions.html.