An approach for complex activity recognition by key frames

Abstract

A new method for complex activity recognition in videos by key frames was presented. The progressive bisection strategy (PBS) was employed to divide the complex activity into a series of simple activities and the key frames representing the simple activities were extracted by the self-splitting competitive learning (SSCL) algorithm. A new similarity criterion of complex activities was defined. Besides the regular visual factor, the order factor and the interference factor measuring the timing matching relationship of the simple activities and the discontinuous matching relationship of the simple activities respectively were considered. On these bases, the complex human activity recognition could be achieved by calculating their similarities. The recognition error was reduced compared with other methods when ignoring the recognition of simple activities. The proposed method was tested and evaluated on the self-built broadcast gymnastic database and the dancing database. The experimental results prove the superior efficiency.

Keywords

human activity recognition / complex activity segmentation / key frame extraction

References

[1] AggarwalJ K, RyooM S. Human activity analysis: A review [J]. ACM Computing Surveys (CSUR), 2011, 43(3): 16-62

[2] OliverN, HorvitzE, GargA. Layered representations for human activity recognition [C]. Fourth IEEE International Conference on Multimodal Interfaces, 2002Pittsburgh, PA, USAIEEE Computer Science3-8

[3] KhoshhalK, AliakbarpourH, MekhnachaK, JulienR S, JoaoQ, JorgeD. Lma-based human behaviour analysis using HMM [C]. 2nd IFIP WG 5.5/SOCOLNET Doctoral Conference on Computing, Electrical and Industrial Systems, 2011Costa de Caparica, PortugalSpringer-Verlag189-196

[4] MozafariK, CharkariN M, BoroujeniH S, BehrouzifarM. A novel fuzzy HMM approach for human action recognition in video [C]. 3rd Conference of Knowledge Technology Week, 2012Kajang, MalaysiaSpringer-Verlag184-193

[5] DaiP, DiH-j, DongL-g, TaoL-m. Group interaction analysis in dynamic context [J]. IEEE Transactions On Systems, Man and Cybernetics—Part B, 2009, 39(1): 34-42

[6] DamenD, HoggD. Recognizing linked events: Searching the space of feasible explanations [C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009Miami, Florida, USAIEEE Computer Science927-934

[7] ArsicD, SchullerB. Real time person tracking and behavior interpretation in multi-camera scenarios applying homography and coupled HMMs [C]. 012 International Conference on Analysis of Verbal and Nonverbal Communication and Enactment, 2011Budapest, HungarySpringer-Verlag1-18

[8] JooS W, ChellappaR. Attribute grammar-based event recognition and anomaly detection [C]. IEEE Conference on Computer Vision and Pattern Recognition Workshop (CVPRW), 2006New YorkIEEE Computer Science107-107

[9] Fernandez-CaballeroA, CastilloJ C, RodriguezsanchezJ M. Human activity monitoring by local and global finite state machines [J]. Expert Systems with Applications, 2012, 39(8): 6982-6993

[10] SanromaG, BurghoutsG, SchutteK. Recognition of long-term behaviors by parsing sequences of short-term actions with a stochastic regular grammar [C]. Proceedings of the 2012 Joint IAPR International Conference on Structural, Syntactic, and Statistical Pattern Recognition (SSPR’12/SPR’12), 2012Berlin HeidelbergSpringer-Verlag225-233

[11] SanromaG, PatinoL, BurghoutsG, SchutteK, FerrymanJ. A unified approach to the recognition of complex actions from sequences of zone-crossings [J]. Image and Vision Computing, 2014, 32(5): 363-378

[12] NieblesJ C, WangH-c, LiF-f. Unsupervised learning of human action categories using spatial-temporal words [J]. International Journal of Computer Vision, 2008, 79(3): 299-318

[13] HamidR, MaddiS, JohnsonA, BobickA, EssaI, IsbellC. A novel sequence representation for unsupervised analysis of human activities [J]. Artificial Intelligence, 2009, 173(14): 1221-1244

[14] KooijJ F P, EnglebienneG, GavrilaD M. A non-parametric hierarchical model to discover behavior dynamics from tracks [C]. 12th European Conference on Computer Vision (ECCV), 2012Florence, ItalySpringer-Verlag270-283

[15] OuananeA, SerirA, KerouhF. New geometric descriptor for the recognition of aggressive human behavior [C]. Fifth IEEE International Congress on Image and Signal Processing (CISP), 2012MoroccoIEEE Computer Science148-153

[16] ZivkovicZ H, FerdinandV D. Efficient adaptive density estimation per image pixel for the task of background subtraction [J]. Pattern Recognition Letters, 2006, 27(7): 773-780

[17] HuoY, ZhangP, WangY. Adaptive threshold video shot boundary detection algorithm based on progressive bisection strategy [J]. Journal of Information & Computational Science, 2014, 11(2): 391-403

[18] XiaL-m, DengK-j. The key frame extraction based on self-splitting competitive learning algorithm [J]. Computer Engineering and Applications, 2011, 47(2): 146-148

[19] LiW-w, LiuC-p, WangZ-h, ZhangS-k. Fuzzy C-means image classification method based on SSCL [J]. Journal of Image and Graphics, 2011, 16(2): 215-220

[20] XiaL-m, HuangJ-x, TanL-z. Human action recognition based on chaotic invariants [J]. Journal of Central South University, 2013, 20: 3171-3179