MiTAR: a study on human activity recognition based on NLP with microscopic perspective

Abstract

Nowadays, human activity recognition is becoming a more and more significant topic, and there is also a wide range of applications for it in real world scenarios. Sensor data is an important data source in engineering and application. At present, some studies have been carried out in the field of human activity recognition based on sensor data in a macroscopic perspective. However, many studies in this perspective face some limitations. One pivotal limitation is uncontrollable data segment length of different kinds of activities. Suitable feature and data form are also influencing factors. This paper carries out the study creatively on a microscopic perspective with an emphasis on the logic and relevance between data segments, attempting to apply the idea of natural language processing and the method of data symbolization to the study of human activity recognition and try to solve the problem above. In this paper, several activity-element definitions and three algorithms are proposed, including the algorithm of dictionary building, the algorithm of corpus building, and activity recognition algorithm improved from a natural language analysis method, TFIDF. Numerous experiments on different aspects of this model are taken. The experiments are carried out on six complex and representative single-level sensor datasets, namely UCI Sports and Daily dataset, Skoda dataset, WISDM Phoneacc dataset, WISDM Watchacc dataset, Healthy Older People dataset and HAPT dataset, which prove that this model can be applied to different datasets and achieve a satisfactory recognition result.

Keywords

human activity recognition / NLP / time series data analysis / data symbolization

References

[1] Lin W, Yang M, Wu J, Ke L, Xiong H. Action recognition with coarse-tofine deep feature integration and asynchronous fusion. In: Proceedings of the National Conference on Artificial Intelligence. 2018, 1–8

[2] Franco R, Facundo Q, Laura L, Cesar E. Distribution of action movements dam a descriptor for human action recognition. Frontiers of Computer Science, 2015, 9(6): 956–965

[3] Chen K, Ding G, Han J. Attribute-based supervised deep learning model for action recognition. Frontiers of Computer Science, 2017, 11(2): 219–229

[4] Wang J, Chen D, Yang J. Human behavior classification by analyzing periodic motions. Frontiers of Computer Science, 2010, 4(4): 580–587

[5] Bracciali A, Larsson E. Data-intensive modelling and simulation in life sciences and socio-economical and physical sciences. Data Science and Engineering, 2017, 2(3): 197–198

[6] Pan W, Li Z, Zhang Y, Weng C. The new hardware development trend and the challenges in data management and analysis. Data Science and Engineering, 2018, 3(3): 263–276

[7] Wu H, Pan W, Xiong X, Xu S. Human activity recognition based on the combined svm&hmm. In: Proceedings of IEEE International Conference on Information & Automation. 2014, 219–224

[8] Anguita D, Ghio A, Oneto L, Parra X, Reyes-Ortiz J L. Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In: Proceedings of International Conference on Ambient Assisted Living & Home Care. 2012, 216–223

[9] Krishnan R, Subedar M, Tickoo O. Bar: Bayesian activity recognition using variational inference. In: Proceedings of the 3rd Workshop on Bayesian Deep Learning. 2018, 1–8

[10] Dave V S, Zhang B, Chen P, Hasan M A. Neural-brane: neural Bayesian personalized ranking for attributed network embedding. Data Science and Engineering, 2019, 4(2): 119–131

[11] Yuan M, Chen E, Lei G. Posture selection based on two-layer AP with application to human action recognition using HMM. In: Proceedings of IEEE International Symposium on Multimedia. 2017, 359–364

[12] Ranjan N, Mundada K, Phaltane K, Ahmad S. A survey on techniques in NLP. International Journal of Computer Applications, 2016, 134: 6–9

[13] Altun K, Barshan B, Tunçel O. Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recognition, 2010, 43(10): 3605–3620

[14] Zappi P, Stiefmeier T, Farella E, Roggen D, Benini L, Troster G. Activity recognition from on-body sensors by classifier fusion: sensor scalability and robustness. In: Proceedings of International Conference on Intelligent Sensors. 2007, 281–286

[15] Roggen D, Calatroni A, Rossi M, Holleczek T, Förster K, Tröster G, Lukowicz P, Bannach D, Pirkl G, Ferscha A. Collecting complex activity data sets in highly rich networked sensor environments. In: Proceedings of the 7th International Conference on Networked Sensing Systems. 2010, 233–240

[16] Chavarriaga R, Sagha H, Calatroni A, Digumarti S, Tröster G, Millán J D R, Roggen D. The opportunity challenge: a benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters, 2013, 34(15): 2033–2042

[17] Xie X. Human action recognition in the range of Wi-Fi with CNN and ELM. Master Thesis, Beijing, University of Posts and Telecommunication, 2018

[18] Khanna R, Awad M. Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers. Berkeley California: Apress, 2015

[19] Bharti P, De D, Chellappan S, Das S K. Human: complex activity recognition with multi-modal multi-positional body sensing. IEEE Transactions on Mobile Computing, 2018, 18(4): 857–870

[20] Stolke A, Omohundro S. Hidden markrov model induction by Bayesian model merging. In: Proceedings of the 5th International Conference on Neural Information Processing Systems. 1992, 11–18

[21] Adil MK, Young-Koo L, Lee S Y, Tae-Seong K. A triaxial accelerometerbased physical-activity recognition via augmented-signal features and a hierarchical recognizer. IEEE Transactions on Information Technology in Biomedicine, 2010, 14(5): 1166–1172

[22] Greff K, Srivastava R K, Koutnik J, Steunebrink B R, Schmidhuber J. LSTM: a search space odyssey. IEEE Transactions on Neural Networks & Learning Systems, 2016, 28(10): 2222–2232

[23] Qi H, Fang K, Wu X, Xu L, Lang Q. Human activity recognition method based on molecular attributes. International Journal of Distributed Sensor Networks, 2019, 15(4): 1–13

[24] Ashish V, Noam S, Niki P, Jakob U, Llion J, Aidan N G, Lukasz K, Illia P. Attention is all you need. In: Proceedings of Annual Conference on Neural Information Processing Systems. 2017, 5998–6008

[25] Devlin J, Chang M, Lee K, Toutanova K. Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4171–4186

[26] Barshan B, Yüksek M C. Recognizing daily and sports activities in two open source machine learning environments using body-worn sensor units. Computer Journal, 2013, 57(11): 1649–1667

[27] Brena R, Garcia-Ceja E. A crowdsourcing approach for personalization in human activities recognition. Intelligent Data Analysis, 2017, 21: 721–738

[28] Wang X, Wang L, Lopes L. Unsupervised categorization of human motion sequences. Intelligent Data Analysis, 2013, 17(6): 1057–1074

[29] Wang X, Zhang B, Teng G, Sun Z, Wei J. Toward robust activity recognition: hierarchical classifier based on gaussian process. Intelligent Data Analysis, 2016, 20(3): 701–717

[30] Kantor P. Foundations of statistical natural language processing. Information Retrieval, 2001, 4(1): 80

[31] Khair E L, Ibrahim A. TF*IDF. Boston: Springer US, 2009

[32] Mika S, Schölkopf B, Smola A, Müller K R, Rätsch G. Kernel PCA and De-noising in Feature Spaces. In: Proceedings of the 12th Annual Conference on Neural Information Processing Systems II. 1999, 536–542

[33] Altun K, Barshan B. Human activity recognition using inertial/magnetic sensor units. Lecture Notes in Computer Science, 2010, 6219: 38–51

[34] Zappi P, Lombriser C, Stiefmeier T, Farella E, Roggen D, Benini L, Tröster G. Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection. Berlin: Springer Berlin Heidelberg, 2008

[35] Roggen D, Troster G. Fusion of string-matched templates forcontinuous activity recognition. In: Proceedings of IEEE International Symposium on Wearable Computers. 2007, 1–4

[36] Gary M W, Kenichi Y, Thaier H. Smartphone and smartwatch-based biometrics using activities of daily living. IEEE Access, 2019, 7: 133190–133202

[37] Wickramasinghe A, Ranasinghe D C, Fumeaux C, Hill K D, Visvanathan R. Sequence learning with passive rfid sensors for real-time bed-egress recognition in older people. IEEE Journal of Biomedical & Health Informatics, 2017, 21(4): 917–929

[38] Roberto S T, Renuka V, Stephen H, Anton V D H, Damith R. Effectiveness of a batteryless and wireless wearable sensor system for identifying bed and chair exits in healthy older people. Sensors, 2016, 16(4): 546–562

[39] Wickramasinghe A, Ranasinghe D C. Recognisingactivities in real time using body worn passive sensors with sparse data streams: to interpolate or not to interpolate? In: Proceedings of the 12th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. 2015, 21–30

[40] Roberto S T, Damith R, Shi Q. Evaluation of wearable sensor tag data segmentation approaches for real time activity classification in elderly. Springer International Publishing, 2014, 131: 384–395

[41] Sample A P, Roberto S T, Ranasinghe D C, Shi Q. Sensor enabled wearable rfid technology for mitigating the risk of falls near beds. In: Proceed ings of IEEE International Conference on RFID. 2013, 191–198

[42] Reyes-Ortiz J L, Oneto L, Sama A, Parra X, Anguita D. Transition-aware human activity recognition using smartphones. Neurocomputing, 2016, 171: 754–767

[43] Trabelsi D, Mohammed S, Amirat Y, Oukhellou L. Activity recognition using body mounted sensors: an unsupervised learning based approach. In: Proceedings of International Joint Conference on Neural Networks. 2012, 1–7

[44] Subasi A. Eeg signal classification using wavelet feature extraction and a mixture of expert model. Expert Systems with Applications, 2007, 32(4): 1084–1093

[45] Li B, Aleksandr D, Gue Y, Liu T, Satoshi M, Du X. Scaling word2vec on big corpus. Data Science and Engineering, 2019, 4(2): 157–175

RIGHTS & PERMISSIONS

Higher Education Press