A Novel Deep Learning Framework for Location Information Assisted Complex Human Activity Recognition

Abstract

With the popularization of smart living and the rapid development of wearable terminal technology in recent years, sensor-based human activity recognition(HAR) has attracted widespread attention and has significant academic research and commercial application value. This paper focuses on enhancing the HAR model's recognition of users' daily simple activities(SAs) and complex activities(CAs), and proposes a deep learning(DL) model. Firstly, two publicly available datasets, UCI HAR and Shoaib CHA, are normalized. Then the characteristics of distinct activities are retrieved by the proposed model for HAR. Given the high association between users' activities and locations, location information is integrated into the dataset by the one-hot encoding technique to boost the model's classification performance. In addition, the proposed DL model is evaluated against eight traditional machine learning(ML) algorithms and six DL algorithms. Finally, the effect of various types of activities on the HAR model's recognition ability is studied. The experimental findings reveal that the proposed model achieves the highest classification accuracy on UCI HAR and Shoaib CHA datasets, with 96.77% and 99.13%, respectively. The classification accuracy of the HAR model is also greatly enhanced for both SAs and CAs by adding location information to the datasets.

Keywords

human activity recognition(HAR) / machine learning(ML) / deep learning(DL) / wearable sensor / convolutional neural network / long short-term memory(LSTM) neural network

References

[1] ANDRADE-AMBRIZ Y A, LEDESMA S, IBARRA-MANZANO M A, et al. Human activity recognition using temporal convolutional neural network architecture[J]. Expert Systems with Applications, 2022,191:116287.

[2] LIN G P, JIANG W W, XU S C, et al. Human activity recognition using smartphones with WiFi signals[J]. IEEE Transactions on Human-Machine Systems, 2023, 53(1):142-153.

[3] XU S G, ZHANG L, HUANG W B, et al. Deformable convolutional networks for multimodal human activity recognition using wearable sensors[J]. IEEE Transactions on Instrumentation and Measurement, 2022,71:1-14.

[4] LI Y, YANG G C, SU Z D, et al. Human activity recognition based on multienvironment sensor data[J]. Information Fusion, 2023,91:47-63.

[5] TANG B, GUAN W. CNN multi-position wearable sensor human activity recognition used in basketball training[J]. Computational Intelligence and Neuroscience, 2022,2022:9918143.

[6] MEKRUKSAVANICH S, JANTAWONG P, HNOOHOM N, et al. Badminton activity recognition and player assessment based on motion signals using deep residual network[C]//2022 IEEE 13th International Conference on Software Engineering and Service Science (ICSESS). New York: IEEE, 2022:80-83.

[7] DASKALOS A C, THEODOROPOULOS P, SPANDONIDIS C, et al. Wearable device for observation of physical activity with the purpose of patient monitoring due to COVID-19[J]. Signals, 2022, 3(1):11-28.

[8] NADEEM A, MEHMOOD A, RIZWAN K. A dataset build using wearable inertial measurement and ECG sensors for activity recognition,fall detection and basic heart anomaly detection system[J]. Data in Brief, 2019,27:104717.

[9] PENG L Y, CHEN L, WU X J, et al. Hierarchical complex activity representation and recognition using topic model and classifier level fusion[J]. IEEE Transactions on Biomedical Engineering, 2017, 64(6):1369-1379.

[10] WU S Y, FAN H H. Activity-based proactive data management in mobile environments[J]. IEEE Transactions on Mobile Computing, 2010, 9(3):390-404.

[11] THAKUR D, BISWAS S. Smartphone based human activity monitoring and recognition using ML and DL:a comprehensive survey[J]. Journal of Ambient Intelligence and Humanized Computing, 2020, 11(11):5433-5444.

[12] ABD RAHIM K N K, ELAMVAZUTHI I, IZHAR L, et al. Classification of human daily activities using ensemble methods based on smartphone inertial sensors[J]. Sensors, 2018, 18(12):4132.

[13] JAIN A, KANHANGAD V. Human activity classification in smartphones using accelerometer and gyroscope sensors[J]. IEEE Sensors Journal, 2018, 18(3):1169-1177.

[14] GUPTA S. Deep learning based human activity recognition (HAR) using wearable sensor data[J]. International Journal of Information Management Data Insights, 2021, 1(2):100046.

[15] MEKRUKSAVANICH S, JITPATTANAKUL A, SITTHITHAKERNGKIET K, et al. ResNet-SE:channel attention-based deep residual network for complex activity recognition using wrist-worn wearable sensors[J]. IEEE Access, 2022,10:51142-51154.

[16] PENG L Y, CHEN L, WU M H, et al. Complex activity recognition using acceleration,vital sign,and location data[J]. IEEE Transactions on Mobile Computing, 2019, 18(7):1488-1498.

[17] ANGUITA D, GHIO A, ONETO L, et al. A public domain dataset for human activity recognition using smartphones[C]//The European Symposium on Artificial Neural Networks (ESANN). Bruges: [s.n.], 2013,437-442.

[18] SHOAIB M, BOSCH S, INCEL O, et al. Complex human activity recognition using smartphone and wrist-worn motion sensors[J]. Sensors, 2016, 16(4):426.

[19] CRISTIANINI N, SHAWE-TAYLOR J. An introduction to support vector machines:and other kernel-based learning methods[M]. Cambridge: Cambridge University Press, 2000.

[20] QUINLAN J R. Improved use of continuous attributes in C4.5[J]. Journal of Artificial Intelligence Research, 1996,4:77-90.

[21] RUSSELL S J. Artificial intelligence a modern approach[M]. New York: Pearson Education, Inc., 2010.

[22] AHA D W, KIBLER D, ALBERT M K. Instance-based learning algorithms[J]. Machine Learning, 1991, 6(1):37-66.

[23] ROBBINS H, MONRO S. A stochastic approximation method[J]. The Annals of Mathematical Statistics, 1951, 22(3):400-407.

[24] KUTNER M H. Applied linear statistical models[M]. 5th ed. Boston: McGraw-Hill Irwin, 2005.

[25] BREIMAN L. Bagging predictors[J]. Machine Learning, 1996, 24(2):123-140.

[26] FRIEDMAN J H. Greedy function approximation:a gradient boosting machine[J]. The Annals of Statistics, 2001, 29(5):1189-1232.

[27] BARUT O, ZHOU L, LUO Y. Multitask LSTM model for human activity recognition and intensity estimation using wearable sensor data[J]. IEEE Internet of Things Journal, 2020, 7(9):8760-8768.

[28] BALOCH Z, SHAIKH F K, ALI UNAR M. CNN-LSTM-based late sensor fusion for human activity recognition in big data networks[J]. Wireless Communications and Mobile Computing, 2022,2022:3434100.

[29] AHMAD Z, KHAN N. CNN-based multistage gated average fusion (MGAF) for human action recognition using depth and inertial sensors[J]. IEEE Sensors Journal, 2021, 21(3):3623-3634.

[30] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2016:770-778.

[31] LU L M, ZHANG C L, CAO K, et al. A multichannel CNN-GRU model for human activity recognition[J]. IEEE Access, 2022,10:66797-66810.

[32] WANG Y, XU H J, LIU Y X, et al. A novel deep multifeature extraction framework based on attention mechanism using wearable sensor data for human activity recognition[J]. IEEE Sensors Journal, 2023, 23(7):7188-7198.

[33] CHALLA S K, KUMAR A, SEMWAL V B. A multibranch CNN-BiLSTM model for human activity recognition using wearable sensor data[J]. The Visual Computer, 2022, 38(12):4095-4109.

[34] MEKRUKSAVANICH S, JITPATTANAKUL A. Deep convolutional neural network with RNNs for complex activity recognition using wrist-worn wearable sensor data[J]. Electronics, 2021, 10(14):1685.

[35] ALO U R, NWEKE H F, TEH Y W, et al. Smartphone motion sensor-based complex human activity identification using deep stacked autoencoder algorithm for enhanced smart healthcare system[J]. Sensors, 2020, 20(21):6300.

Funding

National Natural Science Foundation of China(62371118)

National Natural Science Foundation of China(6210020445)

National Natural Science Foundation of China(61901104)

Natural Science Foundation of Shanghai, China(21ZR1446900)

Natural Science Foundation of Shanghai, China(21511100102)

Science and Technology Research Project of Shanghai Songjiang District, China(20SJKJGG4C)