Vision-Based Fall Detector for Elderly Based on Sliding Window Approach and Feature Engineering

Abstract

A new vision-based fall detector is proposed that uses the tsfresh tool to generate features from the bounding box motion parameters of an object and performs classification in a sliding window mode. The efficiency of the generated features is demonstrated compared to the primary ones. Using the auto-sklearn library and a generalized dataset compiled from the UR Fall Detection and CAUCAFall datasets, the best human fall detection model is found. This model based on a gradient boosting classifier achieved 96% accuracy, which is not inferior to well-known detection algorithms, but uses only two primary motion parameters to generate secondary features. A principal component analysis-based class separability study showed that for secondary features 99% of the variance is captured by the first 4 principal components, while for primary features, the first 10 principal components contain only 80% of the data variance. Furthermore, the processing time for generating secondary features and making predictions was found to be relatively short, taking only a few seconds per sequence, highlighting the practical applicability of the proposed approach in real-time fall monitoring systems.

Keywords

machine learning / sliding window / human fall detection / classification model / tsfresh / auto-sklearn

Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.

Data Availability Statement

Data available on request from the corresponding author upon reasonable request.

Author Contribution Statement

Boris Assanovich: Conceptualization, Methodology, Validation, Investigation, Resources, Writing - original draft, Writing - review & editing, Visualization, Project administration. Katsiaryna Kosarava: Methodology, Software, Validation, Formal analysis, Investigation, Data curation, Writing - review & editing, Visualization.

References

[1] World Health Organization. (2021). Falls. Retrieved from: https://www.who.int/news-room/fact-sheets/detail/falls

[2] Gutiérrez J., Rodríguez V., & Martin S. (2021). Comprehensive review of vision-based fall detection systems. Sensors, 21(3), 947. https://doi.org/10.3390/s21030947

[3] Alam E., Sufian A., Dutta P., & Leo M. (2022). Vision-based human fall detection systems using deep learning: A review. Computers in Biology and Medicine, 146, 105626. https://doi.org/10.1016/j.compbiomed.2022.105626

[4] Redmon J., Divvala S., Girshick R., & Farhadi A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 779-788.

[5] Lezzar F., Benmerzoug D., & Kitouni I. (2020). Camera-based fall detection system for the elderly with occlusion recognition. Applied Medical Informatics, 42(3), 169-179.

[6] Charfi I., Miteran J., Dubois J., Atri M., & Tourki R. (2012). Definition and performance evaluation of a robust SVM based fall detection solution. In 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems, 218-224. https://doi.org/10.1109/SITIS.2012.155

[7] Charfi I., Miteran J., Dubois J., Atri M.,& Tourki R.(2013). Optimized spatio-temporal descriptors for real-time fall detection: Comparison of support vector machine and Adaboost-based classification. Journal of Electronic Imaging, 22(4), 041106.

[8] Beddiar D. R., Oussalah M., & Nini B. (2022). Fall detection using body geometry and human pose estimation in video sequences. Journal of Visual Communication and Image Representation, 82, 103407.

[9] Kwolek B., & Kepski M. (2014). Human fall detection on embedded platform using depth maps and wireless accelerometer. Computer Methods and Programs in Biomedicine, 117(3), 489-501. http://doi.org/10.1016/j.cmpb.2014.09.005

[10] Zhang J., Wu C., & Wang Y. (2020). Human fall detection based on body posture spatio-temporal evolution. Sensors, 20(3), 946. https://doi.org/10.3390/s20030946

[11] Zhao D., Song T., Gao J., Li D., & Niu Y. (2024). YOLO-fall: A novel convolutional neural network model for fall detection in open spaces. IEEE Access, 12, 26137-26149. http://doi.org/10.1109/ACCESS.2024.3362958

[12] Yhdego H., Paolini C., & Audette M. (2023). Toward real-time, robust wearable sensor fall detection using deep learning methods: A feasibility study. Applied Sciences, 13(8), 4988. https://doi.org/10.3390/app13084988

[13] Shukla K., Holderbaum W., Theodoridis T., & Wei G.(2024). Enhancing gearbox fault diagnosis through advanced feature engineering and data segmentation techniques. Machines, 12(4), 261.

[14] Kosarava K., & Assanovich B. (2021). A simple indoor fall control system for the elderly based on the analysis of object bounding box parameters. In Proceedings of the International Conference on Pattern Recognition and Information Processing,92-96.

[15] tsfresh. (n.d.). Introduction - Why tsfresh? Retrieved from: https://tsfresh.readthedocs.io/en/latest/text/introduction.html

[16] Kausar F., Awadalla M., Mesbah M., & AlBadi T. (2022). Automated machine learning based elderly fall detection classification. Procedia Computer Science, 203,16-23. https://doi.org/10.1016/j.procs.2022.07.005

[17] Reimann H., & Bruijn S. M. (2024). The condition for dynamic stability in humans walking with feedback control. PLOS Computational Biology, 20(3), e1011861. http://doi.org/10.1371/journal.pcbi.1011861

[18] Kamberaj H. (2021). Classical mechanics. Germany: De Gruyter.

[19] Gahramanova A. (2019). Locating centers of mass with image processing. Undergraduate Journal of Mathematical Modeling: One+ Two, 10(1), 1. http://doi.org/10.5038/2326-3652.10.1.4906

[20] Eraso J. C., Mu˜noz E., Mu˜noz M., & Pinto J. (2022). Dataset CAUCAFall [Data set]. Mendeley Data. https://library.unimelb.edu.au/recite/referencing-styles/apa7#datasets

[21] tsfresh. (n.d.). Overview on extracted features. Retrieved from: https://tsfresh.readthedocs.io/en/latest/text/list_of_features.html

[22] Feurer M., Eggensperger K., Falkner S., Lindauer M., & Hutter F. (2022). Auto-sklearn 2.0: Hands-free AutoML via meta-learning. Journal of Machine Learning Research, 23(261), 1-61.

[23] Breiman L. (2001). Random forests. Machine Learning, 45(1), 5-32.

[24] Nishiyama D., Arita S., Fukui D., Yamanaka M., & Yamada H.(2024). Accurate fall risk classification in elderly using one gait cycle data and machine learning. Clinical Biomechanics, 115, 106262. https://doi.org/10.1016/j.clinbiomech.2024.106262

[25] Rodriguez J., Mercuri M., Karsmakers P., Soh P. J., Leroux P., & Schreurs D. (2013). Automatic fall detector based on sliding window principle. In 34th WIC Symposium on Information Theory in the Benelux and the Third joint WIC/IEEE SP Symposium on Information Theory and Signal Processing in the Benelux, 215-219.