A rehabilitation training action evaluation method based on CTRAMM-VideoPose3D network and LTDTW matching algorithm

Abstract

In order to achieve the evaluation of human rehabilitation training movements, a human 3D pose estimation network integrating key-frame enhancement method (KFEM) and CTRAMM module is proposed, and a matching algorithm based on location and type dynamic time warping (LTDTW) is developed to evaluate rehabilitation movements. KFEM determines key-frames and adjusts their weights by calculating the coordinate transformation of human key-points. The CTRAMM module dynamically learns different topological structures, improving the feature representation ability of the model. The LTDTW improves the accuracy of sequence matching through adaptive weight coefficients. The experimental results on different datasets have validated the effectiveness of the proposed method.

Keywords

A

References

[1] YANG H, LU Y, ZHENG Z, et al. Reliability enhancement method of attitude estimation for wearable motion capture systems in human lower limb rehabilitation[J]. IEEE sensors journal, 2023.

[2] Ma J, Li S, Qin H, et al. . Self-adjustable hyper-graphs for video pose estimation based on spatial-temporal subspace construction. Science China. Information sciences, 2022, 65(3): 139101 J]

[3] Wu S, Chen H, Yin Y, et al. . Joint-motion mutual learning for pose estimation in video. Proceedings of the 32nd ACM International Conference on Multimedia, October 2024, Melbourne, Australia. 2024, New York, ACM8962-8971[C]

[4] Wang L, Huynh D Q, Koniusz P. A comparative review of recent kinect-based action recognition algorithms. IEEE transactions on image processing, 2019, 29: 15-28 J]

[5] Majumder S, Kehtarnavaz N. Vision and inertial sensing fusion for human action recognition: a review. IEEE sensors journal, 2021, 21(3): 2454-2467 J]

[6] NIU Z, LU K, XUE J, et al. Skeleton cluster tracking for robust multi-view multi-person 3D human pose estimation[J]. Computer vision and image understanding, 2024: 104059.

[7] Zeng A, Sun X, Yang L, et al. . Learning skeletal graph neural networks for hard 3D pose estimation. Proceedings of 2021 IEEE International Conference on Computer Vision, October 10–17, 2021, Montreal, QC, Canada. 2021, New York, IEEE1143611445[C]

[8] Li W, Du R, Chen S. Skeleton-based spatio-temporal U-network for 3D human pose estimation in video. Sensors, 2022, 22(7): 2573 J]

[9] Xia L, Chen C C, Aggarwal J K. View invariant human action recognition using histograms of 3D joints. Proceedings of 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, June 16–21, 2012, Providence, RI, USA. 2012, New York, IEEE20-27[C]

[10] Yu Y, Si X, Hu C, et al. . A review of recurrent neural networks: LSTM cells and network architectures. Neural computation, 2019, 31(7): 1235-1270 J]

[11] Usmani A, Siddiqui N, Islam S. Skeleton joint trajectories based human activity recognition using deep RNN. Multimedia tools and applications, 2023, 82(30): 46845-46869 J]

[12] Xu Y, Wang W, Liu T, et al. . Monocular 3D pose estimation via pose grammar and data augmentation. IEEE transactions on pattern analysis and machine intelligence, 2021, 44(10): 6327-6344 J]

[13] Yang W, Zhang J, Cai J, et al. . HybridNet: integrating GCN and CNN for skeleton-based action recognition. Applied intelligence, 2023, 53(1): 574-585 J]

[14] Xia H, Zhang T. Self-attention network for human pose estimation. Applied sciences, 2021, 11(4): 1826 J]

[15] Guan S, Lu H, Zhu L, et al. . AFE-CNN: 3D skeleton-based action recognition with action feature enhancement. Neurocomputing, 2022, 514: 256-267 J]

[16] Pavllo D, Feichtenhofer C, Grangier D, et al. . 3D human pose estimation in video with temporal convolutions and semi-supervised training. Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition, June 15–20, 2019, Long Beach, CA, USA. 2019, New York, IEEE7753-7762[C]

[17] Chen J, Samuel R D J, Poovendran P. LSTM with bio inspired algorithm for action recognition in sports videos. Image and vision computing, 2021, 112: 104214 J]

[18] Wu G, Wen C, Jiang H. Wushu movement recognition system based on DTW attitude matching algorithm. Entertainment computing, 2025, 52100877 J]

[19] Aldahoul N, Karim H A, Sabri A Q M, et al. . A comparison between various human detectors and CNN-based feature extractors for human activity recognition via aerial captured video sequences. IEEE access, 2022, 1063532-63553 J]

[20] Chen Y, Zhang Z, Yuan C, et al. . Channel-wise topology refinement graph convolution for skeleton-based action recognition. Proceedings of 2021 IEEE International Conference on Computer Vision, October 10–17, 2021, Montreal, QC, Canada. 2021, New York, IEEE1335913368[C]

[21] Zhao L, Peng X, Tian Y, et al. . Semantic graph convolutional networks for 3D human pose regression. Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition, June 15–20, 2019, Long Beach, CA, USA. 2019, New York, IEEE34253435[C]

RIGHTS & PERMISSIONS

Tianjin University of Technology