A gamified virtual reality and inertial measurement unit-based framework for fine-grained upper limb motor assessment in stroke patients

Abstract

Stroke often leads to upper limb motor impairments, underscoring the need for precise assessment to guide personalized rehabilitation. Conventional clinical scales are limited by subjectivity and the absence of detailed kinematic analysis. To address this, we propose a novel assessment framework that integrates gamified virtual reality tasks with inertial measurement unit (IMU)–based kinematic analysis, enabling fine-grained and autonomous evaluation of upper limb movements in stroke patients. Specifically, we introduce a region-based motion normalcy index (rMNI) to quantify motor deficits across five spatial regions, offering a more nuanced characterization of movement impairments. Regression models, including elastic net, ridge, and least absolute shrinkage and selection operator regression, were trained on regional rMNI features to predict Fugl–Meyer assessment upper extremity (FMA-UE) scores. Experiments with 12 stroke patients and 8 healthy controls demonstrated strong correlations between rMNI and both FMA-UE total and subscale scores (|r| > 0.70), highlighting the ability of rMNI to spatially resolve motor dysfunction and identify impaired limbs. The best-performing regression model achieved an R² of 0.90 and a Pearson's correlation coefficient of 0.95, indicating excellent predictive validity. These results suggest that the proposed framework is a promising tool for personalized rehabilitation, providing both fine-grained spatial assessment and patient-specific insights.

Keywords

gamified virtual reality / inertial measurement unit / motor function assessment / stroke / upper limb

References

[1] Virani SS, Alonso A, Benjamin EJ, et al. Heart disease and stroke statistics-2020 update: a report from the American Heart Association. Circulation. 2020; 141(9): e139-e596. https://doi.org/10.1161/cir.0000000000000757

[2] Feigin VL, Stark BA, Johnson CO, et al. Global, regional, and national burden of stroke and its risk factors, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet Neurol. 2021; 20(10): 795-820. https://doi.org/10.1016/s1474-4422(21)00252-0

[3] Mollà-Casanova S, Llorens R, Borrego A, Salinas-Martínez B, Serra-Añó P. Validity, reliability, and sensitivity to motor impairment severity of a multi-touch app designed to assess hand mobility, coordination, and function after stroke. J NeuroEng Rehabil. 2021; 18(1):70. https://doi.org/10.1186/s12984-021-00865-9

[4] See J, Dodakian L, Chou C, et al. A standardized approach to the Fugl-Meyer assessment and its implications for clinical trials. Neurorehabilitation Neural Repair. 2013; 27(8): 732-741. https://doi.org/10.1177/1545968313491000

[5] Gladstone DJ, Danells CJ, Black SE. The Fugl-Meyer assessment of motor recovery after stroke: a critical review of its measurement properties. Neurorehabilitation Neural Repair. 2002; 16(3): 232-240. https://doi.org/10.1177/154596802401105171

[6] Tang L, Halloran S, Shi JQ, Guan Y, Cao C, Eyre J. Evaluating upper limb function after stroke using the free-living accelerometer data. Stat Methods Med Res. 2020; 29(11): 3249-3264. https://doi.org/10.1177/0962280220922259

[7] Chen ZJ, He C, Xia N, et al. Association between finger-to-nose kinematics and upper extremity motor function in subacute stroke: a principal component analysis. Front Bioeng Biotechnol. 2021; 9:660015. https://doi.org/10.3389/fbioe.2021.660015

[8] Bustrén EL, Sunnerhagen KS, Alt Murphy M. Movement kinematics of the ipsilesional upper extremity in persons with moderate or mild stroke. Neurorehabilitation Neural Repair. 2017; 31(4): 376-386. https://doi.org/10.1177/1545968316688798

[9] Centers for Disease Control and Prevention CDC. Outpatient rehabilitation among stroke survivors–21 States and the District of Columbia, 2005. MMWR Morb Mortal Wkly Rep. 2007; 56(20): 504-507.

[10] Villepinte C, Verma A, Dimeglio C, De Boissezon X, Gasq D. Responsiveness of kinematic and clinical measures of upper-limb motor function after stroke: a systematic review and meta-analysis. Ann Phys Rehabil Med. 2021; 64(2):101366. https://doi.org/10.1016/j.rehab.2020.02.005

[11] Oubre B, Lee SI. Detection and assessment of point-to-point movements during functional activities using deep learning and kinematic analyses of the stroke-affected wrist. IEEE J Biomed Health Inform. 2023; 28(2): 1022-1030. https://doi.org/10.1109/jbhi.2023.3337156

[12] Cerfoglio S, Capodaglio P, Rossi P, et al. Evaluation of upper body and lower limbs kinematics through an IMU-based medical system: a comparative study with the optoelectronic system. Sensors. 2023; 23(13):6156. https://doi.org/10.3390/s23136156

[13] Oubre B, Daneault JF, Jung HT, et al. Estimating upper-limb impairment level in stroke survivors using wearable inertial sensors and a minimally-burdensome motor task. IEEE Trans Neural Syst Rehabil Eng. 2020; 28(3): 601-611. https://doi.org/10.1109/tnsre.2020.2966950

[14] Maceira-Elvira P, Popa T, Schmid AC, Hummel FC. Wearable technology in stroke rehabilitation: towards improved diagnosis and treatment of upper-limb motor impairment. J NeuroEng Rehabil. 2019; 16(1):142. https://doi.org/10.1186/s12984-019-0612-y

[15] Bhagubai MMC, Wolterink G, Schwarz A, Held JPO, Van Beijnum BF, Veltink PH. Quantifying pathological synergies in the upper extremity of stroke subjects with the use of inertial measurement units: a pilot study. IEEE J Transl Eng Health Med. 2021; 9:2100211. https://doi.org/10.1109/jtehm.2020.3042931

[16] Ueyama Y, Takebayashi T, Takeuchi K, et al. Attempt to make the upper-limb item of objective Fugl-Meyer assessment using 9-Axis motion sensors. Sensors. 2023; 23(11):5213. https://doi.org/10.3390/s23115213

[17] Yu L, Xiong D, Guo L, Wang J. A remote quantitative Fugl-Meyer assessment framework for stroke patients based on wearable sensor networks. Comput Methods Progr Biomed. 2016; 128: 100-110. https://doi.org/10.1016/j.cmpb.2016.02.012

[18] Saposnik G, Cohen LG, Mamdani M, et al. Efficacy and safety of non-immersive virtual reality exercising in stroke rehabilitation (EVREST): a randomised, multicentre, single-blind, controlled trial. Lancet Neurol. 2016; 15(10): 1019-1027. https://doi.org/10.1016/s1474-4422(16)30121-1

[19] Ali SG, Wang X, Li P, et al. A systematic review: virtual-reality-based techniques for human exercises and health improvement. Front Public Health. 2023; 11:1143947. https://doi.org/10.3389/fpubh.2023.1143947

[20] Shah SHH, Karlsen AST, Solberg M, Hameed IA. A social VR-based collaborative exergame for rehabilitation: codesign, development and user study. Virtual Real. 2023; 27(4): 3403-3420. https://doi.org/10.1007/s10055-022-00721-8

[21] Sporn S, Coll M, Bestmann S, Ward NS. Chronic stroke survivors underestimate their upper limb motor ability in a simple 2D motor task. J NeuroEng Rehabil. 2024; 21(1):175. https://doi.org/10.1186/s12984-024-01471-1

[22] Demers M, Levin MF. Kinematic validity of reaching in a 2D virtual environment for arm rehabilitation after stroke. IEEE Trans Neural Syst Rehabil Eng. 2020; 28(3): 679-686. https://doi.org/10.1109/tnsre.2020.2971862

[23] Arlati S, Keijsers N, Paolini G, Ferrigno G, Sacco M. Kinematics of aimed movements in ecological immersive virtual reality: a comparative study with real world. Virtual Real. 2022; 26(3): 1-17. https://doi.org/10.1007/s10055-021-00603-5

[24] Kim WS, Cho S, Baek D, Bang H, Paik NJ. Upper extremity functional evaluation by Fugl-Meyer assessment scoring using depth-sensing camera in hemiplegic stroke patients. PLoS One. 2016; 11(7):e0158640. https://doi.org/10.1371/journal.pone.0158640

[25] Burton Q, Lejeune T, Dehem S, et al. Performing a shortened version of the Action Research Arm Test in immersive virtual reality to assess post-stroke upper limb activity. J NeuroEng Rehabil. 2022; 19(1):133. https://doi.org/10.1186/s12984-022-01114-3

[26] Everard G, Otmane-Tolba Y, Rosselli Z, et al. Concurrent validity of an immersive virtual reality version of the Box and Block Test to assess manual dexterity among patients with stroke. J NeuroEng Rehabil. 2022; 19(1): 7. https://doi.org/10.1186/s12984-022-00981-0

[27] Saes M, Mohamed Refai MI, van Beijnum BJF, et al. Quantifying quality of reaching movements longitudinally post-stroke: a systematic review. Neurorehabilitation Neural Repair. 2022; 36(3): 183-207. https://doi.org/10.1177/15459683211062890

[28] Jiang Y, Liu Z, Liu T, Ma M, Tang M, Chai Y. A serious game system for upper limb motor function assessment of hemiparetic stroke patients. IEEE Trans Neural Syst Rehabil Eng. 2023; 31: 2640-2653. https://doi.org/10.1109/tnsre.2023.3281408

[29] Wu G, van der Helm FC, Veeger HE, et al. ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion – part II: shoulder, elbow, wrist and hand. J Biomech. 2005; 38(5): 981-992. https://doi.org/10.1016/j.jbiomech.2004.05.042

[30] Meng L, Chen M, Li B, He F, Xu R, Ming D. An inertial-based upper-limb motion assessment model: performance validation across various motion tasks. IEEE Sens J. 2023; 23(7): 7168-7177. https://doi.org/10.1109/JSEN.2022.3233344

[31] Mahony R, Hamel T, Pflimlin JM. Nonlinear complementary filters on the special orthogonal group. IEEE Trans Automat Control. 2008; 53(5): 1203-1218. https://doi.org/10.1109/TAC.2008.923738

[32] Gulde P, Hermsdörfer J. Smoothness metrics in complex movement tasks. Front Neurol. 2018; 9:615. https://doi.org/10.3389/fneur.2018.00615

[33] Balasubramanian S, Melendez-Calderon A, Roby-Brami A, Burdet E. On the analysis of movement smoothness. J NeuroEng Rehabil. 2015; 12(1):112. https://doi.org/10.1186/s12984-015-0090-9

[34] Arntz F, Markov A, Behm DG, et al. Chronic effects of static stretching exercises on muscle strength and power in healthy individuals across the lifespan: a systematic review with multi-level meta-analysis. Sports Med. 2023; 53(3): 723-745. https://doi.org/10.1007/s40279-022-01806-9

[35] Schober P, Mascha EJ, Vetter TR. Statistics from A (agreement) to Z (z score): a guide to interpreting common measures of association, agreement, diagnostic accuracy, effect size, heterogeneity, and reliability in medical research. Anesth Analg. 2021; 133(6): 1633-1641. https://doi.org/10.1213/ane.0000000000005773

[36] Shi X, Wong YD, Li MZ, Palanisamy C, Chai C. A feature learning approach based on XGBoost for driving assessment and risk prediction. Accid Anal Prev. 2019; 129: 170-179. https://doi.org/10.1016/j.aap.2019.05.005

[37] Theng D, Bhoyar KK. Feature selection techniques for machine learning: a survey of more than two decades of research. Knowl Inf Syst. 2024; 66(3): 1575-1637. https://doi.org/10.1007/s10115-023-02010-5

[38] Barro C, Benkert P, Disanto G, et al. Serum neurofilament as a predictor of disease worsening and brain and spinal cord atrophy in multiple sclerosis. Brain. 2018; 141(8): 2382-2391. https://doi.org/10.1093/brain/awy154

[39] Chaker L, Korevaar TIM, Rizopoulos D, et al. Defining optimal health range for thyroid function based on the risk of cardiovascular disease. J Clin Endocrinol Metabol. 2017; 102(8): 2853-2861. https://doi.org/10.1210/jc.2017-00410

[40] Hoonhorst MH, Nijland RH, van den Berg JS, Emmelot CH, Kollen BJ, Kwakkel G. How do Fugl-Meyer arm motor scores relate to dexterity according to the action research arm test at 6 months poststroke? Arch Phys Med Rehabil. 2015; 96(10): 1845-1849. https://doi.org/10.1016/j.apmr.2015.06.009

[41] Persson HC, Alt Murphy M, Danielsson A, Lundgren-Nilsson Å, Sunnerhagen KS. A cohort study investigating a simple, early assessment to predict upper extremity function after stroke – a part of the SALGOT study. BMC Neurol. 2015; 15(1): 92. https://doi.org/10.1186/s12883-015-0349-6

[42] Alt Murphy M, Willén C, Sunnerhagen KS. Movement kinematics during a drinking task are associated with the activity capacity level after stroke. Neurorehabilitation Neural Repair. 2012; 26(9): 1106-1115. https://doi.org/10.1177/1545968312448234

[43] Kalnins A. Multicollinearity: how common factors cause Type 1 errors in multivariate regression. Strateg Manag J. 2018; 39(8): 2362-2385. https://doi.org/10.1002/smj.2783

[44] Schwarz A, Kanzler CM, Lambercy O, Luft AR, Veerbeek JM. Systematic review on kinematic assessments of upper limb movements after stroke. Stroke. 2019; 50(3): 718-727. https://doi.org/10.1161/strokeaha.118.023531

[45] Kwakkel G, Stinear C, Essers B, et al. Motor rehabilitation after stroke: European Stroke Organisation (ESO) consensus-based definition and guiding framework. Eur Stroke J. 2023; 8(4): 880-894. https://doi.org/10.1177/23969873231191304

[46] Adans-Dester C, Hankov N, O'Brien A, et al. Enabling precision rehabilitation interventions using wearable sensors and machine learning to track motor recovery. npj Digit Med. 2020; 3(1):121. https://doi.org/10.1038/s41746-020-00328-w

[47] Werner C, Schönhammer JG, Steitz MK, et al. Using wearable inertial sensors to estimate clinical scores of upper limb movement quality in stroke. Front Physiol. 2022; 13:877563. https://doi.org/10.3389/fphys.2022.877563

[48] Chung CR, Su MC, Lee SH, Wu EH, Tang LH, Yeh SC. An intelligent motor assessment method utilizing a bi-lateral virtual-reality task for stroke rehabilitation on upper extremity. IEEE J Transl Eng Health Med. 2022; 10:2100811. https://doi.org/10.1109/jtehm.2022.3213348

[49] Schwarz A, Bhagubai MMC, Nies SHG, et al. Characterization of stroke-related upper limb motor impairments across various upper limb activities by use of kinematic core set measures. J NeuroEng Rehabil. 2022; 19(1):2. https://doi.org/10.1186/s12984-021-00979-0

[50] Levin MF, Kleim JA, Wolf SL. What do motor “recovery” and “compensation” mean in patients following stroke? Neurorehabilitation Neural Repair. 2009; 23(4): 313-319. https://doi.org/10.1177/1545968308328727

[51] Cirstea MC, Levin MF. Compensatory strategies for reaching in stroke. Brain. 2000; 123(Pt 5): 940-953. https://doi.org/10.1093/brain/123.5.940

[52] Riberto M, Frances JA, Chueire R, et al. Post hoc subgroup analysis of the BCAUSE study assessing the effect of Abobotulinumtoxina on post-stroke shoulder pain in adults. Toxins. 2022; 14(11):809. https://doi.org/10.3390/toxins14110809

[53] Grimm F, Kraugmann J, Naros G, Gharabaghi A. Clinical validation of kinematic assessments of post-stroke upper limb movements with a multi-joint arm exoskeleton. J NeuroEng Rehabil. 2021; 18(1):92. https://doi.org/10.1186/s12984-021-00875-7

[54] Koh K, Oppizzi G, Kehs G, Zhang L.-Q. Abnormal coordination of upper extremity during target reaching in persons post stroke. Sci Rep. 2023; 13(1):12838. https://doi.org/10.1038/s41598-023-39684-4

[55] Zackowski KM, Dromerick AW, Sahrmann SA, Thach WT, Bastian AJ. How do strength, sensation, spasticity and joint individuation relate to the reaching deficits of people with chronic hemiparesis? Brain. 2004; 127(5): 1035-1046. https://doi.org/10.1093/brain/awh116

[56] Wang X, Fu Y, Ye B, Babineau J, Ding Y, Mihailidis A. Technology-based compensation assessment and detection of upper extremity activities of stroke survivors: systematic review. J Med Internet Res. 2022; 24(6):e34307. https://doi.org/10.2196/34307

[57] McPherson JG, Chen A, Ellis MD, Yao J, Heckman CJ, Dewald JPA. Progressive recruitment of contralesional cortico-reticulospinal pathways drives motor impairment post stroke. J Physiol. 2018; 596(7): 1211-1225. https://doi.org/10.1113/jp274968

[58] Phan T, Nguyen H, Vermillion BC, Kamper DG, Lee SW. Abnormal proximal-distal interactions in upper-limb of stroke survivors during object manipulation: a pilot study. Front Hum Neurosci. 2022; 16:1022516. https://doi.org/10.3389/fnhum.2022.1022516

[59] McMorland AJ, Runnalls KD, Byblow WD. A neuroanatomical framework for upper limb synergies after stroke. Front Hum Neurosci. 2015; 9:82. https://doi.org/10.3389/fnhum.2015.00082

[60] Burq M, Rainaldi E, Ho KC, et al. Virtual exam for Parkinson's disease enables frequent and reliable remote measurements of motor function. npj Digit Med. 2022; 5(1):65. https://doi.org/10.1038/s41746-022-00607-8

[61] Kang H. Sample size determination and power analysis using the G*Power software. J Educ Eval Health Prof. 2021; 18:17. https://doi.org/10.3352/jeehp.2021.18.17

[62] Scott CA, Li L, Rothwell PM. Diverging temporal trends in stroke incidence in younger vs older people: a systematic review and meta-analysis. JAMA Neurol. 2022; 79(10): 1036-1048. https://doi.org/10.1001/jamaneurol.2022.1520

[63] Gilliaux M, Lejeune TM, Sapin J, Dehez B, Stoquart G, Detrembleur C. Age effects on upper limb kinematics assessed by the REAplan robot in healthy subjects aged 3 to 93 years. Ann Biomed Eng. 2016; 44(4): 1224-1233. https://doi.org/10.1007/s10439-015-1396-2

RIGHTS & PERMISSIONS

2025 The Author(s). Journal of Intelligent Medicine published by John Wiley & Sons Australia, Ltd on behalf of Tianjin University.