A Combined-Mode Machine Learning Model for Predicting Stroke Recurrence During Hospitalization in Patients with Acute Minor Ischemic Stroke

Abstract

Acute minor ischemic stroke patients often experience recurrence shortly after symptom onset, highlighting the importance of predicting stroke recurrence for guiding treatment decisions. This study evaluated the effectiveness of machine learning models in predicting in-hospital recurrence. The study cohort comprised 322,135 patients with acute minor ischemic stroke from 1439 centers, as established by Chinese Stroke Center Alliance. Patients were randomly allocated into training and test sets by different centers. Models including extreme gradient boosting (XGB), light gradient boosting (LGB), and adaptive boosting (ADA) were developed using fivefold cross-validation on the training set. Optimization was performed for all models based on the most important variable, history of ischemic stroke. Compared with the traditional generalized linear model (GLM), the XGB, LGB, ADA models yielded area under the curve (AUC) values ranging from 0.788 to 0.803 after optimization. All models showed significant improvements in AUC compared with GLM, with LGB exhibiting the most substantial enhancement after optimization. For the first time, this study developed models specifically designed to predict in-hospital stroke recurrence in acute minor ischemic stroke patients. This finding aids in identifying high-risk patients and prompts physicians to provide targeted treatment. However, further external validation is warranted to confirm the model's generalizability.

Keywords

acute minor ischemic stroke / in-hospital recurrence / predictive model / machine learning

References

[1] Y. J. Wang, Z. X. Li, H. Q. Gu, et al., “China Stroke Statistics Report 2019(1),” Chinese Journal of Stroke 15, no. 10 (2020): 1037-1043.

[2] M. Katan and A. Luft, “Global Burden of Stroke,” Seminars in Neurology 38, no. 2 (2018): 208-211.

[3] J. Fan, X. Li, X. Yu, et al., “Global Burden, Risk Factor Analysis, and Prediction Study of Ischemic Stroke, 1990-2030,” Neurology 101, no. 2 (2023): e137-e150.

[4] GBD 2019 Stroke Collaborators. “Global, Regional, and National Burden of Stroke and Its Risk Factors, 1990-2019: A Systematic Analysis for the Global Burden of Disease Study 2019,” Lancet Neurology 2021; 20(10): 795-820.

[5] GBD 2016 Lifetime Risk of Stroke Collaborators, V. L. Feigin, G. Nguyen, et al., GBD 2016 Lifetime Risk of Stroke Collaborators, “Global, Regional, and Country-Specific Lifetime Risks of Stroke, 1990 and 2016,” New England Journal of Medicine 379, no. 25 (2018): 2429-2437.

[6] Y. Pan, J. Jing, W. Chen, et al., “Risks and Benefits of clopidogrel-aspirin in minor Stroke or TIA: Time Course Analysis of CHANCE,” Neurology 93, no. 7 (2019): 322.

[7] Y. Wang, J. Jing, X. Meng, et al., “The Third China National Stroke Registry (CNSR-III) for Patients With Acute Ischaemic Stroke or Transient Ischaemic Attack: Design, Rationale and Baseline Patient Characteristics,” Stroke Vasc Neurol 4, no. 3 (2019): 158-164.

[8] Y. Xiong, H. Gu, X. Q. Zhao, et al., “Clinical Characteristics and in-Hospital Outcomes of Varying Definitions of Minor Stroke: From a Large-Scale Nation-Wide Longitudinal Registry,” Stroke; A Journal of Cerebral Circulation 52, no. 4 (2021): 1253-1258.

[9] Y. Pan, X. Cai, J. Jing, et al., “Stress Hyperglycemia and Prognosis of Minor Ischemic Stroke and Transient Ischemic Attack: The CHANCE Study (Clopidogrel in High-Risk Patients With Acute Nondisabling Cerebrovascular Events),” Stroke; A Journal of Cerebral Circulation 48, no. 11 (2017): 3006-3011.

[10] B. Lin, Z. Zhang, Y. Mei, et al., “Cumulative Risk of Stroke Recurrence Over the Last 10 Years: A Systematic Review and Meta-analysis,” Neurol Sci 42, no. 1 (2021): 61-71.

[11] Y. Xiong, S. Wang, Z. Li, et al., “Thirteen-year Trends in Risk Scores Predictive Values for Subsequent Stroke in Patients With Acute Ischemic Event,” Brain Behav 13, no. 5 (2023): e2962.

[12] Y. Pan, Z. Li, J. Li, et al., “Residual Risk and Its Risk Factors for Ischemic Stroke With Adherence to Guideline-Based Secondary Stroke Prevention,” J Stroke 23, no. 1 (2021): 51-60.

[13] Y. Wang, Y. Wang, X. Zhao, et al., “Clopidogrel With aspirin in Acute minor Stroke or Transient Ischemic Attack,” New England Journal of Medicine 369, no. 1 (2013): 11-19.

[14] C. Hong, M. J. Pencina, D. M. Wojdyla, et al., “Predictive Accuracy of Stroke Risk Prediction Models across Black and White Race, Sex, and Age Groups,” Jama 329, no. 4 (2023): 306-317.

[15] H. Q. Gu, K. X. Yang, Y. Y. Jiang, et al., “Progress and Prospects of Clinical Prediction Models for Risk of Stroke Recurrence in Ischemic Stroke,” Chinese Journal of Stroke 18, no. 07 (2023): 731-739.

[16] A. K. Bonkhoff and C. Grefkes, “Precision Medicine in Stroke: Towards Personalized Outcome Predictions Using Artificial Intelligence,” Brain 145, no. 2 (2022): 457-475.

[17] D. Chaudhary, V. Abedi, J. Li, C. M. Schirmer, C. J. Griessenauer, and R. Zand, “Clinical Risk Score for Predicting Recurrence Following a Cerebral Ischemic Event,” Front Neurol 10 (2019): 1106.

[18] M. Lee, N. Y. Yeo, H. J. Ahn, et al., “Prediction of Post-stroke Cognitive Impairment After Acute Ischemic Stroke Using Machine Learning,” Alzheimers Res Ther 15, no. 1 (2023): 147.

[19] N. B. o. t. C. M. Association, “Cerebrovascular Disease Group of the Neurology Branch of the Chinese Medical Association. Expert Consensus on the Use of the Chinese Ischemic Stroke Risk Assessment Scale,” Chinese Journal of Neurology 2016; 49(7): 519-525.

[20] R. Lemmens, S. Smet, and V. N. Thijs, “Clinical Scores for Predicting Recurrence After Transient Ischemic Attack or Stroke: How Good Are They?,” Stroke; A Journal of Cerebral Circulation 44, no. 4 (2013): 1198-1203.

[21] M. Kamouchi, N. Kumagai, Y. Okada, H. Origasa, T. Yamaguchi, and T. Kitazono, “Risk Score for Predicting Recurrence in Patients With Ischemic Stroke: The Fukuoka Stroke Risk Score for Japanese,” Cerebrovascular Diseases 34, no. 5-6 (2012): 351-357.

[22] H. C. Diener, P. A. Ringleb, and P. Savi, “Clopidogrel for the Secondary Prevention of Stroke,” Expert Opinion on Pharmacotherapy 6, no. 5 (2005): 755-764.

[23] CAPRIE Steering Committee. A Randomised, Blinded, Trial of clopidogrel versus aspirin in Patients at Risk of Ischaemic Events (CAPRIE). CAPRIE Steering Committee. Lancet 1996; 348(9038): 1329-1339.

[24] H. Ay, L. Gungor, E. M. Arsava, et al., “A Score to Predict Early Risk of Recurrence After Ischemic Stroke,” Neurology 74, no. 2 (2010): 128-135.

[25] W. N. Kernan, R. I. Horwitz, L. M. Brass, C. M. Viscoli, and K. J. Taylor, “A Prognostic System for Transient Ischemia or minor Stroke,” Annals of Internal Medicine 114, no. 7 (1991): 552-557.

[26] W. N. Kernan, C. M. Viscoli, L. M. Brass, et al., “The Stroke Prognosis Instrument II (SPI-II): A Clinical Prediction Instrument for Patients With Transient Ischemia and Nondisabling Ischemic Stroke,” Stroke; A Journal of Cerebral Circulation 31, no. 2 (2000): 456-462.

[27] S. B. Coutts, M. Eliasziw, M. D. Hill, et al., “An Improved Scoring System for Identifying Patients at High Early Risk of Stroke and Functional Impairment After an Acute Transient Ischemic Attack or minor Stroke,” Int J Stroke 3, no. 1 (2008): 3-10.

[28] P. M. Rothwell, M. F. Giles, E. Flossmann, et al., “A Simple Score (ABCD) to Identify Individuals at High Early Risk of Stroke After Transient Ischaemic Attack,” Lancet 366, no. 9479 (2005): 29-36.

[29] Predictors of Major Vascular Events in Patients With a Transient Ischemic Attack or Nondisabling Stroke. The Dutch TIA Trial Study Group. Stroke; A Journal of Cerebral Circulation 1993; 24(4): 527-531.

[30] I. van Wijk, L. J. Kappelle, J. van Gijn, et al., “Long-term Survival and Vascular Event Risk After Transient Ischaemic Attack or minor Ischaemic Stroke: A Cohort Study,” Lancet 365, no. 9477 (2005): 2098-2104.

[31] K. M. Mohan, C. D. Wolfe, A. G. Rudd, P. U. Heuschmann, P. L. Kolominsky-Rabas, and A. P. Grieve, “Risk and Cumulative Risk of Stroke Recurrence: A Systematic Review and Meta-analysis,” Stroke; A Journal of Cerebral Circulation 42, no. 5 (2011): 1489-1494.

[32] I. E. Kumar, S. Venkatasubramanian, C. Scheidegger, et al., “Problems With Shapley-value-based Explanations as Feature Importance Measures,” International Conference on Machine Learning. 2020: 5491-5500.

[33] S. Hart, Shapley Value. Game Theory (London: Palgrave Macmillan UK, 1989): 210-216.

[34] A. J. Vickers, B. van Calster, and E. W. Steyerberg, “A Simple, Step-by-step Guide to Interpreting Decision Curve Analysis,” Diagn Progn Res 3 (2019): 18.

[35] A. J. Vickers and F. Holland, “Decision Curve Analysis to Evaluate the Clinical Benefit of Prediction Models,” The Spine Journal: Official Journal of the North American Spine Society 21, no. 10 (2021): 1643-1648.

[36] B. Van Calster, L. Wynants, J. F. M. Verbeek, et al., “Reporting and Interpreting Decision Curve Analysis: A Guide for Investigators,” European Urology 74, no. 6 (2018): 796-804.

[37] S. C. Johnston, D. R. Gress, W. S. Browner, and S. Sidney, “Short-term Prognosis After Emergency Department Diagnosis of TIA,” Jama 284, no. 22 (2000): 2901-2906.

[38] S. C. Johnston, P. M. Rothwell, M. N. Nguyen-Huynh, et al., “Validation and Refinement of Scores to Predict Very Early Stroke Risk After Transient Ischaemic Attack,” Lancet 369, no. 9558 (2007): 283-292.

[39] M. F. Giles, G. W. Albers, P. Amarenco, et al., “Addition of Brain Infarction to the ABCD2 Score (ABCD2I): A Collaborative Analysis of Unpublished Data on 4574 Patients,” Stroke; A Journal of Cerebral Circulation 41, no. 9 (2010): 1907-1913.

[40] A. Merwick, G. W. Albers, P. Amarenco, et al., “Addition of Brain and Carotid Imaging to the ABCD2 Score to Identify Patients at Early Risk of Stroke After Transient Ischaemic Attack: A Multicentre Observational Study,” Lancet Neurology 9, no. 11 (2010): 1060-1069.

[41] S. T. Engelter, M. Amort, F. Jax, et al., “Optimizing the Risk Estimation After a Transient Ischaemic Attack—the ABCDE+ Score,” European Journal of Neurology 19, no. 1 (2012): 55-61.

[42] M. B. Kursa and W. R. Rudnicki, “Feature Selection With the Boruta Package,” Journal of Statistical Software 36, no. 11 (2010): 1-13.

[43] W. J. Powers, A. A. Rabinstein, T. Ackerson, et al. Guidelines for the Early Management of Patients With Acute Ischemic Stroke: 2019 Update to the 2018 Guidelines for the Early Management of Acute Ischemic Stroke: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association [published correction appears in Stroke. 2019; 50(12): e440-e441.

[44] Y. Wang, Z. Li, Y. Wang, et al., “Chinese Stroke Center Alliance: A National Effort to Improve Healthcare Quality for Acute Stroke and Transient Ischaemic Attack: Rationale, Design and Preliminary Findings,” Stroke Vasc Neurol 3, no. 4 (2018): 256-262.

[45] T. Hillen, C. Coshall, K. Tilling, et al., “Cause of Stroke Recurrence Is Multifactorial: Patterns, Risk Factors, and Outcomes of Stroke Recurrence in the South London Stroke Register,” Stroke; A Journal of Cerebral Circulation 34, no. 6 (2003): 1457-1463.

[46] C. Flach, W. Muruet, C. D. A. Wolfe, A. Bhalla, and A. Douiri, “Risk and Secondary Prevention of Stroke Recurrence: A Population-Base Cohort Study,” Stroke; A Journal of Cerebral Circulation 51, no. 8 (2020): 2435-2444.

[47] S. van Buuren and G.-O. K. mice, “Multivariate Imputation by Chained Equations in R,” Journal of Statistical Software 45, no. 3 (2011): 1-67.

[48] J. C. Stoltzfus, “Logistic Regression: A Brief Primer,” Academic Emergency Medicine 18, no. 10 (2011): 1099-1104.

[49] C. Cortes and V. Vapnik, “Support-vector Networks,” Machine learning 20, no. 3 (1995): 273-297.

[50] B. E. Boser, I. M. Guyon, and V. N Vapnik, “A Training Algorithm for Optimal Margin Classifiers,” Proceedings of the fifth annual workshop on Computational learning theory. 1992: 144-152.

[51] T. Joachims. Making large scale SVM learning practical. Advances in Kernel Methods: Support Vector Machines. 1999.

[52] K. N. Fang, J. B. Wu, J. P. Zhu, et al., “Comprehensive Review of the Random Forest Methodology,” Statistical Information Forum 26, no. 3 (2011): 32-38.

[53] S. J. Rigatti, “Random Forest,” Journal of Insurance Medicine 47, no. 1 (2017): 31-39.

[54] T. Q. Chen and C. Guestrin, “Xgboost: A Scalable Tree Boosting System,” Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM. 2016: 785-794.

[55] T. Chen, T. He, M. Benesty, et al., “Xgboost: Extreme Gradient Boosting,” R Package Version 04-2 1, no. 4 (2015): 1-4.

[56] H. Schwenk and Y. Bengio, “Boosting Neural Networks,” Neural Computation 12, no. 8 (2000): 1869-1887.

[57] Y. F. Yin, X. Z. Yang, Y. Gan, et al., “Research on Vulnerability Exploitation Prediction Based on the LightGBM Algorithm,” Journal of Zhengzhou University (Engineering Edition) 43, no. 05 (2022): 24-30.

[58] P. C. Austin and E. A. Stuart, “Moving towards Best Practice When Using Inverse Probability of Treatment Weighting (IPTW) Using the Propensity Score to Estimate Causal Treatment Effects in Observational Studies,” Statistics in Medicine 34, no. 28 (2015): 3661-3679.

[59] Y. Xian, R. G. Holloway, P. S. Chan, et al., “Association Between Stroke Center Hospitalization for Acute Ischemic Stroke and Mortality,” Jama 305, no. 4 (2011): 373-380.

[60] Y. Xian, J. Wu, E. C. O'Brien, et al., “Real World Effectiveness of warfarin Among Ischemic Stroke Patients With Atrial Fibrillation: Observational Analysis From Patient-Centered Research Into Outcomes Stroke Patients Prefer and Effectiveness Research (PROSPER) Study,” Bmj 351 (2015): h3786.

[61] C. Weimar, H. C. Diener, M. J. Alberts, et al., “The Essen Stroke Risk Score Predicts Recurrent Cardiovascular Events: A Validation Within the REduction of Atherothrombosis for Continued Health (REACH) Registry,” Stroke; A Journal of Cerebral Circulation 40, no. 2 (2009): 350-354.

[62] G. J. Hankey, J. M. Slattery, and C. P. Warlow, “Can the Long Term Outcome of Individual Patients With Transient Ischaemic Attacks be Predicted Accurately?,” Journal of Neurology, Neurosurgery, and Psychiatry 56, no. 7 (1993): 752-759.

RIGHTS & PERMISSIONS

2025 The Author(s). MedComm published by Sichuan International Medical Exchange & Promotion Association (SCIMEA) and John Wiley & Sons Australia, Ltd.