Predictive models for patient-reported outcomes (PROs) in elective spine surgery: a systematic review

Abstract

Aim: A growing body of literature reports on prediction models for patient-reported outcomes of spine surgery, carrying broad implications for use in value-based care and decision making. This review assesses the performance and transparency of reporting of these models.

Methods: We queried four studies reporting the development and/or validation of prediction models for patient-reported outcome measures (PROMs) following elective spine surgery with performance metrics such as the area under the receiver operating curve (AUC) scores. Adherence to transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD-AI) guidelines was assessed. One representative model was selected from each study.

Results: Of 4,471 screened studies, 35 were included, with nine development, 24 development and evaluation, and two evaluation studies. Sixteen machine learning models and 19 traditional prediction models were represented. Oswestry disability index (ODI) and modified Japanese Orthopaedic Association (mJOA) scores were most commonly used. Among 29 categorical outcome prediction models, the median [interquartile range (IQR)] AUC was 0.79 [0.73, 0.84]. The median [IQR] AUC was 0.825 [0.76, 0.84] among machine learning models and 0.74 [0.71, 0.81] among traditional models. Adherence to TRIPOD-AI guidelines was inconsistent, with no studies commenting on healthcare inequalities in the sample population, model fairness, or disclosure of study protocols or registration.

Conclusion: We found considerable variation between studies, not only in chosen patient populations and outcome measures, but also in their manner of evaluation and reporting. Agreement about outcome definitions, more frequent external validation, and improved completeness of reporting may facilitate the effective use and interpretation of these models.

Keywords

Patient reported outcomes / predictive modeling / accuracy / machine learning

References

[1] Falavigna A,Teles AR.Current status of worldwide use of patient-reported outcome measures (PROMs) in spine care.World Neurosurg2017;108:328-35

[2] Finkelstein JA.Patient-reported outcomes in spine surgery: past, current, and future directions.J Neurosurg Spine2019;31:155-64

[3] Lapin B,Davin S.Comparison of stratification techniques for optimal management of patients with chronic low back pain in spine clinics.Spine J2023;23:1334-44

[4] Lee TJ,Grandhi NR.Cost-effectiveness applications of patient-reported outcome measures (PROMs) in spine surgery.Clin Spine Surg2020;33:140-5

[5] Pronk Y,Brinkman JM,van der Weegen W.Response rate and costs for automated patient-reported outcomes collection alone compared to combined automated and manual collection.J Patient Rep Outcomes2019;3:31 PMCID:PMC6545294

[6] Beighley A,Huang B.Patient-reported outcome measures in spine surgery: a systematic review.J Craniovertebr Junction Spine2022;13:378-89 PMCID:PMC9910127

[7] Guzman JZ,Connolly J.Patient-reported outcome instruments in spine surgery.Spine2016;41:429-37

[8] Ravishankar P,Rabah N,Mroz T.Analysis of patient-reported outcomes measures used in lumbar fusion surgery research for degenerative spondylolisthesis.Clin Spine Surg2022;35:287-94

[9] Teles AR,Falavigna A.Why and how should we measure outcomes in spine surgery?.J Taibah Univ Med Sci2016;11:91-7

[10] Wellington IJ,Murphy KV,Ng MK.The use of machine learning for predicting candidates for outpatient spine surgery: a review.J Spine Surg2023;9:323-30 PMCID:PMC10570640

[11] Page MJ,Bossuyt PM.The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.BMJ2021;372:n71 PMCID:PMC8005924

[12] Collins GS,Dhiman P.TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods.BMJ2024;385:e078378 PMCID:PMC11019967

[13] Moons KG,Reitsma JB.Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration.Ann Intern Med2015;162:W1-73

[14] Staartjes VE,Ricciardi L.FUSE-ML: development and external validation of a clinical prediction model for mid-term outcomes after lumbar spinal fusion for degenerative disease.Eur Spine J2022;31:2629-38

[15] Jaja BNR,Harrington EM.Analysis of recovery trajectories in degenerative cervical myelopathy to facilitate improved patient counseling and individualized treatment recommendations.J Neurosurg Spine2023;38:644-52

[16] Sundaramoorthy K,Koteswari S.Artificial intelligence and machine learning-driven decision-making in spinal disease treatment.J Theor Appl Inf Technol2023;101:8388-406http://www.jatit.org/volumes/Vol101No24/38Vol101No24.pdf. (accessed 2025-02-13)

[17] Wirries A,Hammad A.AI prediction of neuropathic pain after lumbar disc herniation-machine learning reveals influencing factors.Biomedicines2022;10:1319 PMCID:PMC9219728

[18] Staub LP,Skrivankova V,Haschtmann D.Development and temporal validation of a prognostic model for 1-year clinical outcome after decompression surgery for lumbar disc herniation.Eur Spine J2020;29:1742-51

[19] Ford JJ,Page P,McMeeken JM.A multivariate prognostic model for pain and activity limitation in people undergoing lumbar discectomy.Br J Neurosurg2020;34:381-7

[20] Rundell SD,Nian H.Adding 3-month patient data improves prognostic models of 12-month disability, pain, and satisfaction after specific lumbar spine surgical procedures: development and validation of a prediction model.Spine J2020;20:600-13

[21] Debnath UK,Freeman BJC.Predictive factors for the outcome of surgical treatment of lumbar spondylolysis in young sporting individuals.Global Spine J2018;8:121-8 PMCID:PMC5898674

[22] Çorbacıoğlu ŞK.Receiver operating characteristic curve analysis in diagnostic accuracy studies: a guide to interpreting the area under the curve value.Turk J Emerg Med2023;23:195-8 PMCID:PMC10664195

[23] Stiglic G,Fijacko N,Verbert K.Interpretability of machine learning-based prediction models in healthcare.WIREs Data Min Knowl Discov2020;10:e1379

[24] Berg B,Fjeld O.Machine learning models for predicting disability and pain following lumbar disc herniation surgery.JAMA Netw Open2024;7:e2355024 PMCID:PMC10851101

[25] Berjano P,Ventriglia L.The influence of baseline clinical status and surgical strategy on early good to excellent result in spinal lumbar arthrodesis: a machine learning approach.J Pers Med2021;11:1377

[26] Ogink PT,Karhade AV.Wide range of applications for machine-learning prediction models in orthopedic surgical outcome: a systematic review.Acta Orthop2021;92:526-31 PMCID:PMC8519550

[27] Collins GS,Dutton S.External validation of multivariable prediction models: a systematic review of methodological conduct and reporting.BMC Med Res Methodol2014;14:40 PMCID:PMC3999945

[28] Polzer C,Meyer C.AI-based automated detection and stability analysis of traumatic vertebral body fractures on computed tomography.Eur J Radiol2024;173:111364

[29] Carreon LY,Archer KR,Hansen KH.Performance of the streamlined quality outcomes database web-based calculator: internal and external validation.Spine J2024;24:662-9

[30] Pedersen CF,Carreon LY,Doering P.PROPOSE. Development and validation of a prediction model for shared decision making for patients with lumbar spinal stenosis.N Am Spine Soc J2024;17:100309 PMCID:PMC10831309

[31] Halicka M,Duarte R.Predicting patient-reported outcomes following lumbar spine surgery: development and external validation of multivariable prediction models.BMC Musculoskelet Disord2023;24:333 PMCID:PMC10134672

[32] Matsukura Y,Inose H.Preoperative symptom duration influences neurological recovery and patient-reported outcome measures after surgical treatment of cervical ossification of the posterior longitudinal ligament.Spine2023;48:1259-65

[33] Rushton AB,Verra ML.Predictors of poor outcome following lumbar spinal fusion surgery: a prospective observational study to derive two clinical prediction rules using British Spine Registry data.Eur Spine J2023;32:2303-18

[34] Geere JH,Swamy GN,Rai AS.Development and temporal validation of clinical prediction models for 1-year disability and pain after lumbar decompressive surgery. The Norwich Lumbar Surgery Predictor (development version).Eur Spine J2023;32:4210-9

[35] Zhang JK,Javeed S.Diffusion basis spectrum imaging predicts long-term clinical outcomes following surgery in cervical spondylotic myelopathy.Spine J2023;23:504-12 PMCID:PMC10629376

[36] Chen X,Xu X,Wang R.Development, validation, and visualization of a web-based nomogram to predict the effect of tubular microdiscectomy for lumbar disc herniation.Front Surg2023;10:1024302 PMCID:PMC10069648

[37] Dong S,Yang H.Evaluation of the predictors for unfavorable clinical outcomes of degenerative lumbar spondylolisthesis after lumbar interbody fusion using machine learning.Front Public Health2022;10:835938 PMCID:PMC8927688

[38] Pedersen CF,Carreon LY.Applied machine learning for spine surgeons: predicting outcome for patients undergoing treatment for lumbar disc herniation using PRO data.Global Spine J2022;12:866-76 PMCID:PMC9344505

[39] Coric D,Derman P,Situ A.Predictors of long-term clinical outcomes in adult patients after lumbar total disc replacement: development and validation of a prediction model.J Neurosurg Spine2022;36:399-407

[40] Purohit G,Sinha VD.Use of artificial intelligence for the development of predictive model to help in decision-making for patients with degenerative lumbar spine disease.Asian J Neurosurg2022;17:274-9 PMCID:PMC9473813

[41] Khan O,Akbar MA.Prediction of worse functional status after surgery for degenerative cervical myelopathy: a machine learning approach.Neurosurgery2021;88:584-91

[42] Budiono GR,Parr WCH.Development of a multivariate prediction model for successful oswestry disability index changes in L5/S1 anterior lumbar interbody fusion for degenerative disc disease.World Neurosurg2021;148:e1-9

[43] Werner DAT,Småstuen MC.A prognostic model for failure and worsening after lumbar microdiscectomy: a multicenter study from the Norwegian Registry for Spine Surgery.Acta Neurochir2021;163:2567-80 PMCID:PMC8357664

[44] Pilato F,Distefano M.Multidimensional assessment of cervical spondylotic myelopathy patients. Usefulness of a comprehensive score system.Neurol Sci2021;42:1507-14 PMCID:PMC7956005

[45] Karhade AV,Cha TD.Development of prediction models for clinically meaningful improvement in PROMIS scores after lumbar decompression.Spine J2021;21:397-404

[46] Zhang MZ,Jiang L.Optimal machine learning methods for radiomic prediction models: clinical application for preoperative T₂^*-weighted images of cervical spondylotic myelopathy.JOR Spine2021;4:e1178 PMCID:PMC8717093

[47] Quddusi A,Klukowska AM.External validation of a prediction model for pain and functional outcome after elective lumbar spinal fusion.Eur Spine J2020;29:374-83

[48] Siccoli A,Schröder ML.Machine learning-based preoperative predictive analytics for lumbar spinal stenosis.Neurosurg Focus2019;46:E5

[49] Merali ZG,Badhiwala JH,Fehlings MG.Using a machine learning approach to predict outcome after surgery for degenerative cervical myelopathy.PLoS One2019;14:e0215133 PMCID:PMC6448910

[50] Staartjes VE,Vandertop WP.Deep learning-based preoperative predictive analytics for patient-reported outcomes following lumbar discectomy: feasibility of center-specific modeling.Spine J2019;19:853-61

[51] De la Garza Ramos R,Nakhla J.Predictors of return to normal neurological function after surgery for moderate and severe degenerative cervical myelopathy: an analysis of a global AOSpine cohort of patients.Neurosurgery2019;85:E917-23

[52] Rubery PT,Mesfin A,Papuga MO.Preoperative patient reported outcomes measurement information system scores assist in predicting early postoperative success in lumbar discectomy.Spine2019;44:325-33

[53] Nouri A,Côté P,Dalzell K.Does magnetic resonance imaging improve the predictive performance of a validated clinical prediction rule developed to evaluate surgical outcome in patients with degenerative cervical myelopathy?.Spine2015;40:1092-100