Implementation and evaluation of a Sepsis Best Practice Advisory alert utilizing predictive analytics to improve patient outcomes: An implementation study

Abstract

Background: Sepsis increases mortality and is a global healthcare concern. In the United States evidence-based early identification and treatment protocols are required in some states. Predictive analytics is one option to meet this requirement.
Objective: The purpose of this system-wide initiative was to develop an interprofessional team to implement, evaluate, and optimize an early alert system for patients at high risk for sepsis utilizing predictive analytics to improve patient outcomes.
Methods: The Sepsis Predictive Model and customized Best Practice Advisory (BPA) were evaluated utilizing a phased-rollout and pilot-testing. Regular meetings were conducted to analyze data and strategize iterative changes. End users were educated.
Results: Pilot tests established the most effective alert-sepsis trigger scores; scores > 5 resulted in a 54% decrease in alerts. The Concordance Statistic (C-Stat) for the system-wide roll out was 0.765. The proportion of patients who had a BPA alert, had sepsis and an intervention (18.83%) was significantly greater than the proportion of patients who had a BPA alert had sepsis, and did not have an intervention (4.35%, p-value <.001). Similar results were found for the proportion of patients with a final diagnosis of infection who had a BPA alert, and an intervention, compared to those who did not. Early warning of potential sepsis resulted in a reduction in sepsis mortality rates not present on admission.
Conclusions: An interprofessional team approach to leveraging established evidence and harnessing predictive analytics, fostered a customized collaborative protocol that improved sepsis care. Predictive analytics, tailored to clinical settings, is a powerful tool for advancing sepsis management.

Keywords

Best Practice Advisory / Epic Sepsis Predictive Model / Interprofessional collaboration / Predictive analytics / Sepsis

ACKNOWLEDGEMENTS

We would like to thank the nurses and physicians who supported this initiative through the many iterations and provided valuable feedback. We would also like to acknowledge the Sepsis Steering Committee, the CNOs, and the CMOs for their guidance and support.

AUTHORS CONTRIBUTIONS

Laura L. Reilly: Conception and design, acquisition of data, analysis and interpretation, drafting, critical revision; Natalie Peleg: Analysis and interpretation, drafting, critical; Maria Stratton: Conception and design, acquisition of data, analysis and interpretation, drafting, critical revision; Mildred Ortu Kowalski: Conception and design, analysis and interpretation, drafting, critical revision; Jyothi Jagadeesh: Acquisition of data, analysis and interpretation, drafting, critical revision; Michelle T. Martins: Conception and design, analysis and interpretation, drafting, critical revision; Stephanie Chiu: Acquisition of data, analysis and interpretation, critical revision; Cristen Mackwell: Acquisition of data, analysis and interpretation, drafting; Jeanne Giaquinto, Janet Pagulayan: Acquisition of data, drafting; Florise Altino-Pierre: Analysis and interpretation, drafting; Michael Robes, Danielle L. Wolf: Acquisition of data, analysis and interpretation, drafting, critical revision.

FUNDING

This research was not funded.

CONFLICTS OF INTEREST DISCLOSURE

The authors declare they have no conflicts of interest.

INFORMED CONSENT

There were no experimental interactions with individuals, therefore a waiver of consent was provided by the IRB.

ETHICS APPROVAL

The Publication Ethics Committee of the Association for Health Sciences and Education. The journal’s policies adhere to the Core Practices established by the Committee on Publication Ethics (COPE).

CLINICAL TRIAL REGISTRATION

Because no active recruitment was done for this retrospective analysis, the study was not registered.

PROVENANCE AND PEER REVIEW

Not commissioned; externally double-blind peer reviewed.

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.

DATA SHARING STATEMENT

The raw data from this study are not publicly available due to institutional confidentiality requirements.

OPEN ACCESS

This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).

COPYRIGHTS

Copyright for this article is retained by the author(s), with first publication rights granted to the journal.

References

[1] Deluder JM, Hulton L. An interdisciplinary code sepsis team to improve sepsis-bundle compliance: a quality improvement project. J Emerg Nurs. 2020; 46(1): 91-8. PMid: 31563282. https://doi.org/10.1016/j.jen.2019.07.001

[2] Rudd KE, Johnson SC, Agesa KM, et al. Global, regional, and national sepsis incidence and mortality, 1990-2017: analysis for the Global Burden of Disease Study. Lancet. 2020; 395(10219): 200-211. PMid: 31954465. https://doi.org/10.1016/s0140-6736(19)32989-7

[3] Bauer M, Gerlach H, Fogelman T, et al. Mortality in sepsis and septic shock in Europe, North America and Australia between 2009 and 2019-results from a systematic review and meta-analysis. Crit Care. 2020; 24(1): 239. PMid: 32430052. https://doi.org/10.1186/s13054-020-02950-2

[4] Markwart R, Saito H, Harder T, et al. Epidemiology and burden of sepsis acquired in hospitals and intensive care units: a systematic review and meta-analysis. Intensive Care Med. 2020; 46(8): 1536-51. PMid: 32591853. https://doi.org/10.1007/s00134-020-06106-2

[5] Kule A, Stassen W, Flores GE, et al. Recognition and awareness of sepsis by first-aid providers in adults with suspected infection: a scoping review. Cureus. 2024; 16(6): e61612. PMid: 38962620. https://doi.org/10.7759/cureus.61612

[6] Nemati S, Holder A, Razmi F, et al. An interpretable machine learning model for accurate prediction of sepsis in the ICU. Crit Care Med. 2018; 46(4): 547-53. PMid: 29286945. https://doi.org/10.1097/CCM.0000000000002936

[7] Rababa M, Bani Hamad D, Hayajneh AA. Sepsis assessment and management in critically ill adults: a systematic review. PLoS One. 2022; 17(7): e0270711. PMid: 35776738. https://doi.org/10.1371/journal.pone.0270711

[8] NJ Administrative Code for Sepsis Protocols in Medical Centers. New Jersey Administrative Code 8:43G-14.9. 2025. Available from: https://www.law.cornell.edu/regulations/new-jersey/N-J-A-C-8-43G-14-9

[9] Fleuren LM, Klausch TLT, Zwager CL, et al. Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy. Intensive Care Med. 2020; 46(3): 383-400. PMid: 31965266. https://doi.org/10.1007/s00134-019-05872-y

[10] Kausch SL, Moorman JR, Lake DE, et al. Physiological machine learning models for prediction of sepsis in hospitalized adults: an integrative review. Intensive Crit Care Nurs. 2021; 65: 103035. PMid: 33875337. https://doi.org/10.1016/j.iccn.2021.103035

[11] Islam MM, Nasrin T, Walther BA, et al. Prediction of sepsis patients using machine learning approach: a meta-analysis. Comput Methods Programs Biomed. 2019; 170: 1-9. PMid: 30712598. https://doi.org/10.1016/j.cmpb.2018.12.027

[12] Goh KH, Wang L, Yeow AYK, et al. Artificial intelligence in sepsis early prediction and diagnosis using unstructured data in healthcare. Nat Commun. 2021; 12(1): 711. PMid: 33514699. https://doi.org/10.1038/s41467-021-20910-4

[13] Wong A, Otles E, Donnelly JP, et al. External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Intern Med. 2021; 181(8): 1065-70. PMid: 34152373. https://doi.org/10.1001/jamainternmed.2021.2626

[14] Mahyoub MA, Yadav RR, Dougherty K, et al. Development and validation of a machine learning model integrated with the clinical workflow for early detection of sepsis. Front Med. 2023; 10: 1284081. PMid: 38076259. https://doi.org/10.3389/fmed.2023.1284081

[15] Sendak MP, Ratliff W, Sarro D, et al. Real-world integration of a sepsis deep learning technology into routine clinical care: implementation study. JMIR Med Inform. 2020; 8(7): e15182. PMid: 32673244. https://doi.org/10.2196/15182

[16] Shreffler J, Huecker MR. Diagnostic testing accuracy: sensitivity, specificity, predictive values and likelihood ratios. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2023. Available from: http://europepmc.org/books/NBK557491

[17] Cull J, Brevetta R, Gerac J, Kothari S, Blackhurst D. Epic sepsis model inpatient predictive analytic tool: a validation study. Critical Care Explorations. 2023; 5(7): e0941. https://doi.org/10.1097/CCE.0000000000000941

[18] Caetano SJ, Nonpaved G, Pond GR. C-statistic: a brief explanation of its construction, interpretation and limitations. Eur J Cancer. 2018; 90: 130-2. PMid: 29221899. https://doi.org/10.1016/j.ejca.2017.10.027

[19] Dellinger RP, Levy MM, Rhodes A, et al. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock, 2012. Intensive Care Med. 2013; 39(2): 165-228. PMid: 23361625. https://doi.org/10.1007/s00134-012-2769-8

[20] Evans L, Rhodes A, Alhazzani W, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Crit Care Med. 2021; 49(11): e1063-1143. PMid: 34605781. https://doi.org/10.1097/CCM.0000000000005337

[21] Gao Y, Wang HL, Zhang ZJ, et al. A standardized step-by-step approach for the diagnosis and treatment of sepsis. J Intensive Care Med. 2022; 37(10): 1281-7. PMid: 35285730. https://doi.org/10.1177/08850666221085181

[22] Rashidzada Z, Cairns KA, Peel TN, et al. Early antimicrobial stewardship team intervention on appropriateness of antimicrobial therapy in suspected sepsis: a randomized controlled trial. JAC Antimicrob Resist. 2021; 3(3): dlab097. PMid: 34458731. https://doi.org/10.1093/jacamr/dlab097