Machine Learning-Based Pediatric Early Warning Score: Patient Outcomes in a Pre- Versus Post-Implementation Study, 2019-2023

Overview

Journal Pediatr Crit Care Med

Specialty Pediatrics

Date 2025 Feb 21

PMID 39982152

Authors

Anoop Mayampurath

Kyle Carey

Brett Palama

Monica Gonzalez

Joe Reid

Allison H Bartlett

Matthew Churpek

Dana Edelson

Priti Jani

Affiliations

Soon will be listed here.

Abstract

Objectives: To describe the deployment of pediatric Calculated Assessment of Risk and Triage (pCART), a machine learning (ML) model to predict the risk of the direct ward to the ICU transfer within 12 hours, and the associated improved outcomes among hospitalized children.

Design: Pre- vs. post-implementation study.

Setting: An urban, tertiary-care, academic hospital.

Patients: Pediatric (age < 18 yr) admissions from May 1, 2019, to April 30, 2023.

Interventions: None.

Measurements And Main Results: Patients were divided into baseline, pre-pCART implementation (May 1, 2019, to April 30 2021), and post-pCART implementation (May 1, 2021, to April 30, 2023) cohorts. First-ward admissions with a high-risk score (pCART score ≥ 92) were considered as the main cohort. The primary outcome was the occurrence of critical events, defined as invasive mechanical ventilation, vasoactive drug administration, or death within 12 hours of the first high-risk pCART score. There were 2763 and 3943 patients in the baseline and implementation cohorts, respectively. pCART implementation was associated with a decrease in the percentage of the primary outcome from baseline 1.4% to 0.4% (p < 0.001), which converted to more than two-thirds lower adjusted odds of the primary outcome (odds ratio, 0.22 [95% CI, 0.11-0.40]; p < 0.001). pCART implementation was also associated with a decreased prevalence of critical events at 24 and 48 hours after a first high-risk score. We failed to identify any association between cohort period and overall hospital and ICU length-of-stay, number of ICU transfers, and time to ICU transfer. However, there was a difference in hospital length-of-stay among a subpopulation of patients transferred to the ICU (median 6 vs. 7 d; p < 0.001). Analysis of compliance metrics indicates sustained compliance achievements over time.

Conclusions: The deployment of pCART, a ML-based pediatric risk stratification tool, for clinical decision support for pediatric ward patients, was associated with lower odds of critical events among high-risk patients.

References

Chapman S, Wray J, Oulton K, Pagel C, Ray S, Peters M . 'The Score Matters': wide variations in predictive performance of 18 paediatric track and trigger systems. Arch Dis Child. 2017; 102(6):487-495. DOI: 10.1136/archdischild-2016-311088. View

Rubin J, Potes C, Xu-Wilson M, Dong J, Rahman A, Nguyen H . An ensemble boosting model for predicting transfer to the pediatric intensive care unit. Int J Med Inform. 2018; 112:15-20. DOI: 10.1016/j.ijmedinf.2018.01.001. View

Wellner B, Grand J, Canzone E, Coarr M, Brady P, Simmons J . Predicting Unplanned Transfers to the Intensive Care Unit: A Machine Learning Approach Leveraging Diverse Clinical Elements. JMIR Med Inform. 2017; 5(4):e45. PMC: 5719228. DOI: 10.2196/medinform.8680. View

Gawronski O, Ferro F, Cecchetti C, Ciofi Degli Atti M, DallOglio I, Tiozzo E . Adherence to the bedside paediatric early warning system (BedsidePEWS) in a pediatric tertiary care hospital. BMC Health Serv Res. 2021; 21(1):852. PMC: 8380378. DOI: 10.1186/s12913-021-06809-2. View

Liang H, Carey K, Jani P, Gilbert E, Afshar M, Sanchez-Pinto L . Association between mortality and critical events within 48 hours of transfer to the pediatric intensive care unit. Front Pediatr. 2024; 11:1284672. PMC: 10768058. DOI: 10.3389/fped.2023.1284672. View

Dewan M, Muthu N, Shelov E, Bonafide C, Brady P, Davis D . Performance of a Clinical Decision Support Tool to Identify PICU Patients at High Risk for Clinical Deterioration. Pediatr Crit Care Med. 2019; 21(2):129-135. PMC: 7007854. DOI: 10.1097/PCC.0000000000002106. View

Bonafide C, Roberts K, Priestley M, Tibbetts K, Huang E, Nadkarni V . Development of a pragmatic measure for evaluating and optimizing rapid response systems. Pediatrics. 2012; 129(4):e874-81. DOI: 10.1542/peds.2011-2784. View

da Silva Y, Hamilton M, Horvat C, Fink E, Palmer F, Nowalk A . Evaluation of Electronic Medical Record Vital Sign Data Versus a Commercially Available Acuity Score in Predicting Need for Critical Intervention at a Tertiary Children's Hospital. Pediatr Crit Care Med. 2015; 16(7):644-51. DOI: 10.1097/PCC.0000000000000444. View

Mayampurath A, Jani P, Dai Y, Gibbons R, Edelson D, Churpek M . A Vital Sign-Based Model to Predict Clinical Deterioration in Hospitalized Children. Pediatr Crit Care Med. 2020; 21(9):820-826. PMC: 7483876. DOI: 10.1097/PCC.0000000000002414. View

10.

Long D, Fink E . Transitions from short to long-term outcomes in pediatric critical care: considerations for clinical practice. Transl Pediatr. 2021; 10(10):2858-2874. PMC: 8578758. DOI: 10.21037/tp-21-61. View

11.

Parshuram C, Hutchison J, Middaugh K . Development and initial validation of the Bedside Paediatric Early Warning System score. Crit Care. 2009; 13(4):R135. PMC: 2750193. DOI: 10.1186/cc7998. View

12.

Rust L, Gorham T, Bambach S, Bode R, Maa T, Hoffman J . The Deterioration Risk Index: Developing and Piloting a Machine Learning Algorithm to Reduce Pediatric Inpatient Deterioration. Pediatr Crit Care Med. 2023; 24(4):322-333. DOI: 10.1097/PCC.0000000000003186. View

13.

Chong S, Goh M, Ong G, Acworth J, Sultana R, Yao S . Do paediatric early warning systems reduce mortality and critical deterioration events among children? A systematic review and meta-analysis. Resusc Plus. 2022; 11:100262. PMC: 9253845. DOI: 10.1016/j.resplu.2022.100262. View

14.

Hopkins R . Life after Pediatric Critical Illness: Risk Factors for Reduced Health-related Quality of Life and Functional Decline. Am J Respir Crit Care Med. 2019; 200(7):804-805. PMC: 6812450. DOI: 10.1164/rccm.201905-0977ED. View

15.

Zhai H, Brady P, Li Q, Lingren T, Ni Y, Wheeler D . Developing and evaluating a machine learning based algorithm to predict the need of pediatric intensive care unit transfer for newly hospitalized children. Resuscitation. 2014; 85(8):1065-71. PMC: 4087062. DOI: 10.1016/j.resuscitation.2014.04.009. View

16.

Choong K, Fraser D, Al-Harbi S, Borham A, Cameron J, Cameron S . Functional Recovery in Critically Ill Children, the "WeeCover" Multicenter Study. Pediatr Crit Care Med. 2018; 19(2):145-154. DOI: 10.1097/PCC.0000000000001421. View

17.

McCaffery K, Carey K, Campbell V, Gifford S, Smith K, Edelson D . Predicting transfers to intensive care in children using CEWT and other early warning systems. Resusc Plus. 2024; 17:100540. PMC: 10801303. DOI: 10.1016/j.resplu.2023.100540. View

18.

Chapman S, Maconochie I . Early warning scores in paediatrics: an overview. Arch Dis Child. 2018; 104(4):395-399. DOI: 10.1136/archdischild-2018-314807. View

19.

Hussain F, Sosa T, Ambroggio L, Gallagher R, Brady P . Emergency Transfers: An Important Predictor of Adverse Outcomes in Hospitalized Children. J Hosp Med. 2019; 14(8):482-485. PMC: 6686735. DOI: 10.12788/jhm.3219. View

20.

Winslow C, Edelson D, Churpek M, Taneja M, Shah N, Datta A . The Impact of a Machine Learning Early Warning Score on Hospital Mortality: A Multicenter Clinical Intervention Trial. Crit Care Med. 2022; 50(9):1339-1347. DOI: 10.1097/CCM.0000000000005492. View