A Prospective Study of Consecutive Emergency Medical Admissions to Compare a Novel Automated Computer-aided Mortality Risk Score and Clinical Judgement of Patient Mortality Risk

Overview

Journal BMJ Open

Specialty General Medicine

Date 2019 Jun 22

PMID 31221885

Citations 2

Authors

Muhammad Faisal

Binish Khatoon

Andy Scally

Donald Richardson

Sally Irwin

Rachel Davidson

David Heseltine

Alison Corlett

Javed Ali

Rebecca Hampson

Sandeep Kesavan

Gerry McGonigal

Karen Goodman

Michael Harkness

Mohammed Mohammed

Affiliations

Soon will be listed here.

Abstract

Objectives: To compare the performance of a validated automatic computer-aided risk of mortality (CARM) score versus medical judgement in predicting the risk of in-hospital mortality for patients following emergency medical admission.

Design: A prospective study.

Setting: Consecutive emergency medical admissions in York hospital.

Participants: Elderly medical admissions in one ward were assigned a risk of death at the first post-take ward round by consultant staff over a 2-week period. The consultant medical staff used the same variables to assign a risk of death to the patient as the CARM (age, sex, National Early Warning Score and blood test results) but also had access to the clinical history, examination findings and any immediately available investigations such as ECGs. The performance of the CARM versus consultant medical judgement was compared using the c-statistic and the positive predictive value (PPV).

Results: The in-hospital mortality was 31.8% (130/409). For patients with complete blood test results, the c-statistic for CARM was 0.75 (95% CI: 0.69 to 0.81) versus 0.72 (95% CI: 0.66 to 0.78) for medical judgements (p=0.28). For patients with at least one missing blood test result, the c-statistics were similar (medical judgements 0.70 (95% CI: 0.60 to 0.81) vs CARM 0.70 (95% CI: 0.59 to 0.80)). At a 10% mortality risk, the PPV for CARM was higher than medical judgements in patients with complete blood test results, 62.0% (95% CI: 51.2 to 71.9) versus 49.2% (95% CI: 39.8 to 58.5) but not when blood test results were missing, 50.0% (95% CI: 24.7 to 75.3) versus 53.3% (95% CI: 34.3 to 71.7).

Conclusions: CARM is comparable with medical judgements in discriminating in-hospital mortality following emergency admission to an elderly care ward. CARM may have a promising role in supporting medical judgements in determining the patient's risk of death in hospital. Further evaluation of CARM in routine practice is required.

Citing Articles

Implementation and clinical utility of a Computer-Aided Risk Score for Mortality (CARM): a qualitative study.

Dyson J, McCrorie C, Benn J, Richardson D, Marsh C, Bowskill G BMJ Open. 2023; 13(1):e061298.

PMID: 36653055 PMC: 9853152. DOI: 10.1136/bmjopen-2022-061298.

Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study.

Faisal M, Mohammed M, Richardson D, Fiori M, Beatson K BMJ Open. 2022; 12(8):e050274.

PMID: 36041761 PMC: 9437732. DOI: 10.1136/bmjopen-2021-050274.

References

Brabrand M, Folkestad L, Clausen N, Knudsen T, Hallas J . Risk scoring systems for adults admitted to the emergency department: a systematic review. Scand J Trauma Resusc Emerg Med. 2010; 18:8. PMC: 2835641. DOI: 10.1186/1757-7241-18-8. View

McClish D, Powell S . How well can physicians estimate mortality in a medical intensive care unit?. Med Decis Making. 1989; 9(2):125-32. DOI: 10.1177/0272989X8900900207. View

Kruse J, Carlson R . Comparison of clinical assessment with APACHE II for predicting mortality risk in patients admitted to a medical intensive care unit. JAMA. 1988; 260(12):1739-42. View

Meyer A, Messick W, Young P, Baker C, Fakhry S, Muakkassa F . Prospective comparison of clinical judgment and APACHE II score in predicting the outcome in critically ill surgical patients. J Trauma. 1992; 32(6):747-53; discussion 753-4. DOI: 10.1097/00005373-199206000-00013. View

Sinuff T, Adhikari N, Cook D, Schunemann H, Griffith L, Rocker G . Mortality predictions in the intensive care unit: comparing physicians with scoring systems. Crit Care Med. 2006; 34(3):878-85. DOI: 10.1097/01.CCM.0000201881.58644.41. View

Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J . pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011; 12:77. PMC: 3068975. DOI: 10.1186/1471-2105-12-77. View

Goulden R, Hoyle M, Monis J, Railton D, Riley V, Martin P . qSOFA, SIRS and NEWS for predicting inhospital mortality and ICU admission in emergency admissions treated as sepsis. Emerg Med J. 2018; 35(6):345-349. DOI: 10.1136/emermed-2017-207120. View

Faisal M, Scally A, Jackson N, Richardson D, Beatson K, Howes R . Development and validation of a novel computer-aided score to predict the risk of in-hospital mortality for acutely ill medical admissions in two acute hospitals using their first electronically recorded blood test results and vital signs: a.... BMJ Open. 2018; 8(12):e022939. PMC: 6286481. DOI: 10.1136/bmjopen-2018-022939. View

Sanz J, Paternain D, Galar M, Fernandez J, Reyero D, Belzunegui T . A new survival status prediction system for severe trauma patients based on a multiple classifier system. Comput Methods Programs Biomed. 2017; 142:1-8. DOI: 10.1016/j.cmpb.2017.02.011. View

10.

Steyerberg E, Vickers A, Cook N, Gerds T, Gonen M, Obuchowski N . Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2009; 21(1):128-38. PMC: 3575184. DOI: 10.1097/EDE.0b013e3181c30fb2. View

11.

DeLong E, Delong D, Clarke-Pearson D . Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988; 44(3):837-45. View

12.

Hanley J, McNeil B . The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982; 143(1):29-36. DOI: 10.1148/radiology.143.1.7063747. View

13.

Smith M, Chiovaro J, ONeil M, Kansagara D, Quinones A, Freeman M . Early warning system scores for clinical deterioration in hospitalized patients: a systematic review. Ann Am Thorac Soc. 2014; 11(9):1454-65. DOI: 10.1513/AnnalsATS.201403-102OC. View

14.

Christensen C, Cottrell J, Murakami J, Mackesy M, Fetzer A, Elstein A . Forecasting survival in the medical intensive care unit: a comparison of clinical prognoses with formal estimates. Methods Inf Med. 1993; 32(4):302-8. View

15.

Brannen 2nd A, Godfrey L, Goetter W . Prediction of outcome from critical illness. A comparison of clinical judgment with a prediction rule. Arch Intern Med. 1989; 149(5):1083-6. View

16.

Pierpont G, Parenti C . Physician risk assessment and APACHE scores in cardiac care units. Clin Cardiol. 1999; 22(5):366-8. PMC: 6655287. DOI: 10.1002/clc.4960220514. View