Validation of Automated Sepsis Surveillance Based on the Sepsis-3 Clinical Criteria Against Physician Record Review in a General Hospital Population: Observational Study Using Electronic Health Records Data

Overview

Journal BMJ Qual Saf

Specialty Health Services

Date 2020 Feb 8

PMID 32029574

Citations 23

Authors

John Karlsson Valik

Logan Ward

Hideyuki Tanushi

Kajsa Mullersdorf

Anders Ternhag

Ewa Aufwerber

Anna Farnert

Anders F Johansson

Mads Lause Mogensen

Brian Pickering

Hercules Dalianis

Aron Henriksson

Vitaly Herasevich

Pontus Naucler

Affiliations

Soon will be listed here.

Abstract

Background: Surveillance of sepsis incidence is important for directing resources and evaluating quality-of-care interventions. The aim was to develop and validate a fully-automated Sepsis-3 based surveillance system in non-intensive care wards using electronic health record (EHR) data, and demonstrate utility by determining the burden of hospital-onset sepsis and variations between wards.

Methods: A rule-based algorithm was developed using EHR data from a cohort of all adult patients admitted at an academic centre between July 2012 and December 2013. Time in intensive care units was censored. To validate algorithm performance, a stratified random sample of 1000 hospital admissions (674 with and 326 without suspected infection) was classified according to the Sepsis-3 clinical criteria (suspected infection defined as having any culture taken and at least two doses of antimicrobials administered, and an increase in Sequential Organ Failure Assessment (SOFA) score by 2 points) and the likelihood of infection by physician medical record review.

Results: In total 82 653 hospital admissions were included. The Sepsis-3 clinical criteria determined by physician review were met in 343 of 1000 episodes. Among them, 313 (91%) had possible, probable or definite infection. Based on this reference, the algorithm achieved sensitivity 0.887 (95% CI: 0.799 to 0.964), specificity 0.985 (95% CI: 0.978 to 0.991), positive predictive value 0.881 (95% CI: 0.833 to 0.926) and negative predictive value 0.986 (95% CI: 0.973 to 0.996). When applied to the total cohort taking into account the sampling proportions of those with and without suspected infection, the algorithm identified 8599 (10.4%) sepsis episodes. The burden of hospital-onset sepsis (>48 hour after admission) and related in-hospital mortality varied between wards.

Conclusions: A fully-automated Sepsis-3 based surveillance algorithm using EHR data performed well compared with physician medical record review in non-intensive care wards, and exposed variations in hospital-onset sepsis incidence between wards.

Citing Articles

A dynamic customized electronic health record rule based clinical decision support tool for standardized adult intensive care metrics.

Cucchi E, Burzynski J, Marshall N, Greenberg B JAMIA Open. 2024; 7(4):ooae143.

PMID: 39664648 PMC: 11633943. DOI: 10.1093/jamiaopen/ooae143.

Automated Identification of Postoperative Infections to Allow Prediction and Surveillance Based on Electronic Health Record Data: Scoping Review.

van der Meijden S, van Boekel A, Goor H, Nelissen R, Schoones J, Steyerberg E JMIR Med Inform. 2024; 12:e57195.

PMID: 39255011 PMC: 11422734. DOI: 10.2196/57195.

Validity of Routinely Reported Rutherford Scores Reported by Clinicians as Part of Daily Clinical Practice.

van der Heijden L, Marang-van de Mheen P, Thielman L, Stijnen P, Hamming J, Fourneau I Int J Angiol. 2024; 33(3):148-155.

PMID: 39131806 PMC: 11315596. DOI: 10.1055/s-0043-1761280.

A scoping review of the methodological approaches used in retrospective chart reviews to validate adverse event rates in administrative data.

Connolly A, Kirwan M, Matthews A Int J Qual Health Care. 2024; 36(2).

PMID: 38662407 PMC: 11086704. DOI: 10.1093/intqhc/mzae037.

The accuracy of fully-automated algorithms for the surveillance of central venous catheter-related bloodstream infection in hospitalised patients.

Karmefors Idvall M, Tanushi H, Berge A, Naucler P, van der Werff S Antimicrob Resist Infect Control. 2024; 13(1):15.

PMID: 38317207 PMC: 10840273. DOI: 10.1186/s13756-024-01373-w.

References

Rhee C, Gohil S, Klompas M . Regulatory mandates for sepsis care--reasons for caution. N Engl J Med. 2014; 370(18):1673-6. PMC: 4718398. DOI: 10.1056/NEJMp1400276. View

Singer M, Deutschman C, Seymour C, Shankar-Hari M, Annane D, Bauer M . The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016; 315(8):801-10. PMC: 4968574. DOI: 10.1001/jama.2016.0287. View

Levy M, Evans L, Rhodes A . The Surviving Sepsis Campaign Bundle: 2018 Update. Crit Care Med. 2018; 46(6):997-1000. DOI: 10.1097/CCM.0000000000003119. View

Reinhart K, Daniels R, Kissoon N, Machado F, Schachter R, Finfer S . Recognizing Sepsis as a Global Health Priority - A WHO Resolution. N Engl J Med. 2017; 377(5):414-417. DOI: 10.1056/NEJMp1707170. View

Mellhammar L, Wullt S, Lindberg A, Lanbeck P, Christensson B, Linder A . Sepsis Incidence: A Population-Based Study. Open Forum Infect Dis. 2016; 3(4):ofw207. PMC: 5144652. DOI: 10.1093/ofid/ofw207. View

Aakre C, Moreno Franco P, Ferreyra M, Kitson J, Li M, Herasevich V . Prospective validation of a near real-time EHR-integrated automated SOFA score calculator. Int J Med Inform. 2017; 103:1-6. DOI: 10.1016/j.ijmedinf.2017.04.001. View

Rhee C, Wang R, Zhang Z, Fram D, Kadri S, Klompas M . Epidemiology of Hospital-Onset Versus Community-Onset Sepsis in U.S. Hospitals and Association With Mortality: A Retrospective Analysis Using Electronic Clinical Data. Crit Care Med. 2019; 47(9):1169-1176. PMC: 6697188. DOI: 10.1097/CCM.0000000000003817. View

Prescott H, Cope T, Gesten F, Ledneva T, Friedrich M, Iwashyna T . Reporting of Sepsis Cases for Performance Measurement Versus for Reimbursement in New York State. Crit Care Med. 2018; 46(5):666-673. PMC: 5899037. DOI: 10.1097/CCM.0000000000003005. View

Page D, Donnelly J, Wang H . Community-, Healthcare-, and Hospital-Acquired Severe Sepsis Hospitalizations in the University HealthSystem Consortium. Crit Care Med. 2015; 43(9):1945-51. PMC: 4537676. DOI: 10.1097/CCM.0000000000001164. View

10.

Wang H, Addis D, Donnelly J, Shapiro N, Griffin R, Safford M . Discharge diagnoses versus medical record review in the identification of community-acquired sepsis. Crit Care. 2015; 19:42. PMC: 4340494. DOI: 10.1186/s13054-015-0771-6. View

11.

Rhee C, Jentzsch M, Kadri S, Seymour C, Angus D, Murphy D . Variation in Identifying Sepsis and Organ Dysfunction Using Administrative Versus Electronic Clinical Data and Impact on Hospital Outcome Comparisons. Crit Care Med. 2018; 47(4):493-500. PMC: 7970408. DOI: 10.1097/CCM.0000000000003554. View

12.

Henriksen D, Laursen C, Jensen T, Hallas J, Pedersen C, Lassen A . Incidence rate of community-acquired sepsis among hospitalized acute medical patients-a population-based survey. Crit Care Med. 2014; 43(1):13-21. DOI: 10.1097/CCM.0000000000000611. View

13.

Rhee C, Dantes R, Epstein L, Klompas M . Using objective clinical data to track progress on preventing and treating sepsis: CDC's new 'Adult Sepsis Event' surveillance strategy. BMJ Qual Saf. 2018; 28(4):305-309. PMC: 6557151. DOI: 10.1136/bmjqs-2018-008331. View

14.

Rhee C, Kadri S, Danner R, Suffredini A, Massaro A, Kitch B . Diagnosing sepsis is subjective and highly variable: a survey of intensivists using case vignettes. Crit Care. 2016; 20:89. PMC: 4822273. DOI: 10.1186/s13054-016-1266-9. View

15.

Gaieski D, Edwards J, Kallan M, Carr B . Benchmarking the incidence and mortality of severe sepsis in the United States. Crit Care Med. 2013; 41(5):1167-74. DOI: 10.1097/CCM.0b013e31827c09f8. View

16.

Quan H, Li B, Couris C, Fushimi K, Graham P, Hider P . Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol. 2011; 173(6):676-82. DOI: 10.1093/aje/kwq433. View

17.

Fleischmann-Struzek C, Mikolajetz A, Schwarzkopf D, Cohen J, Hartog C, Pletz M . Challenges in assessing the burden of sepsis and understanding the inequalities of sepsis outcomes between National Health Systems: secular trends in sepsis and infection incidence and mortality in Germany. Intensive Care Med. 2018; 44(11):1826-1835. PMC: 6244521. DOI: 10.1007/s00134-018-5377-4. View

18.

van Mourik M, Perencevich E, Gastmeier P, Bonten M . Designing Surveillance of Healthcare-Associated Infections in the Era of Automation and Reporting Mandates. Clin Infect Dis. 2018; 66(6):970-976. DOI: 10.1093/cid/cix835. View

19.

Seymour C, Gesten F, Prescott H, Friedrich M, Iwashyna T, Phillips G . Time to Treatment and Mortality during Mandated Emergency Care for Sepsis. N Engl J Med. 2017; 376(23):2235-2244. PMC: 5538258. DOI: 10.1056/NEJMoa1703058. View

20.

Klein Klouwenberg P, Cremer O, van Vught L, Ong D, Frencken J, Schultz M . Likelihood of infection in patients with presumed sepsis at the time of intensive care unit admission: a cohort study. Crit Care. 2015; 19:319. PMC: 4562354. DOI: 10.1186/s13054-015-1035-1. View