Capturing Surgical Data: Comparing a Quality Improvement Registry to Natural Language Processing and Manual Chart Review

Overview

Journal J Gastrointest Surg

Specialty Gastroenterology

Date 2022 Mar 1

PMID 35229252

Authors

Benjamin T Miller

Aldo Fafaj

Luciano Tastaldi

Hemasat Alkhatib

Samuel Zolin

Raha AlMarzooqi

Chao Tu

Diya Alaedeen

Ajita S Prabhu

David M Krpata

Michael J Rosen

Clayton C Petro

Affiliations

Soon will be listed here.

Abstract

Introduction: Collecting accurate operative details remains a limitation of surgical research. Surgeon-entered data in clinical registries offers one solution, but natural language processing (NLP) has emerged as a modality for automating manual chart review (MCR). This study aims to compare the accuracy and efficiency of NLP and MCR with a surgeon-entered, prospective registry data in determining the rate of gross bile spillage (GBS) during cholecystectomy.

Methods: Bile spillage rates were abstracted from an institutional, surgeon-entered clinical registry from July 2018 to January 2019. These rates were compared to those documented in the electronic medical record (EMR) using NLP and MCR to determine the sensitivity, specificity, and efficiency of each approach.

Results: Of the 782 registry entries, 191 cases (24.4%) had surgeon-reported bile spillage. MCR identified bile spillage in 121 cases (15.6%); however, bile spillage information was either missing or ambiguous in 454 cases (58.1%). NLP identified 99 cases (12.7%) of bile spillage. Data abstraction times for the registry, NLP, and MCR were 3 min, 5 min, and 12 h, respectively. When compared to the registry, MCR was 45% sensitive and 94% specific, while NLP was 27.2% sensitive and 92% specific for detecting bile spillage. These differences were significant (X = 19.446, P = < 0.001).

Conclusion: Operative details, such as GBS, may not be abstracted by NLP or MCR if not clearly documented in the EMR. Clinical registries capture operative details, but they rely on surgeons to input the data.

References

Edhemovic I, Temple W, de Gara C, Stuart G . The computer synoptic operative report--a leap forward in the science of surgery. Ann Surg Oncol. 2004; 11(10):941-7. DOI: 10.1245/ASO.2004.12.045. View

Payne K, Jones K, Dickenson A . Improving the Standard of Operative Notes within an Oral and Maxillofacial Surgery Department, using an Operative Note Proforma. J Maxillofac Oral Surg. 2012; 10(3):203-8. PMC: 3238552. DOI: 10.1007/s12663-011-0231-z. View

Singh R, Chauhan R, Anwar S . Improving the quality of general surgical operation notes in accordance with the Royal College of Surgeons guidelines: a prospective completed audit loop study. J Eval Clin Pract. 2011; 18(3):578-80. DOI: 10.1111/j.1365-2753.2010.01626.x. View

Khuri S . The NSQIP: a new frontier in surgery. Surgery. 2005; 138(5):837-43. DOI: 10.1016/j.surg.2005.08.016. View

Poulose B, Roll S, Murphy J, Matthews B, Heniford B, Voeller G . Design and implementation of the Americas Hernia Society Quality Collaborative (AHSQC): improving value in hernia care. Hernia. 2016; 20(2):177-89. DOI: 10.1007/s10029-016-1477-7. View

Tvardik N, Kergourlay I, Bittar A, Segond F, Darmoni S, Metzger M . Accuracy of using natural language processing methods for identifying healthcare-associated infections. Int J Med Inform. 2018; 117:96-102. DOI: 10.1016/j.ijmedinf.2018.06.002. View

Koleck T, Dreisbach C, Bourne P, Bakken S . Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review. J Am Med Inform Assoc. 2019; 26(4):364-379. PMC: 6657282. DOI: 10.1093/jamia/ocy173. View

Branch-Elliman W, Strymish J, Kudesia V, Rosen A, Gupta K . Natural Language Processing for Real-Time Catheter-Associated Urinary Tract Infection Surveillance: Results of a Pilot Implementation Trial. Infect Control Hosp Epidemiol. 2015; 36(9):1004-10. DOI: 10.1017/ice.2015.122. View

Peponis T, Eskesen T, Mesar T, Saillant N, Kaafarani H, Yeh D . Bile Spillage as a Risk Factor for Surgical Site Infection after Laparoscopic Cholecystectomy: A Prospective Study of 1,001 Patients. J Am Coll Surg. 2018; 226(6):1030-1035. DOI: 10.1016/j.jamcollsurg.2017.11.025. View

10.

Mehta N, Devarakonda M . Machine learning, natural language programming, and electronic health records: The next step in the artificial intelligence journey?. J Allergy Clin Immunol. 2018; 141(6):2019-2021.e1. DOI: 10.1016/j.jaci.2018.02.025. View

11.

Thomson D, Baldwin M, Bellini M, Silva M . Improving the quality of operative notes for laparoscopic cholecystectomy: Assessing the impact of a standardized operation note proforma. Int J Surg. 2016; 27:17-20. DOI: 10.1016/j.ijsu.2016.01.037. View

12.

Cohen M, Liu Y, Ko C, Hall B . Improved Surgical Outcomes for ACS NSQIP Hospitals Over Time: Evaluation of Hospital Cohorts With up to 8 Years of Participation. Ann Surg. 2015; 263(2):267-73. DOI: 10.1097/SLA.0000000000001192. View

13.

Henderson W, Daley J . Design and statistical methodology of the National Surgical Quality Improvement Program: why is it what it is?. Am J Surg. 2009; 198(5 Suppl):S19-27. DOI: 10.1016/j.amjsurg.2009.07.025. View