Applying a Natural Language Processing Tool to Electronic Health Records to Assess Performance on Colonoscopy Quality Measures

Background: Gastroenterology specialty societies have advocated that providers routinely assess their performance on colonoscopy quality measures. Such routine measurement has been hampered by the costs and time required to manually review colonoscopy and pathology reports. Natural language processing (NLP) is a field of computer science in which programs are trained to extract relevant information from text reports in an automated fashion.

Objective: To demonstrate the efficiency and potential of NLP-based colonoscopy quality measurement.

Design: In a cross-sectional study design, we used a previously validated NLP program to analyze colonoscopy reports and associated pathology notes. The resulting data were used to generate provider performance on colonoscopy quality measures.

Setting: Nine hospitals in the University of Pittsburgh Medical Center health care system.

Patients: Study sample consisted of the 24,157 colonoscopy reports and associated pathology reports from 2008 to 2009.

Main Outcome Measurements: Provider performance on 7 quality measures.

Results: Performance on the colonoscopy quality measures was generally poor, and there was a wide range of performance. For example, across hospitals, the adequacy of preparation was noted overall in only 45.7% of procedures (range 14.6%-86.1% across 9 hospitals), cecal landmarks were documented in 62.7% of procedures (range 11.6%-90.0%), and the adenoma detection rate was 25.2% (range 14.9%-33.9%).

Limitations: Our quality assessment was limited to a single health care system in western Pennsylvania.

Conclusions: Our study illustrates how NLP can mine free-text data in electronic records to measure and report on the quality of care. Even within a single academic hospital system, there is considerable variation in the performance on colonoscopy quality measures, demonstrating the need for better methods to regularly and efficiently assess quality.