Development and Pilot Evaluation of an Electronic Health Record Usability and Safety Self-assessment Tool

Overview

Journal JAMIA Open

Date 2022 Aug 3

PMID 35919379

Authors

Zoe Pruitt

Jessica L Howe

Seth A Krevat

Saif Khairat

Raj M Ratwani

Affiliations

Soon will be listed here.

Abstract

Objective: Poor electronic health record (EHR) usability contributes to clinician burnout and poses patent safety risks. Site-specific customization and configuration of EHRs require individual EHR system usability and safety testing which is resource intensive. We developed and pilot-tested a self-administered EHR usability and safety assessment tool, focused on computerized provider order entry (CPOE), which can be used by any facility to identify specific issues. In addition, the tool provides recommendations for improvement.

Materials And Methods: An assessment tool consisting of 104 questions was developed and pilot-tested at 2 hospitals, one using a Cerner EHR and the other using Epic. Five physicians at each site participated in and completed the assessment. Participant response accuracy compared to actual EHR interactions, consistency across participants, and usability issues identified through the tool were measured at each site.

Results: Across sites, participants answered an average of 46 questions in 23 min with 89.9% of responses either correct or partially correct. The tool identified 8 usability and safety issues at one site and 7 at the other site across medication, laboratory, and radiology CPOE functions.

Discussion: The tool shows promise as a method to rapidly evaluate EHR usability and safety and provide guidance on specific areas for improvement. Important improvements to the evaluation tool were identified including the need to clarify certain questions and provide definitions for usability terminology.

Conclusion: A self-administered usability and safety assessment tool can serve to identify specific usability and safety issues in the EHR and provide guidance for improvements.

Citing Articles

Realizing the Full Potential of Clinical Decision Support: Translating Usability Testing into Routine Practice in Health Care Operations.

Kandaswamy S, Williams H, Thompson S, Dawson T, Muthu N, Orenstein E Appl Clin Inform. 2024; 15(5):1039-1048.

PMID: 39191426 PMC: 11617071. DOI: 10.1055/a-2404-2129.

Inpatient EHR User Experience and Hospital EHR Safety Performance.

Classen D, Longhurst C, Davis T, Adler Milstein J, Bates D JAMA Netw Open. 2023; 6(9):e2333152.

PMID: 37695581 PMC: 10495862. DOI: 10.1001/jamanetworkopen.2023.33152.

References

Ratwani R, Fairbanks T, Savage E, Adams K, Wittie M, Boone E . Mind the Gap. A systematic review to identify usability and safety challenges and practices during electronic health record implementation. Appl Clin Inform. 2016; 7(4):1069-1087. PMC: 5228144. DOI: 10.4338/ACI-2016-06-R-0105. View

Gomes K, Ratwani R . Evaluating Improvements and Shortcomings in Clinician Satisfaction With Electronic Health Record Usability. JAMA Netw Open. 2019; 2(12):e1916651. PMC: 6991193. DOI: 10.1001/jamanetworkopen.2019.16651. View

Koppel R, Metlay J, Cohen A, Abaluck B, Russell Localio A, Kimmel S . Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005; 293(10):1197-203. DOI: 10.1001/jama.293.10.1197. View

Metzger J, Welebob E, Bates D, Lipsitz S, Classen D . Mixed results in the safety performance of computerized physician order entry. Health Aff (Millwood). 2010; 29(4):655-63. DOI: 10.1377/hlthaff.2010.0160. View

Guo J, Iribarren S, Kapsandoy S, Perri S, Staggers N . Usability Evaluation of An Electronic Medication Administration Record (eMAR) Application. Appl Clin Inform. 2013; 2(2):202-24. PMC: 3631925. DOI: 10.4338/ACI-2011-01-RA-0004. View

Khairat S, Burke G, Archambault H, Schwartz T, Larson J, Ratwani R . Perceived Burden of EHRs on Physicians at Different Stages of Their Career. Appl Clin Inform. 2018; 9(2):336-347. PMC: 5955717. DOI: 10.1055/s-0038-1648222. View

Howe J, Adams K, Hettinger A, Ratwani R . Electronic Health Record Usability Issues and Potential Contribution to Patient Harm. JAMA. 2018; 319(12):1276-1278. PMC: 5885839. DOI: 10.1001/jama.2018.1171. View

. Safe use of health information technology. Sentinel Event Alert. 2015; (54):1-6. View

Melnick E, Dyrbye L, Sinsky C, Trockel M, West C, Nedelec L . The Association Between Perceived Electronic Health Record Usability and Professional Burnout Among US Physicians. Mayo Clin Proc. 2019; 95(3):476-487. DOI: 10.1016/j.mayocp.2019.09.024. View

10.

Middleton B, Bloomrosen M, Dente M, Hashmat B, Koppel R, Overhage J . Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc. 2013; 20(e1):e2-8. PMC: 3715367. DOI: 10.1136/amiajnl-2012-001458. View

11.

Ratwani R, Savage E, Will A, Fong A, Karavite D, Muthu N . Identifying Electronic Health Record Usability And Safety Challenges In Pediatric Settings. Health Aff (Millwood). 2018; 37(11):1752-1759. DOI: 10.1377/hlthaff.2018.0699. View

12.

Ratwani R, Savage E, Will A, Arnold R, Khairat S, Miller K . A usability and safety analysis of electronic health records: a multi-center study. J Am Med Inform Assoc. 2018; 25(9):1197-1201. PMC: 7646875. DOI: 10.1093/jamia/ocy088. View

13.

Classen D, Holmgren A, Co Z, Newmark L, Seger D, Danforth M . National Trends in the Safety Performance of Electronic Health Record Systems From 2009 to 2018. JAMA Netw Open. 2020; 3(5):e205547. PMC: 7260621. DOI: 10.1001/jamanetworkopen.2020.5547. View

14.

Leung A, Keohane C, Lipsitz S, Zimlichman E, Amato M, Simon S . Relationship between medication event rates and the Leapfrog computerized physician order entry evaluation tool. J Am Med Inform Assoc. 2013; 20(e1):e85-90. PMC: 3715361. DOI: 10.1136/amiajnl-2012-001549. View

15.

Chaparro J, Classen D, Danforth M, Stockwell D, Longhurst C . National trends in safety performance of electronic health record systems in children's hospitals. J Am Med Inform Assoc. 2016; 24(2):268-274. PMC: 7651940. DOI: 10.1093/jamia/ocw134. View

16.

Shah T, Kitts A, Gold J, Horvath K, Ommaya A, Frank O . Electronic Health Record Optimization and Clinician Well-Being: A Potential Roadmap Toward Action. NAM Perspect. 2022; 2020. PMC: 8916811. DOI: 10.31478/202008a. View

17.

Fairbanks R, Bisantz A, Hettinger A, Ratwani R, Patterson E, Roth E . Usability in Health IT: Beyond Compliance to Meaningful Design and Assessment. Proc Hum Factors Ergon Soc Annu Meet. 2020; 60(1):643-646. PMC: 7536540. DOI: 10.1177/1541931213601148. View