Development and Application of Breadth-Depth-Context (BDC), a Conceptual Framework for Measuring Technology Engagement with a Qualified Clinical Data Registry

Overview

Journal JAMIA Open

Date 2024 Jul 29

PMID 39070967

Authors

Emma Kersey

Jing Li

Julia Kay

Julia Adler-Milstein

Jinoos Yazdany

Gabriela Schmajuk

Affiliations

Soon will be listed here.

Abstract

Objectives: Despite the proliferation of dashboards that display performance data derived from Qualified Clinical Data Registries (QCDR), the degree to which clinicians and practices engage with such dashboards has not been well described. We aimed to develop a conceptual framework for assessing user engagement with dashboard technology and to demonstrate its application to a rheumatology QCDR.

Materials And Methods: We developed the BDC (Breadth-Depth-Context) framework, which included concepts of breadth (derived from dashboard sessions), depth (derived from dashboard actions), and context (derived from practice characteristics). We demonstrated its application via user log data from the American College of Rheumatology's Rheumatology Informatics System for Effectiveness (RISE) registry to define engagement profiles and characterize practice-level factors associated with different profiles.

Results: We applied the BDC framework to 213 ambulatory practices from the RISE registry in 2020-2021, and classified practices into 4 engagement profiles: not engaged (8%), minimally engaged (39%), moderately engaged (34%), and most engaged (19%). Practices with more patients and with specific electronic health record vendors (eClinicalWorks and eMDs) had a higher likelihood of being in the most engaged group, even after adjusting for other factors.

Discussion: We developed the BDC framework to characterize user engagement with a registry dashboard and demonstrated its use in a specialty QCDR. The application of the BDC framework revealed a wide range of breadth and depth of use and that specific contextual factors were associated with nature of engagement.

Conclusion: Going forward, the BDC framework can be used to study engagement with similar dashboards.

References

Coles E, Anderson J, Maxwell M, Harris F, M Gray N, Milner G . The influence of contextual factors on healthcare quality improvement initiatives: a realist review. Syst Rev. 2020; 9(1):94. PMC: 7184709. DOI: 10.1186/s13643-020-01344-3. View

Khairat S, Marc D, Crosby W, Al Sanousi A . Reasons For Physicians Not Adopting Clinical Decision Support Systems: Critical Analysis. JMIR Med Inform. 2018; 6(2):e24. PMC: 5932331. DOI: 10.2196/medinform.8912. View

Miller R, Sim I, Newman J . Electronic medical records in solo/small groups: a qualitative study of physician user types. Stud Health Technol Inform. 2004; 107(Pt 1):658-62. View

Li J, Ringold S, Curtis J, Michaud K, Johansson T, Yun H . Effects of the SARS-CoV-2 global pandemic on U.S. rheumatology outpatient care delivery and use of telemedicine: an analysis of data from the RISE registry. Rheumatol Int. 2021; 41(10):1755-1761. PMC: 8327041. DOI: 10.1007/s00296-021-04960-x. View

Greenberg K, Poupko T, Sorotzkin D, Keidar O, Zwas D . Development and usage of a health recommendation web tool (HeaRT) designed to inform women of personalized preventive health recommendations. Internet Interv. 2023; 31:100599. PMC: 9813537. DOI: 10.1016/j.invent.2022.100599. View

Ho J, Corden M, Caccamo L, Tomasino K, Duffecy J, Begale M . Design and evaluation of a peer network to support adherence to a web-based intervention for adolescents. Internet Interv. 2016; 6:50-56. PMC: 5052813. DOI: 10.1016/j.invent.2016.09.005. View

Hester G, Lang T, Madsen L, Tambyraja R, Zenker P . Timely Data for Targeted Quality Improvement Interventions: Use of a Visual Analytics Dashboard for Bronchiolitis. Appl Clin Inform. 2019; 10(1):168-174. PMC: 6402943. DOI: 10.1055/s-0039-1679868. View

Mulhall C, Lam J, Rich P, Dobell L, Greenberg A . Enhancing Quality Care in Ontario Long-Term Care Homes Through Audit and Feedback for Physicians. J Am Med Dir Assoc. 2020; 21(3):420-425. DOI: 10.1016/j.jamda.2019.11.017. View

Jeffries M, Gude W, Keers R, Phipps D, Williams R, Kontopantelis E . Understanding the utilisation of a novel interactive electronic medication safety dashboard in general practice: a mixed methods study. BMC Med Inform Decis Mak. 2020; 20(1):69. PMC: 7164282. DOI: 10.1186/s12911-020-1084-5. View

10.

Brown B, Gude W, Blakeman T, van der Veer S, Ivers N, Francis J . Clinical Performance Feedback Intervention Theory (CP-FIT): a new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research. Implement Sci. 2019; 14(1):40. PMC: 6486695. DOI: 10.1186/s13012-019-0883-5. View

11.

Moynihan R, Sanders S, Michaleff Z, Scott A, Clark J, To E . Impact of COVID-19 pandemic on utilisation of healthcare services: a systematic review. BMJ Open. 2021; 11(3):e045343. PMC: 7969768. DOI: 10.1136/bmjopen-2020-045343. View

12.

Sanders J, Gokal K, Thomas J, Rawstorn J, Sherar L, Maddison R . Development of a Mobile Health Snacktivity App to Promote Physical Activity in Inactive Adults (SnackApp): Intervention Mapping and User Testing Study. JMIR Form Res. 2023; 7:e41114. PMC: 10242461. DOI: 10.2196/41114. View

13.

Smit D, Vrijsen J, Broekman T, Groeneweg B, Spijker J . User Engagement Within an Online Peer Support Community (Depression Connect) and Recovery-Related Changes in Empowerment: Longitudinal User Survey. JMIR Form Res. 2022; 6(11):e39912. PMC: 9669893. DOI: 10.2196/39912. View

14.

van Deen W, Cho E, Pustolski K, Wixon D, Lamb S, Valente T . Involving end-users in the design of an audit and feedback intervention in the emergency department setting - a mixed methods study. BMC Health Serv Res. 2019; 19(1):270. PMC: 6489283. DOI: 10.1186/s12913-019-4084-3. View

15.

Morton S, Shih S, Winther C, Tinoco A, Kessler R, Hudson Scholle S . Health IT-Enabled Care Coordination: A National Survey of Patient-Centered Medical Home Clinicians. Ann Fam Med. 2015; 13(3):250-6. PMC: 4427420. DOI: 10.1370/afm.1797. View

16.

Shrestha R, Altice F, Khati A, Azwa I, Gautam K, Gupta S . Clinic-Integrated Smartphone App (JomPrEP) to Improve Uptake of HIV Testing and Pre-exposure Prophylaxis Among Men Who Have Sex With Men in Malaysia: Mixed Methods Evaluation of Usability and Acceptability. JMIR Mhealth Uhealth. 2023; 11:e44468. PMC: 9982718. DOI: 10.2196/44468. View

17.

Ye J, Zhang R, Bannon J, Wang A, Walunas T, Kho A . Identifying Practice Facilitation Delays and Barriers in Primary Care Quality Improvement. J Am Board Fam Med. 2020; 33(5):655-664. DOI: 10.3122/jabfm.2020.05.200058. View

18.

Schnock K, Roulier S, Butler J, Dykes P, Fiskio J, Gibson B . Engaging Patients in the Use of Real-Time Electronic Clinical Data to Improve the Safety and Reliability of Their Own Care. J Patient Saf. 2021; 18(2):e407-e413. DOI: 10.1097/PTS.0000000000000831. View

19.

Neprash H, Vock D, Hanson A, Elert B, Short S, Karaca-Mandic P . Effect of Integrating Access to a Prescription Drug Monitoring Program Within the Electronic Health Record on the Frequency of Queries by Primary Care Clinicians: A Cluster Randomized Clinical Trial. JAMA Health Forum. 2022; 3(6):e221852. PMC: 9168784. DOI: 10.1001/jamahealthforum.2022.1852. View

20.

Jeffries M, Keers R, Phipps D, Williams R, Brown B, Avery A . Developing a learning health system: Insights from a qualitative process evaluation of a pharmacist-led electronic audit and feedback intervention to improve medication safety in primary care. PLoS One. 2018; 13(10):e0205419. PMC: 6203246. DOI: 10.1371/journal.pone.0205419. View