Applying Cognitive Walkthrough Methodology to Improve the Usability of an Equity-focused Implementation Strategy

Overview

Journal Implement Sci Commun

Publisher Biomed Central

Specialty Health Services

Date 2024 Sep 3

PMID 39227912

Authors

Kelly A Aschbrenner

Emily R Haines

Gina R Kruse

Ayotola O Olugbenga

Annette N Thomas

Tanveer Khan

Stephanie Martinez

Karen M Emmons

Stephen J Bartels

Affiliations

Soon will be listed here.

Abstract

Background: Our research team partnered with primary care and quality improvement staff in Federally Qualified Community Health Centers (CHCs) to develop Partnered and Equity Data-Driven Implementation (PEDDI) to promote equitable implementation of evidence-based interventions. The current study used a human-centered design methodology to evaluate the usability of PEDDI and generate redesign solutions to address usability issues in the context of a cancer screening intervention.

Methods: We applied the Cognitive Walkthrough for Implementation Strategies (CWIS), a pragmatic assessment method with steps that include group testing with end users to identify and prioritize usability problems. We conducted three facilitated 60-min CWIS sessions with end users (N = 7) from four CHCs that included scenarios and related tasks for implementing a colorectal cancer (CRC) screening intervention. Participants rated the likelihood of completing each task and identified usability issues and generated ideas for redesign solutions during audio-recorded CWIS sessions. Participants completed a pre-post survey of PEDDI usability. Our research team used consensus coding to synthesize usability problems and redesign solutions from transcribed CWIS sessions.

Results: Usability ratings (scale 0-100: higher scores indicating higher usability) of PEDDI averaged 66.3 (SD = 12.4) prior to the CWIS sessions. Scores averaged 77.8 (SD = 9.1) following the three CWIS sessions improving usability ratings from "marginal acceptability" to "acceptable". Ten usability problems were identified across four PEDDI tasks, comprised of 2-3 types of usability problems per task. CWIS participants suggested redesign solutions that included making data fields for social determinants of health and key background variables for identifying health equity targets mandatory in the electronic health record and using asynchronous communication tools to elicit ideas from staff for adaptations.

Conclusions: Usability ratings indicated PEDDI was in the acceptable range following CWIS sessions. Staff identified usability problems and redesign solutions that provide direction for future improvements in PEDDI. In addition, this study highlights opportunities to use the CWIS methodology to address inequities in the implementation of cancer screening and other clinical innovations in resource-constrained healthcare settings.

References

Gruss I, Bunce A, Davis J, Dambrun K, Cottrell E, Gold R . Initiating and Implementing Social Determinants of Health Data Collection in Community Health Centers. Popul Health Manag. 2020; 24(1):52-58. PMC: 7875127. DOI: 10.1089/pop.2019.0205. View

Kruse G, Lee R, Aschbrenner K, Daly J, Dargon-Hart S, Davies M . Embedding community-engaged research principles in implementation science: The implementation science center for cancer control equity. J Clin Transl Sci. 2023; 7(1):e82. PMC: 10130842. DOI: 10.1017/cts.2023.32. View

Nishii A, Campos-Castillo C, Anthony D . Disparities in patient portal access by US adults before and during the COVID-19 pandemic. JAMIA Open. 2022; 5(4):ooac104. PMC: 9757864. DOI: 10.1093/jamiaopen/ooac104. View

Ogrinc G, Dolansky M, Berman A, Chambers D, Davies L . Different approaches to making and testing change in healthcare. BMJ. 2021; 374:n1010. PMC: 8369383. DOI: 10.1136/bmj.n1010. View

Richardson-Parry A, Silva M, Valderas J, Donde S, Woodruff S, van Vugt J . Interactive or tailored digital interventions to increase uptake in cervical, breast, and colorectal cancer screening to reduce health inequity: a systematic review. Eur J Cancer Prev. 2023; 32(4):396-409. PMC: 10249608. DOI: 10.1097/CEJ.0000000000000796. View

Baumann A, Shelton R, Kumanyika S, Haire-Joshu D . Advancing healthcare equity through dissemination and implementation science. Health Serv Res. 2023; 58 Suppl 3:327-344. PMC: 10684051. DOI: 10.1111/1475-6773.14175. View

Ponce-Chazarri L, Ponce-Blandon J, Immordino P, Giordano A, Morales F . Barriers to Breast Cancer-Screening Adherence in Vulnerable Populations. Cancers (Basel). 2023; 15(3). PMC: 9913751. DOI: 10.3390/cancers15030604. View

Moyal-Smith R, Etheridge J, Karlage A, Sonnay Y, Yuan C, Havens J . Defining re-implementation. Implement Sci Commun. 2023; 4(1):60. PMC: 10240764. DOI: 10.1186/s43058-023-00440-4. View

Steitz B, Unertl K, Levy M . An Analysis of Electronic Health Record Work to Manage Asynchronous Clinical Messages among Breast Cancer Care Teams. Appl Clin Inform. 2021; 12(4):877-887. PMC: 8443401. DOI: 10.1055/s-0041-1735257. View

10.

Chambers D . Advancing adaptation of evidence-based interventions through implementation science: progress and opportunities. Front Health Serv. 2023; 3:1204138. PMC: 10277471. DOI: 10.3389/frhs.2023.1204138. View

11.

Rabin B, McCreight M, Battaglia C, Ayele R, Burke R, Hess P . Systematic, Multimethod Assessment of Adaptations Across Four Diverse Health Systems Interventions. Front Public Health. 2018; 6:102. PMC: 5900443. DOI: 10.3389/fpubh.2018.00102. View

12.

Lin Y, Liang L, Ding R, Prosper A, Aberle D, Hsu W . Factors Associated With Nonadherence to Lung Cancer Screening Across Multiple Screening Time Points. JAMA Netw Open. 2023; 6(5):e2315250. PMC: 10214032. DOI: 10.1001/jamanetworkopen.2023.15250. View

13.

Leeman J, Rohweder C, Lee M, Brenner A, Dwyer A, Ko L . Aligning implementation science with improvement practice: a call to action. Implement Sci Commun. 2021; 2(1):99. PMC: 8424169. DOI: 10.1186/s43058-021-00201-1. View

14.

Kerkhoff A, Farrand E, Marquez C, Cattamanchi A, Handley M . Addressing health disparities through implementation science-a need to integrate an equity lens from the outset. Implement Sci. 2022; 17(1):13. PMC: 8802460. DOI: 10.1186/s13012-022-01189-5. View

15.

Cook L, Sachs J, Weiskopf N . The quality of social determinants data in the electronic health record: a systematic review. J Am Med Inform Assoc. 2021; 29(1):187-196. PMC: 8714289. DOI: 10.1093/jamia/ocab199. View

16.

Vega Perez R, Hayden L, Mesa J, Bickell N, Abner P, Richardson L . Improving Patient Race and Ethnicity Data Capture to Address Health Disparities: A Case Study From a Large Urban Health System. Cureus. 2022; 14(1):e20973. PMC: 8815799. DOI: 10.7759/cureus.20973. View

17.

Lew N, Sommers B . Addressing Social Determinants of Health in Federal Programs. JAMA Health Forum. 2022; 3(3):e221064. DOI: 10.1001/jamahealthforum.2022.1064. View

18.

Muthukrishnan M, Arnold L, James A . Patients' self-reported barriers to colon cancer screening in federally qualified health center settings. Prev Med Rep. 2019; 15:100896. PMC: 6531912. DOI: 10.1016/j.pmedr.2019.100896. View

19.

Kahn M, Callahan T, Barnard J, Bauck A, Brown J, Davidson B . A Harmonized Data Quality Assessment Terminology and Framework for the Secondary Use of Electronic Health Record Data. EGEMS (Wash DC). 2016; 4(1):1244. PMC: 5051581. DOI: 10.13063/2327-9214.1244. View

20.

Steitz B, Sulieman L, Warner J, Fabbri D, Brown J, Davis A . Classification and analysis of asynchronous communication content between care team members involved in breast cancer treatment. JAMIA Open. 2021; 4(3):ooab049. PMC: 8358477. DOI: 10.1093/jamiaopen/ooab049. View