Using Human Factors Methods to Mitigate Bias in Artificial Intelligence-based Clinical Decision Support

Overview

Journal J Am Med Inform Assoc

Publisher Oxford University Press

Specialty Medical Informatics

Date 2024 Nov 21

PMID 39569464

Authors

Laura G Militello

Julie Diiulio

Debbie L Wilson

Khoa A Nguyen

Christopher A Harle

Walid Gellad

Wei-Hsuan Lo-Ciganic

Affiliations

Soon will be listed here.

Abstract

Objectives: To highlight the often overlooked role of user interface (UI) design in mitigating bias in artificial intelligence (AI)-based clinical decision support (CDS).

Materials And Methods: This perspective paper discusses the interdependency between AI-based algorithm development and UI design and proposes strategies for increasing the safety and efficacy of CDS.

Results: The role of design in biasing user behavior is well documented in behavioral economics and other disciplines. We offer an example of how UI designs play a role in how bias manifests in our machine learning-based CDS development.

Discussion: Much discussion on bias in AI revolves around data quality and algorithm design; less attention is given to how UI design can exacerbate or mitigate limitations of AI-based applications.

Conclusion: This work highlights important considerations including the role of UI design in reinforcing/mitigating bias, human factors methods for identifying issues before an application is released, and risk communication strategies.

Citing Articles

Hot topics in artificial intelligence.

Bakken S, Poon E J Am Med Inform Assoc. 2025; 32(2):265-267.

PMID: 39836895 PMC: 11756649. DOI: 10.1093/jamia/ocae324.

References

Duffy S, Tabar L, Olsen A, Vitak B, Allgood P, Chen T . Absolute numbers of lives saved and overdiagnosis in breast cancer screening, from a randomized trial and from the Breast Screening Programme in England. J Med Screen. 2010; 17(1):25-30. PMC: 3104821. DOI: 10.1258/jms.2009.009094. View

Bailey C, Fisher C, Dvorak M . Type II error in the spine surgical literature. Spine (Phila Pa 1976). 2004; 29(10):1146-9. DOI: 10.1097/00007632-200405150-00018. View

Horsky J, Schiff G, Johnston D, Mercincavage L, Bell D, Middleton B . Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform. 2012; 45(6):1202-16. DOI: 10.1016/j.jbi.2012.09.002. View

OConnor A, Legare F, Stacey D . Risk communication in practice: the contribution of decision aids. BMJ. 2003; 327(7417):736-40. PMC: 200814. DOI: 10.1136/bmj.327.7417.736. View

Eaneff S, Obermeyer Z, Butte A . The Case for Algorithmic Stewardship for Artificial Intelligence and Machine Learning Technologies. JAMA. 2020; 324(14):1397-1398. DOI: 10.1001/jama.2020.9371. View

Olson J, Hollenbeak C, Donaldson K, Abendroth T, Castellani W . Default settings of computerized physician order entry system order sets drive ordering habits. J Pathol Inform. 2015; 6:16. PMC: 4382759. DOI: 10.4103/2153-3539.153916. View

Militello L, Saleem J, Borders M, Sushereba C, Haverkamp D, Wolf S . Designing Colorectal Cancer Screening Decision Support: A Cognitive Engineering Enterprise. J Cogn Eng Decis Mak. 2016; 10(1):74-90. PMC: 4784691. DOI: 10.1177/1555343416630875. View

Bastien J . Usability testing: a review of some methodological and technical aspects of the method. Int J Med Inform. 2009; 79(4):e18-23. DOI: 10.1016/j.ijmedinf.2008.12.004. View

Obermeyer Z, Powers B, Vogeli C, Mullainathan S . Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019; 366(6464):447-453. DOI: 10.1126/science.aax2342. View

10.

Naik G, Ahmed H, Edwards A . Communicating risk to patients and the public. Br J Gen Pract. 2012; 62(597):213-6. PMC: 3310025. DOI: 10.3399/bjgp12X636236. View

11.

Sloss E, McPherson J, Beck A, Guo J, Scheese C, Flake N . Patient and Caregiver Perceptions of an Interface Design to Communicate Artificial Intelligence-Based Prognosis for Patients With Advanced Solid Tumors. JCO Clin Cancer Inform. 2024; 8:e2300187. PMC: 11161249. DOI: 10.1200/CCI.23.00187. View

12.

Bottorff J, Ratner P, Johnson J, Lovato C, Joab S . Communicating cancer risk information: the challenges of uncertainty. Patient Educ Couns. 1998; 33(1):67-81. DOI: 10.1016/s0738-3991(97)00047-5. View

13.

Brust-Renck P, Royer C, Reyna V . Communicating Numerical Risk: Human Factors That Aid Understanding in Health Care. Rev Hum Factors Ergon. 2014; 8(1):235-276. PMC: 4078918. DOI: 10.1177/1557234X13492980. View

14.

Wawira Gichoya J, McCoy L, Celi L, Ghassemi M . Equity in essence: a call for operationalising fairness in machine learning for healthcare. BMJ Health Care Inform. 2021; 28(1). PMC: 8733939. DOI: 10.1136/bmjhci-2020-100289. View

15.

Diehl C, Martins A, Almeida A, Silva T, Ribeiro O, Santinha G . Defining Recommendations to Guide User Interface Design: Multimethod Approach. JMIR Hum Factors. 2022; 9(3):e37894. PMC: 9568819. DOI: 10.2196/37894. View

16.

Ti L, Ho A, Knight R . Towards Equitable AI Interventions for People Who Use Drugs: Key Areas That Require Ethical Investment. J Addict Med. 2020; 15(2):96-98. DOI: 10.1097/ADM.0000000000000722. View

17.

Chouldechova A . Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments. Big Data. 2017; 5(2):153-163. DOI: 10.1089/big.2016.0047. View

18.

Staes C, Beck A, Chalkidis G, Scheese C, Taft T, Guo J . Design of an interface to communicate artificial intelligence-based prognosis for patients with advanced solid tumors: a user-centered approach. J Am Med Inform Assoc. 2023; 31(1):174-187. PMC: 10746322. DOI: 10.1093/jamia/ocad201. View

19.

Bell N, Dickinson J, Grad R, Singh H, Kasperavicius D, Thombs B . Understanding and communicating risk: Measures of outcome and the magnitude of benefits and harms. Can Fam Physician. 2018; 64(3):181-185. PMC: 5851390. View