Improving Access to Online Health Information With Conversational Agents: A Randomized Controlled Experiment

Overview

Journal J Med Internet Res

Publisher JMIR Publications

Specialty Medical Informatics

Date 2016 Jan 6

PMID 26728964

Citations 41

Authors

Timothy W Bickmore

Dina Utami

Robin Matsuyama

Michael K Paasche-Orlow

Affiliations

Soon will be listed here.

Abstract

Background: Conventional Web-based search engines may be unusable by individuals with low health literacy for finding health-related information, thus precluding their use by this population.

Objective: We describe a conversational search engine interface designed to allow individuals with low health and computer literacy identify and learn about clinical trials on the Internet.

Methods: A randomized trial involving 89 participants compared the conversational search engine interface (n=43) to the existing conventional keyword- and facet-based search engine interface (n=46) for the National Cancer Institute Clinical Trials database. Each participant performed 2 tasks: finding a clinical trial for themselves and finding a trial that met prespecified criteria.

Results: Results indicated that all participants were more satisfied with the conversational interface based on 7-point self-reported satisfaction ratings (task 1: mean 4.9, SD 1.8 vs mean 3.2, SD 1.8, P<.001; task 2: mean 4.8, SD 1.9 vs mean 3.2, SD 1.7, P<.001) compared to the conventional Web form-based interface. All participants also rated the trials they found as better meeting their search criteria, based on 7-point self-reported scales (task 1: mean 3.7, SD 1.6 vs mean 2.7, SD 1.8, P=.01; task 2: mean 4.8, SD 1.7 vs mean 3.4, SD 1.9, P<.01). Participants with low health literacy failed to find any trials that satisfied the prespecified criteria for task 2 using the conventional search engine interface, whereas 36% (5/14) were successful at this task using the conversational interface (P=.05).

Conclusions: Conversational agents can be used to improve accessibility to Web-based searches in general and clinical trials in particular, and can help decrease recruitment bias against disadvantaged populations.

Citing Articles

Efficacy of a conversational chatbot for cigarette smoking cessation: Protocol of the QuitBot full-scale randomized controlled trial.

Bricker J, Sullivan B, Mull K, Lavista-Ferres J, Santiago-Torres M Contemp Clin Trials. 2024; 147:107727.

PMID: 39490766 PMC: 11620904. DOI: 10.1016/j.cct.2024.107727.

Effectiveness of the Medical Chatbot PROSCA to Inform Patients About Prostate Cancer: Results of a Randomized Controlled Trial.

Baumgartner K, Byczkowski M, Schmid T, Muschko M, Woessner P, Gerlach A Eur Urol Open Sci. 2024; 69:80-88.

PMID: 39329071 PMC: 11424957. DOI: 10.1016/j.euros.2024.08.022.

Recruitment in Appalachian, Rural and Older Adult Populations in an Artificial Intelligence World: Study Using Human-Mediated Follow-Up.

Milliken T, Beiler D, Hoffman S, Olenginski A, Troiani V JMIR Form Res. 2024; 8():e38189.

PMID: 39173153 PMC: 11377916. DOI: 10.2196/38189.

Conversational Chatbot for Cigarette Smoking Cessation: Results From the 11-Step User-Centered Design Development Process and Randomized Controlled Trial.

Bricker J, Sullivan B, Mull K, Santiago-Torres M, Lavista Ferres J JMIR Mhealth Uhealth. 2024; 12:e57318.

PMID: 38913882 PMC: 11303891. DOI: 10.2196/57318.

Validation of a Thai artificial chatmate designed for cheering up the elderly during the COVID-19 pandemic.

Deepaisarn S, Imkome E, Wongpatikaseree K, Yuenyong S, Lakanavisid P, Soonthornchaiva R F1000Res. 2024; 11:1411.

PMID: 38725544 PMC: 11079584. DOI: 10.12688/f1000research.127431.3.

References

Allam A, Schulz P, Nakamoto K . The impact of search engine selection and sorting criteria on vaccination beliefs and attitudes: two experiments manipulating Google output. J Med Internet Res. 2014; 16(4):e100. PMC: 4004139. DOI: 10.2196/jmir.2642. View

Ghaddar S, Valerio M, Garcia C, Hansen L . Adolescent health literacy: the importance of credible sources for online health information. J Sch Health. 2011; 82(1):28-36. DOI: 10.1111/j.1746-1561.2011.00664.x. View

Meppelink C, van Weert J, Haven C, Smit E . The effectiveness of health animations in audiences with different health literacy levels: an experimental study. J Med Internet Res. 2015; 17(1):e11. PMC: 4319081. DOI: 10.2196/jmir.3979. View

Agree E, King A, Castro C, Wiley A, Borzekowski D . "It's got to be on this page": age and cognitive style in a study of online health information seeking. J Med Internet Res. 2015; 17(3):e79. PMC: 4389106. DOI: 10.2196/jmir.3352. View

Bol N, van Weert J, de Haes H, Loos E, Smets E . The effect of modality and narration style on recall of online health information: results from a Web-based experiment. J Med Internet Res. 2015; 17(4):e104. PMC: 4425819. DOI: 10.2196/jmir.4164. View

Diviani N, van den Putte B, Giani S, Van Weert J . Low health literacy and evaluation of online health information: a systematic review of the literature. J Med Internet Res. 2015; 17(5):e112. PMC: 4468598. DOI: 10.2196/jmir.4018. View

Wang C, Bickmore T, Bowen D, Norkunas T, Campion M, Cabral H . Acceptability and feasibility of a virtual counselor (VICKY) to collect family health histories. Genet Med. 2015; 17(10):822-30. PMC: 4503525. DOI: 10.1038/gim.2014.198. View

Davis T, Long S, Jackson R, Mayeaux E, George R, Murphy P . Rapid estimate of adult literacy in medicine: a shortened screening instrument. Fam Med. 1993; 25(6):391-5. View

Paasche-Orlow M, Parker R, Gazmararian J, Nielsen-Bohlman L, Rudd R . The prevalence of limited health literacy. J Gen Intern Med. 2005; 20(2):175-84. PMC: 1490053. DOI: 10.1111/j.1525-1497.2005.40245.x. View

10.

Bickmore T, Gruber A, Picard R . Establishing the computer-patient working alliance in automated health behavior change interventions. Patient Educ Couns. 2005; 59(1):21-30. DOI: 10.1016/j.pec.2004.09.008. View

11.

Sudore R, Yaffe K, Satterfield S, Harris T, Mehta K, Simonsick E . Limited literacy and mortality in the elderly: the health, aging, and body composition study. J Gen Intern Med. 2006; 21(8):806-12. PMC: 1831586. DOI: 10.1111/j.1525-1497.2006.00539.x. View

12.

Mancuso C, Rincon M . Impact of health literacy on longitudinal asthma outcomes. J Gen Intern Med. 2006; 21(8):813-7. PMC: 1831574. DOI: 10.1111/j.1525-1497.2006.00528.x. View

13.

Lincoln A, Paasche-Orlow M, Cheng D, Lloyd-Travaglini C, Caruso C, Saitz R . Impact of health literacy on depressive symptoms and mental health-related: quality of life among adults with addiction. J Gen Intern Med. 2006; 21(8):818-22. PMC: 1831585. DOI: 10.1111/j.1525-1497.2006.00533.x. View

14.

Lindau S, Basu A, Leitsch S . Health literacy as a predictor of follow-up after an abnormal Pap smear: a prospective study. J Gen Intern Med. 2006; 21(8):829-34. PMC: 1831576. DOI: 10.1111/j.1525-1497.2006.00534.x. View

15.

Paasche-Orlow M, Wolf M . The causal pathways linking health literacy to health outcomes. Am J Health Behav. 2007; 31 Suppl 1:S19-26. DOI: 10.5555/ajhb.2007.31.supp.S19. View

16.

Atkinson N, Saperstein S, Massett H, Leonard C, Grama L, Manrow R . Using the Internet to search for cancer clinical trials: a comparative audit of clinical trial search tools. Contemp Clin Trials. 2008; 29(4):555-64. PMC: 2724745. DOI: 10.1016/j.cct.2008.01.007. View

17.

Volandes A, Paasche-Orlow M, Gillick M, Cook E, Shaykevich S, Abbo E . Health literacy not race predicts end-of-life care preferences. J Palliat Med. 2008; 11(5):754-62. DOI: 10.1089/jpm.2007.0224. View

18.

Bickmore T, Pfeifer L, Paasche-Orlow M . Using computer agents to explain medical documents to patients with low health literacy. Patient Educ Couns. 2009; 75(3):315-20. PMC: 2692364. DOI: 10.1016/j.pec.2009.02.007. View

19.

Mackert M, Kahlor L, Tyler D, Gustafson J . Designing e-health interventions for low-health-literate culturally diverse parents: addressing the obesity epidemic. Telemed J E Health. 2009; 15(7):672-7. PMC: 2956510. DOI: 10.1089/tmj.2009.0012. View

20.

Bickmore T, Pfeifer L, Byron D, Forsythe S, Henault L, Jack B . Usability of conversational agents by patients with inadequate health literacy: evidence from two clinical trials. J Health Commun. 2010; 15 Suppl 2:197-210. DOI: 10.1080/10810730.2010.499991. View