Effectiveness of the Medical Chatbot PROSCA to Inform Patients About Prostate Cancer: Results of a Randomized Controlled Trial

Overview

Journal Eur Urol Open Sci

Date 2024 Sep 27

PMID 39329071

Authors

Kilian Baumgartner

Michael Byczkowski

Tamara Schmid

Marc Muschko

Philipp Woessner

Axel Gerlach

David Bonekamp

Heinz-Peter Schlemmer

Markus Hohenfellner

Magdalena Gortz

Affiliations

Soon will be listed here.

Abstract

Background And Objective: Artificial intelligence (AI)-powered conversational agents are increasingly finding application in health care, as these can provide patient education at any time. However, their effectiveness in medical settings remains largely unexplored. This study aimed to assess the impact of the chatbot "PROState cancer Conversational Agent" (PROSCA), which was trained to provide validated support from diagnostic tests to treatment options for men facing prostate cancer (PC) diagnosis.

Methods: The chatbot PROSCA, developed by urologists at Heidelberg University Hospital and SAP SE, was evaluated through a randomized controlled trial (RCT). Patients were assigned to either the chatbot group, receiving additional access to PROSCA alongside standard information by urologists, or the control group (1:1), receiving standard information. A total of 112 men were included, of whom 103 gave feedback at study completion.

Key Findings And Limitations: Over time, patients' information needs decreased significantly more in the chatbot group than in the control group ( = 0.035). In the chatbot group, 43/54 men (79.6%) used PROSCA, and all of them found it easy to use. Of the men, 71.4% agreed that the chatbot improved their informedness about PC and 90.7% would like to use PROSCA again. Limitations are study sample size, single-center design, and specific clinical application.

Conclusions And Clinical Implications: With the introduction of the PROSCA chatbot, we created and evaluated an innovative, evidence-based AI health information tool as an additional source of information for PC. Our RCT results showed significant benefits of the chatbot in reducing patients' information needs and enhancing their understanding of PC. This easy-to-use AI tool provides accurate, timely, and accessible support, demonstrating its value in the PC diagnosis process. Future steps include further customization of the chatbot's responses and integration with the existing health care systems to maximize its impact on patient outcomes.

Patient Summary: This study evaluated an artificial intelligence-powered chatbot-PROSCA, a digital tool designed to support men facing prostate cancer diagnosis by providing validated information from diagnosis to treatment. Results showed that patients who used the chatbot as an additional tool felt better informed than those who received standard information from urologists. The majority of users appreciated the ease of use of the chatbot and expressed a desire to use it again; this suggests that PROSCA could be a valuable resource to improve patient understanding in prostate cancer diagnosis.

Citing Articles

Usefulness and Ease of Use of an Artificial Intelligence Chatbot for Kidney Stone Formers.

Huang M, Scotland K Cureus. 2025; 17(1):e77398.

PMID: 39949434 PMC: 11822055. DOI: 10.7759/cureus.77398.

References

Bibault J, Chaix B, Guillemasse A, Cousin S, Escande A, Perrin M . A Chatbot Versus Physicians to Provide Information for Patients With Breast Cancer: Blind, Randomized Controlled Noninferiority Trial. J Med Internet Res. 2019; 21(11):e15787. PMC: 6906616. DOI: 10.2196/15787. View

Bickmore T, Utami D, Matsuyama R, Paasche-Orlow M . Improving Access to Online Health Information With Conversational Agents: A Randomized Controlled Experiment. J Med Internet Res. 2016; 18(1):e1. PMC: 4717285. DOI: 10.2196/jmir.5239. View

Goerling U, Faller H, Hornemann B, Honig K, Bergelt C, Maatouk I . Information needs in cancer patients across the disease trajectory. A prospective study. Patient Educ Couns. 2019; 103(1):120-126. DOI: 10.1016/j.pec.2019.08.011. View

Siegel R, Miller K, Fuchs H, Jemal A . Cancer statistics, 2022. CA Cancer J Clin. 2022; 72(1):7-33. DOI: 10.3322/caac.21708. View

Ferber D, Kather J . Large Language Models in Uro-oncology. Eur Urol Oncol. 2023; 7(1):157-159. DOI: 10.1016/j.euo.2023.09.019. View

Hershenhouse J, Mokhtar D, Eppler M, Rodler S, Ramacciotti L, Ganjavi C . Accuracy, readability, and understandability of large language models for prostate cancer information to the public. Prostate Cancer Prostatic Dis. 2024; . DOI: 10.1038/s41391-024-00826-y. View

Palanica A, Flaschner P, Thommandram A, Li M, Fossat Y . Physicians' Perceptions of Chatbots in Health Care: Cross-Sectional Web-Based Survey. J Med Internet Res. 2019; 21(4):e12887. PMC: 6473203. DOI: 10.2196/12887. View

Gandaglia G, Leni R, Bray F, Fleshner N, Freedland S, Kibel A . Epidemiology and Prevention of Prostate Cancer. Eur Urol Oncol. 2021; 4(6):877-892. DOI: 10.1016/j.euo.2021.09.006. View

Cocci A, Pezzoli M, Lo Re M, Russo G, Asmundo M, Fode M . Quality of information and appropriateness of ChatGPT outputs for urology patients. Prostate Cancer Prostatic Dis. 2023; 27(1):103-108. DOI: 10.1038/s41391-023-00705-y. View

10.

Lamers R, Cuypers M, Husson O, de Vries M, Kil P, Bosch J . Patients are dissatisfied with information provision: perceived information provision and quality of life in prostate cancer patients. Psychooncology. 2015; 25(6):633-40. DOI: 10.1002/pon.3981. View

11.

Xu A, Myrie A, Taylor J, Matulewicz R, Gao T, Perez-Rosas V . Instagram and prostate cancer: using validated instruments to assess the quality of information on social media. Prostate Cancer Prostatic Dis. 2021; 25(4):791-793. DOI: 10.1038/s41391-021-00473-7. View

12.

Robbelen A, Schmieding M, Kopka M, Balzer F, Feufel M . Interactive Versus Static Decision Support Tools for COVID-19: Randomized Controlled Trial. JMIR Public Health Surveill. 2021; 8(4):e33733. PMC: 9015012. DOI: 10.2196/33733. View

13.

Huo B, Cacciamani G, Collins G, McKechnie T, Lee Y, Guyatt G . Reporting standards for the use of large language model-linked chatbots for health advice. Nat Med. 2023; 29(12):2988. DOI: 10.1038/s41591-023-02656-2. View

14.

de Vos I, Meertens A, Hogenhout R, Remmers S, Roobol M . A Detailed Evaluation of the Effect of Prostate-specific Antigen-based Screening on Morbidity and Mortality of Prostate Cancer: 21-year Follow-up Results of the Rotterdam Section of the European Randomised Study of Screening for Prostate Cancer. Eur Urol. 2023; 84(4):426-434. DOI: 10.1016/j.eururo.2023.03.016. View

15.

Denecke K, May R . Developing a Technical-Oriented Taxonomy to Define Archetypes of Conversational Agents in Health Care: Literature Review and Cluster Analysis. J Med Internet Res. 2023; 25:e41583. PMC: 9926340. DOI: 10.2196/41583. View

16.

Ghosh S, Bhatia S, Bhatia A . Quro: Facilitating User Symptom Check Using a Personalised Chatbot-Oriented Dialogue System. Stud Health Technol Inform. 2018; 252:51-56. View

17.

Milne-Ives M, de Cock C, Lim E, Harper Shehadeh M, de Pennington N, Mole G . The Effectiveness of Artificial Intelligence Conversational Agents in Health Care: Systematic Review. J Med Internet Res. 2020; 22(10):e20346. PMC: 7644372. DOI: 10.2196/20346. View

18.

Pan A, Musheyev D, Bockelman D, Loeb S, Kabarriti A . Assessment of Artificial Intelligence Chatbot Responses to Top Searched Queries About Cancer. JAMA Oncol. 2023; 9(10):1437-1440. PMC: 10450581. DOI: 10.1001/jamaoncol.2023.2947. View

19.

Rodler S, Kopliku R, Ulrich D, Kaltenhauser A, Casuscelli J, Eismann L . Patients' Trust in Artificial Intelligence-based Decision-making for Localized Prostate Cancer: Results from a Prospective Trial. Eur Urol Focus. 2023; 10(4):654-661. DOI: 10.1016/j.euf.2023.10.020. View

20.

Davis R, Eppler M, Ayo-Ajibola O, Loh-Doyle J, Nabhani J, Samplaski M . Evaluating the Effectiveness of Artificial Intelligence-powered Large Language Models Application in Disseminating Appropriate and Readable Health Information in Urology. J Urol. 2023; 210(4):688-694. DOI: 10.1097/JU.0000000000003615. View