Randomized Controlled Trials of Artificial Intelligence in Clinical Practice: Systematic Review

Overview

Journal J Med Internet Res

Publisher JMIR Publications

Specialty Medical Informatics

Date 2022 Jul 29

PMID 35904087

Authors

Thomas Y T Lam

Max F K Cheung

Yasmin L Munro

Kong Meng Lim

Dennis Shung

Joseph J Y Sung

Affiliations

Soon will be listed here.

Abstract

Background: The number of artificial intelligence (AI) studies in medicine has exponentially increased recently. However, there is no clear quantification of the clinical benefits of implementing AI-assisted tools in patient care.

Objective: This study aims to systematically review all published randomized controlled trials (RCTs) of AI-assisted tools to characterize their performance in clinical practice.

Methods: CINAHL, Cochrane Central, Embase, MEDLINE, and PubMed were searched to identify relevant RCTs published up to July 2021 and comparing the performance of AI-assisted tools with conventional clinical management without AI assistance. We evaluated the primary end points of each study to determine their clinical relevance. This systematic review was conducted following the updated PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines.

Results: Among the 11,839 articles retrieved, only 39 (0.33%) RCTs were included. These RCTs were conducted in an approximately equal distribution from North America, Europe, and Asia. AI-assisted tools were implemented in 13 different clinical specialties. Most RCTs were published in the field of gastroenterology, with 15 studies on AI-assisted endoscopy. Most RCTs studied biosignal-based AI-assisted tools, and a minority of RCTs studied AI-assisted tools drawn from clinical data. In 77% (30/39) of the RCTs, AI-assisted interventions outperformed usual clinical care, and clinically relevant outcomes improved with AI-assisted intervention in 70% (21/30) of the studies. Small sample size and single-center design limited the generalizability of these studies.

Conclusions: There is growing evidence supporting the implementation of AI-assisted tools in daily clinical practice; however, the number of available RCTs is limited and heterogeneous. More RCTs of AI-assisted tools integrated into clinical practice are needed to advance the role of AI in medicine.

Trial Registration: PROSPERO CRD42021286539; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=286539.

Citing Articles

Innovations in Digital Health From a Global Perspective: Proceedings of PRC-HI 2024.

Feng X, Sun Y, Wu Y, Wang H, Wu Y Health Care Sci. 2025; 4(1):66-69.

PMID: 40026636 PMC: 11869371. DOI: 10.1002/hcs2.128.

Artificial intelligence for individualized treatment of persistent atrial fibrillation: a randomized controlled trial.

Deisenhofer I, Albenque J, Busch S, Gitenay E, Mountantonakis S, Roux A Nat Med. 2025; .

PMID: 39953289 DOI: 10.1038/s41591-025-03517-w.

Artificial intelligence in digital pathology - time for a reality check.

Aggarwal A, Bharadwaj S, Corredor G, Pathak T, Badve S, Madabhushi A Nat Rev Clin Oncol. 2025; .

PMID: 39934323 DOI: 10.1038/s41571-025-00991-6.

Pathology Foundation Models.

Ochi M, Komura D, Ishikawa S JMA J. 2025; 8(1):121-130.

PMID: 39926091 PMC: 11799676. DOI: 10.31662/jmaj.2024-0206.

Satisfactory Evaluation of Call Service Using AI After Ureteral Stent Insertion: Randomized Controlled Trial.

Cho U, Gwon Y, Chong S, Han J, Kim D, Doo S J Med Internet Res. 2025; 27:e56039.

PMID: 39836955 PMC: 11795156. DOI: 10.2196/56039.

References

Yeo M, Tahayori B, Kok H, Maingard J, Kutaiba N, Russell J . Review of deep learning algorithms for the automatic detection of intracranial hemorrhages on computed tomography head imaging. J Neurointerv Surg. 2021; 13(4):369-378. DOI: 10.1136/neurintsurg-2020-017099. View

Brown J, Campbell J, Beers A, Chang K, Ostmo S, Paul Chan R . Automated Diagnosis of Plus Disease in Retinopathy of Prematurity Using Deep Convolutional Neural Networks. JAMA Ophthalmol. 2018; 136(7):803-810. PMC: 6136045. DOI: 10.1001/jamaophthalmol.2018.1934. View

Blomberg S, Christensen H, Lippert F, Ersboll A, Torp-Petersen C, Sayre M . Effect of Machine Learning on Dispatcher Recognition of Out-of-Hospital Cardiac Arrest During Calls to Emergency Medical Services: A Randomized Clinical Trial. JAMA Netw Open. 2021; 4(1):e2032320. PMC: 7788469. DOI: 10.1001/jamanetworkopen.2020.32320. View

Stromblad C, Baxter-King R, Meisami A, Yee S, Levine M, Ostrovsky A . Effect of a Predictive Model on Planned Surgical Duration Accuracy, Patient Wait Time, and Use of Presurgical Resources: A Randomized Clinical Trial. JAMA Surg. 2021; 156(4):315-321. PMC: 7841577. DOI: 10.1001/jamasurg.2020.6361. View

Wang P, Liu X, Berzin T, Glissen Brown J, Liu P, Zhou C . Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study. Lancet Gastroenterol Hepatol. 2020; 5(4):343-351. DOI: 10.1016/S2468-1253(19)30411-X. View

Lin H, Li R, Liu Z, Chen J, Yang Y, Chen H . Diagnostic Efficacy and Therapeutic Decision-making Capacity of an Artificial Intelligence Platform for Childhood Cataracts in Eye Clinics: A Multicentre Randomized Controlled Trial. EClinicalMedicine. 2019; 9:52-59. PMC: 6510889. DOI: 10.1016/j.eclinm.2019.03.001. View

Zhang L, Long J, Jiang W, Shi Y, He X, Zhou Z . Trends in Chronic Kidney Disease in China. N Engl J Med. 2016; 375(9):905-6. DOI: 10.1056/NEJMc1602469. View

Browning M, Bilderbeck A, Dias R, Dourish C, Kingslake J, Deckert J . The clinical effectiveness of using a predictive algorithm to guide antidepressant treatment in primary care (PReDicT): an open-label, randomised controlled trial. Neuropsychopharmacology. 2021; 46(7):1307-1314. PMC: 8134561. DOI: 10.1038/s41386-021-00981-z. View

Yao X, Rushlow D, Inselman J, McCoy R, Thacher T, Behnken E . Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial. Nat Med. 2021; 27(5):815-819. DOI: 10.1038/s41591-021-01335-4. View

10.

Persell S, Peprah Y, Lipiszko D, Lee J, Li J, Ciolino J . Effect of Home Blood Pressure Monitoring via a Smartphone Hypertension Coaching Application or Tracking Application on Adults With Uncontrolled Hypertension: A Randomized Clinical Trial. JAMA Netw Open. 2020; 3(3):e200255. PMC: 7052730. DOI: 10.1001/jamanetworkopen.2020.0255. View

11.

Repici A, Spadaccini M, Antonelli G, Correale L, Maselli R, Galtieri P . Artificial intelligence and colonoscopy experience: lessons from two randomised trials. Gut. 2021; 71(4):757-765. DOI: 10.1136/gutjnl-2021-324471. View

12.

Liu P, Wang P, Glissen Brown J, Berzin T, Zhou G, Liu W . The single-monitor trial: an embedded CADe system increased adenoma detection during colonoscopy: a prospective randomized study. Therap Adv Gastroenterol. 2021; 13:1756284820979165. PMC: 7745558. DOI: 10.1177/1756284820979165. View

13.

Luo Y, Zhang Y, Liu M, Lai Y, Liu P, Wang Z . Artificial Intelligence-Assisted Colonoscopy for Detection of Colon Polyps: a Prospective, Randomized Cohort Study. J Gastrointest Surg. 2020; 25(8):2011-2018. PMC: 8321985. DOI: 10.1007/s11605-020-04802-4. View

14.

Benjamens S, Dhunnoo P, Mesko B . The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. NPJ Digit Med. 2020; 3:118. PMC: 7486909. DOI: 10.1038/s41746-020-00324-0. View

15.

Wang P, Liu P, Glissen Brown J, Berzin T, Zhou G, Lei S . Lower Adenoma Miss Rate of Computer-Aided Detection-Assisted Colonoscopy vs Routine White-Light Colonoscopy in a Prospective Tandem Study. Gastroenterology. 2020; 159(4):1252-1261.e5. DOI: 10.1053/j.gastro.2020.06.023. View

16.

Shung D, Sung J . Challenges of developing artificial intelligence-assisted tools for clinical medicine. J Gastroenterol Hepatol. 2021; 36(2):295-298. PMC: 11874508. DOI: 10.1111/jgh.15378. View

17.

El Solh A, Akinnusi M, Patel A, Bhat A, TenBrock R . Predicting optimal CPAP by neural network reduces titration failure: a randomized study. Sleep Breath. 2009; 13(4):325-30. DOI: 10.1007/s11325-009-0247-5. View

18.

Hassan C, Spadaccini M, Iannone A, Maselli R, Jovani M, Chandrasekar V . Performance of artificial intelligence in colonoscopy for adenoma and polyp detection: a systematic review and meta-analysis. Gastrointest Endosc. 2020; 93(1):77-85.e6. DOI: 10.1016/j.gie.2020.06.059. View

19.

Shung D, Au B, Taylor R, Tay J, Laursen S, Stanley A . Validation of a Machine Learning Model That Outperforms Clinical Risk Scoring Systems for Upper Gastrointestinal Bleeding. Gastroenterology. 2019; 158(1):160-167. PMC: 7004228. DOI: 10.1053/j.gastro.2019.09.009. View

20.

Samarasena J . Guns, germs, and steel…and artificial intelligence. Gastrointest Endosc. 2020; 93(1):99-101. DOI: 10.1016/j.gie.2020.07.021. View