Artificial Intelligence in Medical Education Curriculum: An E-Delphi Study for Competencies

Overview

Journal PLoS One

Specialties General Medicine
Science

Date 2022 Jul 21

PMID 35862401

Authors

S Ayhan Caliskan

Kadir Demir

Ozan Karaca

Affiliations

Soon will be listed here.

Abstract

Background: Artificial intelligence (AI) has affected our day-to-day in a great extent. Healthcare industry is one of the mainstream fields among those and produced a noticeable change in treatment and education. Medical students must comprehend well why AI technologies mediate and frame their decisions on medical issues. Formalizing of instruction on AI concepts can facilitate learners to grasp AI outcomes in association with their sensory perceptions and thinking in the dynamic and ambiguous reality of daily medical practice. The purpose of this study is to provide consensus on the competencies required by medical graduates to be ready for artificial intelligence technologies and possible applications in medicine and reporting the results.

Materials And Methods: A three-round e-Delphi survey was conducted between February 2020 and November 2020. The Delphi panel accorporated experts from different backgrounds; (i) healthcare professionals/ academicians; (ii) computer and data science professionals/ academics; (iii) law and ethics professionals/ academics; and (iv) medical students. Round 1 in the Delphi survey began with exploratory open-ended questions. Responses received in the first round evaluated and refined to a 27-item questionnaire which then sent to the experts to be rated using a 7-point Likert type scale (1: Strongly Disagree-7: Strongly Agree). Similar to the second round, the participants repeated their assessments in the third round by using the second-round analysis. The agreement level and strength of the consensus was decided based on third phase results. Median scores was used to calculate the agreement level and the interquartile range (IQR) was used for determining the strength of the consensus.

Results: Among 128 invitees, a total of 94 agreed to become members of the expert panel. Of them 75 (79.8%) completed the Round 1 questionnaire, 69/75 (92.0%) completed the Round 2 and 60/69 (87.0%) responded to the Round 3. There was a strong agreement on the 23 items and weak agreement on the 4 items.

Conclusions: This study has provided a consensus list of the competencies required by the medical graduates to be ready for AI implications that would bring new perspectives to medical education curricula. The unique feature of the current research is providing a guiding role in integrating AI into curriculum processes, syllabus content and training of medical students.

Citing Articles

Factors affecting medical artificial intelligence (AI) readiness among medical students: taking stock and looking forward.

Ziapour A, Darabi F, Janjani P, Amani M, Yildirim M, Motevaseli S BMC Med Educ. 2025; 25(1):264.

PMID: 39966878 PMC: 11837483. DOI: 10.1186/s12909-025-06852-1.

AI in the Health Sector: Systematic Review of Key Skills for Future Health Professionals.

Gazquez-Garcia J, Sanchez-Bocanegra C, Sevillano J JMIR Med Educ. 2025; 11:e58161.

PMID: 39912237 PMC: 11822726. DOI: 10.2196/58161.

Beginner-Level Tips for Medical Educators: Guidance on Selection, Prompt Engineering, and the Use of Artificial Intelligence Chatbots.

Kiyak Y Med Sci Educ. 2025; 34(6):1571-1576.

PMID: 39758489 PMC: 11699172. DOI: 10.1007/s40670-024-02146-1.

Shaping the future: perspectives on the Integration of Artificial Intelligence in health profession education: a multi-country survey.

Bani Issa W, Shorbagi A, Al-Sharman A, Rababa M, Al-Majeed K, Radwan H BMC Med Educ. 2024; 24(1):1166.

PMID: 39425151 PMC: 11488068. DOI: 10.1186/s12909-024-06076-9.

Clinicians' roles and necessary levels of understanding in the use of artificial intelligence: A qualitative interview study with German medical students.

Funer F, Tinnemeyer S, Liedtke W, Salloch S BMC Med Ethics. 2024; 25(1):107.

PMID: 39375660 PMC: 11457475. DOI: 10.1186/s12910-024-01109-w.

References

Keeney S, Hasson F, McKenna H . A critical review of the Delphi technique as a research methodology for nursing. Int J Nurs Stud. 2001; 38(2):195-200. DOI: 10.1016/s0020-7489(00)00044-4. View

Hasselager A, Lauritsen T, Kristensen T, Bohnstedt C, Sonderskov C, Ostergaard D . What should be included in the assessment of laypersons' paediatric basic life support skills? Results from a Delphi consensus study. Scand J Trauma Resusc Emerg Med. 2018; 26(1):9. PMC: 5774155. DOI: 10.1186/s13049-018-0474-5. View

Courtenay M, Deslandes R, Harries-Huntley G, Hodson K, Morris G . Classic e-Delphi survey to provide national consensus and establish priorities with regards to the factors that promote the implementation and continued development of non-medical prescribing within health services in Wales. BMJ Open. 2018; 8(9):e024161. PMC: 6150146. DOI: 10.1136/bmjopen-2018-024161. View

De Villiers M, de Villiers P, Kent A . The Delphi technique in health sciences education research. Med Teach. 2005; 27(7):639-43. DOI: 10.1080/13611260500069947. View

Prideaux D . ABC of learning and teaching in medicine. Curriculum design. BMJ. 2003; 326(7383):268-70. PMC: 1125124. DOI: 10.1136/bmj.326.7383.268. View

Birko S, Dove E, Ozdemir V . Evaluation of Nine Consensus Indices in Delphi Foresight Research and Their Dependency on Delphi Survey Characteristics: A Simulation Study and Debate on Delphi Design and Interpretation. PLoS One. 2015; 10(8):e0135162. PMC: 4535950. DOI: 10.1371/journal.pone.0135162. View

Gensichen J, Vollmar H, Sonnichsen A, Waldmann U, Sandars J . E-learning for education in primary healthcare--turning the hype into reality: a Delphi study. Eur J Gen Pract. 2009; 15(1):11-4. DOI: 10.1080/13814780902864160. View

Hasson F, Keeney S, McKenna H . Research guidelines for the Delphi survey technique. J Adv Nurs. 2000; 32(4):1008-15. View

Topol E . High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019; 25(1):44-56. DOI: 10.1038/s41591-018-0300-7. View

10.

Goodman C . The Delphi technique: a critique. J Adv Nurs. 1987; 12(6):729-34. DOI: 10.1111/j.1365-2648.1987.tb01376.x. View

11.

Sapci A, Sapci H . Artificial Intelligence Education and Tools for Medical and Health Informatics Students: Systematic Review. JMIR Med Educ. 2020; 6(1):e19285. PMC: 7367541. DOI: 10.2196/19285. View

12.

Waggoner J, Carline J, Durning S . Is There a Consensus on Consensus Methodology? Descriptions and Recommendations for Future Consensus Research. Acad Med. 2016; 91(5):663-8. DOI: 10.1097/ACM.0000000000001092. View

13.

Pinto Dos Santos D, Giese D, Brodehl S, Chon S, Staab W, Kleinert R . Medical students' attitude towards artificial intelligence: a multicentre survey. Eur Radiol. 2018; 29(4):1640-1646. DOI: 10.1007/s00330-018-5601-1. View

14.

van der Niet A, Bleakley A . Where medical education meets artificial intelligence: 'Does technology care?'. Med Educ. 2020; 55(1):30-36. DOI: 10.1111/medu.14131. View

15.

Yu K, Beam A, Kohane I . Artificial intelligence in healthcare. Nat Biomed Eng. 2019; 2(10):719-731. DOI: 10.1038/s41551-018-0305-z. View

16.

Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C . Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011; 6(6):e20476. PMC: 3111406. DOI: 10.1371/journal.pone.0020476. View

17.

Santaguida P, Dolovich L, Oliver D, Lamarche L, Gilsing A, Griffith L . Protocol for a Delphi consensus exercise to identify a core set of criteria for selecting health related outcome measures (HROM) to be used in primary health care. BMC Fam Pract. 2018; 19(1):152. PMC: 6123958. DOI: 10.1186/s12875-018-0831-5. View

18.

Taylor R, Feltbower R, Aslam N, Raine R, Whelan J, Gibson F . Modified international e-Delphi survey to define healthcare professional competencies for working with teenagers and young adults with cancer. BMJ Open. 2016; 6(5):e011361. PMC: 4861123. DOI: 10.1136/bmjopen-2016-011361. View

19.

Wartman S, Combs C . Medical Education Must Move From the Information Age to the Age of Artificial Intelligence. Acad Med. 2017; 93(8):1107-1109. DOI: 10.1097/ACM.0000000000002044. View

20.

Sit C, Srinivasan R, Amlani A, Muthuswamy K, Azam A, Monzon L . Attitudes and perceptions of UK medical students towards artificial intelligence and radiology: a multicentre survey. Insights Imaging. 2020; 11(1):14. PMC: 7002761. DOI: 10.1186/s13244-019-0830-7. View