The Use of a Formative OSCE to Prepare Emergency Medicine Residents for Summative OSCEs: a Mixed-methods Cohort Study

Overview

Journal Int J Emerg Med

Publisher Biomed Central

Specialty Emergency Medicine

Date 2021 Oct 2

PMID 34598669

Citations 3

Authors

Magdalene Hui Min Lee

Dong Haur Phua

Kenneth Wei Jian Heng

Affiliations

Soon will be listed here.

Abstract

Background: The objective structured clinical examination (OSCE) is a part of emergency medicine (EM) examinations such as the Masters of Medicine in Emergency Medicine (MMed) examination and the equivalent Member of the Royal College of Emergency Medicine (MRCEM) examination. The use of formative OSCEs to prepare EM residents for summative OSCEs has not been investigated. This study aimed to evaluate the role of formative OSCEs in preparing EM residents for the MMed and MRCEM OSCE.

Methods: This was an observational, retrospective, mixed-methods cohort study. We analysed data from formative OSCEs conducted by the National Healthcare Group EM residency programme from 2013 to 2019, and from a questionnaire distributed to all residents during the study period. Residents' formative OSCE participation and scores were compared with first-attempt summative OSCE success. Qualitative analysis of residents' opinions on the usefulness of the formative OSCE was performed.

Results: Forty-three of the 50 (86.0%) residents attended at least one formative OSCE. Of the 46 who responded to the questionnaire, 40 (87.0%) had attempted and succeeded in the MMed or MRCEM OSCE, of whom 35 (87.5%) had succeeded on the first attempt. Residents who succeeded in the summative OSCE on the first attempt tended to have higher proximate (mean = 70.6, SD = 8.9 vs mean = 64.3, SD = 10.8) and mean (mean = 67.4, SD = 7.1 vs mean = 62.8, SD = 7.3) formative OSCE scores. All 8/40 (20.0%) residents who attended more than three formative OSCEs succeeded in the summative OSCE on their first attempt. Residents' formative OSCE scores tended to improve with successive formative OSCEs, demonstrating a positive training effect. All residents felt that the formative OSCE was useful in preparing them for the summative OSCE.

Conclusions: Participation in multiple formative OSCEs was beneficial in preparing residents for the summative OSCE. The formative OSCE was useful in familiarising residents with the examination, giving them an opportunity to perform in near-examination conditions, and providing feedback to residents and faculty about their progress. Our findings may support the implementation of formative OSCEs in other training programmes to prepare learners for high-stake summative OSCEs.

Citing Articles

Investigating the Effectiveness of Formative OSCE Combined with Visual Feedback in Improving Clinical Competence among Iranian Nurse Anesthesia Students: A Quasi-experimental study.

Khalafi A, Abbasi A, Sarvi Sarmeydani N, Albooghobeish M J Adv Med Educ Prof. 2024; 12(4):251-260.

PMID: 39463757 PMC: 11511839. DOI: 10.30476/jamp.2024.102318.1969.

Stress and value: the student perspective on utilizing real vs. actor patients in objective structured clinical examinations.

Vercio C, Tan G, Maxson I, Matta Y, Cacho B, Calaguas D BMC Med Educ. 2024; 24(1):760.

PMID: 39010096 PMC: 11247776. DOI: 10.1186/s12909-024-05673-y.

Objective structured clinical examination versus traditional written examinations: a prospective observational study.

Lebdai S, Bouvard B, Martin L, Annweiler C, Lerolle N, Rineau E BMC Med Educ. 2023; 23(1):69.

PMID: 36707797 PMC: 9883896. DOI: 10.1186/s12909-023-04050-5.

References

Larsen D, Butler A, Roediger 3rd H . Comparative effects of test-enhanced learning and self-explanation on long-term retention. Med Educ. 2013; 47(7):674-82. DOI: 10.1111/medu.12141. View

Turner J, Dankoski M . Objective structured clinical exams: a critical review. Fam Med. 2008; 40(8):574-8. View

Hodder R, Rivington R, Calcutt L, Hart I . The effectiveness of immediate feedback during the objective structured clinical examination. Med Educ. 1989; 23(2):184-8. DOI: 10.1111/j.1365-2923.1989.tb00884.x. View

Patricio M, Juliao M, Fareleira F, Vaz Carneiro A . Is the OSCE a feasible tool to assess competencies in undergraduate medical education?. Med Teach. 2013; 35(6):503-14. DOI: 10.3109/0142159X.2013.774330. View

TOWNSEND A, McLlvenny S, Miller C, Dunn E . The use of an objective structured clinical examination (OSCE) for formative and summative assessment in a general practice clinical attachment and its relationship to final medical school examination performance. Med Educ. 2001; 35(9):841-6. DOI: 10.1046/j.1365-2923.2001.00957.x. View

McGaghie W, Issenberg S, Cohen E, Barsuk J, Wayne D . Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011; 86(6):706-11. PMC: 3102783. DOI: 10.1097/ACM.0b013e318217e119. View

Lien H, Hsu S, Chen S, Yeh J . Can teaching hospitals use serial formative OSCEs to improve student performance?. BMC Res Notes. 2016; 9(1):464. PMC: 5064934. DOI: 10.1186/s13104-016-2266-1. View

Casey P, Goepfert A, Espey E, Hammoud M, Kaczmarczyk J, Katz N . To the point: reviews in medical education--the Objective Structured Clinical Examination. Am J Obstet Gynecol. 2009; 200(1):25-34. DOI: 10.1016/j.ajog.2008.09.878. View

Alkhateeb N, Al-Dabbagh A, Ibrahim M, Al-Tawil N . Effect of a Formative Objective Structured Clinical Examination on the Clinical Performance of Undergraduate Medical Students in a Summative Examination: A Randomized Controlled Trial. Indian Pediatr. 2019; 56(9):745-748. View

10.

Wallenstein J, Heron S, Santen S, Shayne P, Ander D . A core competency-based objective structured clinical examination (OSCE) can predict future resident performance. Acad Emerg Med. 2011; 17 Suppl 2:S67-71. DOI: 10.1111/j.1553-2712.2010.00894.x. View

11.

Dupras D, Li J . Use of an objective structured clinical examination to determine clinical competence. Acad Med. 1995; 70(11):1029-34. DOI: 10.1097/00001888-199511000-00022. View

12.

Barman A . Critiques on the Objective Structured Clinical Examination. Ann Acad Med Singap. 2005; 34(8):478-82. View

13.

Bevan J, Russell B, Marshall B . A new approach to OSCE preparation - PrOSCEs. BMC Med Educ. 2019; 19(1):126. PMC: 6498564. DOI: 10.1186/s12909-019-1571-5. View

14.

Hall A, Dagnone J, Lacroix L, Pickett W, Klinger D . Queen's simulation assessment tool: development and validation of an assessment tool for resuscitation objective structured clinical examination stations in emergency medicine. Simul Healthc. 2015; 10(2):98-105. DOI: 10.1097/SIH.0000000000000076. View

15.

Wood T, Humphrey-Murto S, Norman G . Standard setting in a small scale OSCE: a comparison of the Modified Borderline-Group Method and the Borderline Regression Method. Adv Health Sci Educ Theory Pract. 2006; 11(2):115-22. DOI: 10.1007/s10459-005-7853-1. View

16.

Chisnall B, Vince T, Hall S, Tribe R . Evaluation of outcomes of a formative objective structured clinical examination for second-year UK medical students. Int J Med Educ. 2015; 6:76-83. PMC: 4491429. DOI: 10.5116/ijme.5572.a534. View

17.

Wang E, Beaumont J, Kharasch M, Vozenilek J . Resident response to integration of simulation-based education into emergency medicine conference. Acad Emerg Med. 2008; 15(11):1207-10. DOI: 10.1111/j.1553-2712.2008.00208.x. View

18.

Madrazo L, Lee C, McConnell M, Khamisa K, Pugh D . No observed effect from a student-led mock objective structured clinical examination on subsequent performance scores in medical students in Canada. J Educ Eval Health Prof. 2019; 16:14. PMC: 6609294. DOI: 10.3352/jeehp.2019.16.14. View

19.

Hilliard R, Tallett S . The use of an objective structured clinical examination with postgraduate residents in pediatrics. Arch Pediatr Adolesc Med. 1998; 152(1):74-8. DOI: 10.1001/archpedi.152.1.74. View

20.

Okuda Y, Bond W, Bonfante G, McLaughlin S, Spillane L, Wang E . National growth in simulation training within emergency medicine residency programs, 2003-2008. Acad Emerg Med. 2008; 15(11):1113-6. DOI: 10.1111/j.1553-2712.2008.00195.x. View