The Sights and Insights of Examiners in Objective Structured Clinical Examinations

Overview

Journal J Educ Eval Health Prof

Specialty Medical Education

Date 2017 Dec 28

PMID 29278906

Citations 21

Authors

Lauren Chong

Silas Taylor

Matthew Haywood

Barbara-Ann Adelstein

Boaz Shulruf

Affiliations

Soon will be listed here.

Abstract

Purpose: The objective structured clinical examination (OSCE) is considered to be one of the most robust methods of clinical assessment. One of its strengths lies in its ability to minimise the effects of examiner bias due to the standardisation of items and tasks for each candidate. However, OSCE examiners' assessment scores are influenced by several factors that may jeopardise the assumed objectivity of OSCEs. To better understand this phenomenon, the current review aims to determine and describe important sources of examiner bias and the factors affecting examiners' assessments.

Methods: We performed a narrative review of the medical literature using Medline. All articles meeting the selection criteria were reviewed, with salient points extracted and synthesised into a clear and comprehensive summary of the knowledge in this area.

Results: OSCE examiners' assessment scores are influenced by factors belonging to 4 different domains: examination context, examinee characteristics, examinee-examiner interactions, and examiner characteristics. These domains are composed of several factors including halo, hawk/dove and OSCE contrast effects; the examiner's gender and ethnicity; training; lifetime experience in assessing; leadership and familiarity with students; station type; and site effects.

Conclusion: Several factors may influence the presumed objectivity of examiners' assessments, and these factors need to be addressed to ensure the objectivity of OSCEs. We offer insights into directions for future research to better understand and address the phenomenon of examiner bias.

Citing Articles

Optimizing cost-effectiveness in remote objective structured clinical examinations through targeted double scoring methodologies.

Fu Z, Wu Y, Xu L, Cai F, Liu R, Jiang Z Med Educ Online. 2025; 30(1):2467477.

PMID: 39963052 PMC: 11837930. DOI: 10.1080/10872981.2025.2467477.

Knowledge Mapping and Global Trends in the Field of the Objective Structured Clinical Examination: Bibliometric and Visual Analysis (2004-2023).

Ba H, Zhang L, He X, Li S JMIR Med Educ. 2024; 10:e57772.

PMID: 39348890 PMC: 11474118. DOI: 10.2196/57772.

Impact of simulated patient-based communication training vs. real patient-based communication training on empathetic behaviour in undergraduate students - a prospective evaluation study.

Britz V, Sterz J, Koch Y, Schreckenbach T, Stefanescu M, Zinsser U BMC Med Educ. 2024; 24(1):870.

PMID: 39134984 PMC: 11318334. DOI: 10.1186/s12909-024-05801-8.

Raters and examinees training for objective structured clinical examination: comparing the effectiveness of three instructional methodologies.

Guerrero J, Alqarni A, Estadilla L, Benjamin L, Rani V BMC Nurs. 2024; 23(1):500.

PMID: 39039471 PMC: 11265468. DOI: 10.1186/s12912-024-02183-6.

Interactions between the sex of the clinician grader and the sex of the chiropractic student intern on spinal manipulation assessment grade.

Sheppard M, Johnson S, Quiroz V, Ward J J Chiropr Educ. 2023; 37(2):157-161.

PMID: 37655808 PMC: 11095654. DOI: 10.7899/JCE-22-12.

References

Swygert K, Cuddy M, van Zanten M, Haist S, Jobe A . Gender differences in examinee performance on the Step 2 Clinical Skills data gathering (DG) and patient note (PN) components. Adv Health Sci Educ Theory Pract. 2011; 17(4):557-71. DOI: 10.1007/s10459-011-9333-0. View

Taylor S, Shulruf B . Australian medical students have fewer opportunities to do physical examination of peers of the opposite gender. J Educ Eval Health Prof. 2016; 13:42. PMC: 5286215. DOI: 10.3352/jeehp.2016.13.42. View

Bartman I, Smee S, Roy M . A method for identifying extreme OSCE examiners. Clin Teach. 2013; 10(1):27-31. DOI: 10.1111/j.1743-498X.2012.00607.x. View

McManus I, Elder A, Dacre J . Investigating possible ethnicity and sex bias in clinical examiners: an analysis of data from the MRCP(UK) PACES and nPACES examinations. BMC Med Educ. 2013; 13:103. PMC: 3737060. DOI: 10.1186/1472-6920-13-103. View

Stroud L, Herold J, Tomlinson G, Cavalcanti R . Who you know or what you know? Effect of examiner familiarity with residents on OSCE scores. Acad Med. 2011; 86(10 Suppl):S8-11. DOI: 10.1097/ACM.0b013e31822a729d. View

Denney M, Freeman A, Wakeford R . MRCGP CSA: are the examiners biased, favouring their own by sex, ethnicity, and degree source?. Br J Gen Pract. 2013; 63(616):e718-25. PMC: 3809424. DOI: 10.3399/bjgp13X674396. View

Wood T . Exploring the role of first impressions in rater-based assessments. Adv Health Sci Educ Theory Pract. 2013; 19(3):409-27. DOI: 10.1007/s10459-013-9453-9. View

Chesser A, Cameron H, Evans P, Cleland J, Boursicot K, Mires G . Sources of variation in performance on a shared OSCE station across four UK medical schools. Med Educ. 2009; 43(6):526-32. DOI: 10.1111/j.1365-2923.2009.03370.x. View

Sakurai H, Kanada Y, Sugiura Y, Motoya I, Wada Y, Yamada M . OSCE-based Clinical Skill Education for Physical and Occupational Therapists. J Phys Ther Sci. 2014; 26(9):1387-97. PMC: 4175242. DOI: 10.1589/jpts.26.1387. View

10.

Harasym P, Woloschuk W, Cunning L . Undesired variance due to examiner stringency/leniency effect in communication skill scores assessed in OSCEs. Adv Health Sci Educ Theory Pract. 2007; 13(5):617-32. DOI: 10.1007/s10459-007-9068-0. View

11.

Wong M, Fones C, Aw M, Tan C, Low P, Amin Z . Should non-expert clinician examiners be used in objective structured assessment of communication skills among final year medical undergraduates?. Med Teach. 2007; 29(9):927-32. DOI: 10.1080/01421590701601535. View

12.

Iramaneerat C, Yudkowsky R . Rater errors in a clinical skills assessment of medical students. Eval Health Prof. 2007; 30(3):266-83. DOI: 10.1177/0163278707304040. View

13.

Cuddy M, Swygert K, Swanson D, Jobe A . A multilevel analysis of examinee gender, standardized patient gender, and United States medical licensing examination step 2 clinical skills communication and interpersonal skills scores. Acad Med. 2011; 86(10 Suppl):S17-20. DOI: 10.1097/ACM.0b013e31822a6c05. View

14.

Hope D, Cameron H . Examiners are most lenient at the start of a two-day OSCE. Med Teach. 2014; 37(1):81-5. DOI: 10.3109/0142159X.2014.947934. View

15.

Sibbald M, Panisko D, Cavalcanti R . Role of clinical context in residents' physical examination diagnostic accuracy. Med Educ. 2011; 45(4):415-21. DOI: 10.1111/j.1365-2923.2010.03896.x. View

16.

Govaerts M, van der Vleuten C, Schuwirth L, Muijtjens A . Broadening perspectives on clinical performance assessment: rethinking the nature of in-training assessment. Adv Health Sci Educ Theory Pract. 2006; 12(2):239-60. DOI: 10.1007/s10459-006-9043-1. View

17.

Denney M, Wakeford R . Do role-players affect the outcome of a high-stakes postgraduate OSCE, in terms of candidate sex or ethnicity? Results from an analysis of the 52,702 anonymised case scores from one year of the MRCGP clinical skills assessment. Educ Prim Care. 2016; 27(1):39-43. DOI: 10.1080/14739879.2015.1113724. View

18.

Schwartzman E, Hsu D, Law A, Chung E . Assessment of patient communication skills during OSCE: examining effectiveness of a training program in minimizing inter-grader variability. Patient Educ Couns. 2011; 83(3):472-7. DOI: 10.1016/j.pec.2011.04.001. View

19.

Casey M, Wilkinson D, FitzGerald J, Eley D, Connor J . Clinical communication skills learning outcomes among first year medical students are consistent irrespective of participation in an interview for admission to medical school. Med Teach. 2014; 36(7):640-2. DOI: 10.3109/0142159X.2014.907880. View

20.

Brennan P, Croke D, Reed M, Smith L, Munro E, Foulkes J . Does Changing Examiner Stations During UK Postgraduate Surgery Objective Structured Clinical Examinations Influence Examination Reliability and Candidates' Scores?. J Surg Educ. 2016; 73(4):616-23. DOI: 10.1016/j.jsurg.2016.01.010. View