Who You Know or What You Know? Effect of Examiner Familiarity with Residents on OSCE Scores

Background: Despite the goal of objective structured clinical examinations (OSCEs) to be objective, examiner biases may influence scores. Examiner familiarity with candidates is a potential bias that has not been well studied.

Method: To determine the effect of familiarity, OSCE scores for 158 internal medicine residents were analyzed by whether examiners were familiar with them, based on previous clinical encounters, and if previous impressions were positive or negative. A hierarchical multivariable analysis of variance was performed to control for resident, examiner, and level of training.

Results: Across 480 interactions (50 examiners, 158 residents), multivariable analysis showed that positive familiarity was associated with a significant increase in ratings (+0.37 on a 5-point scale), comparable to the difference between first- and third/fourth-year residents.

Conclusions: Familiarity with candidates is a significant source of examiner bias in OSCE scores. Consideration should be paid to the influence of examiners' previous knowledge of examinees and attempts made to mitigate this bias.