OSCE Checklists Do Not Capture Increasing Levels of Expertise

Purpose: To evaluate the effectiveness of binary content checklists in measuring increasing levels of clinical competence.

Method: Fourteen clinical clerks, 14 family practice residents, and 14 family physicians participated in two 15-minute standardized patient interviews. An examiner rated each participant's performance using a binary content checklist and a global process rating. The participants provided a diagnosis two minutes into and at the end of the interview.

Results: On global scales, the experienced clinicians scored significantly better than did the residents and clerks, but on checklists, the experienced clinicians scored significantly worse than did the residents and clerks. Diagnostic accuracy increased for all groups between the two-minute and 15-minute marks without significant differences between the groups.

Conclusion: These findings are consistent with the hypothesis that binary checklists may not be valid measures of increasing clinical competence.