The Feasibility of Blinding Residency Programs to USMLE Step 1 Scores During GME Application, Interview, and Match Processes

Background: With the recent announcement that Step 1 score reporting will soon change to pass/fail, residency programs will need to reconsider their recruitment processes.

Objective: We (1) evaluated the feasibility of blinding residency programs to applicants' Step 1 scores and their number of attempts throughout the recruitment process; (2) described the selection process that resulted from the blinding; and (3) reviewed if a program's initial rank list, created before scores were known, would be changed before submission for the Match.

Methods: During the 2018-2019 and 2019-2020 recruitment seasons, all programs at a single sponsoring institution were invited to develop selection criteria in the absence of Step 1 data, and to remain blinded to this data throughout recruitment. Participating programs were surveyed to determine factors affecting feasibility and metrics used for residency selection. Once unblinded to Step 1 scores, programs had the option to change their initial rank lists.

Results: Of 24 residency programs, 4 participated (17%) in the first year: emergency medicine, neurology, pediatrics, and psychiatry. The second year had the same participants, with the addition of family and community medicine and radiation oncology (n = 6, 25%). Each program was able to determine mission-specific qualities in the absence of Step 1 data. In both years, one program made changes to the final rank list.

Conclusions: It was feasible for programs to establish metrics for residency recruitment in the absence of Step 1 data, and most programs made no changes to final rank lists after Step 1 scores were known.