Validity Evidence for a New Checklist Evaluating Consultations, the 5Cs Model

Purpose: Consultation evaluation tools with strong validity evidence are scarce. The goal of this study was to demonstrate evidence of the validity of a new checklist, the 5Cs model (contact, communicate, core question, collaboration, and closing the loop), used to assess simulated emergent telephone consultations between residents and attending physician consultants.

Method: The authors developed items for the checklist based on a theoretical model, and content experts constructed two simulated emergent cases (one psychiatric and one surgical). In 2010, residents conducted consultations on the two cases by telephone with a standardized physician consultant, and three clinical faculty members rated the residents' consultations using the checklist. The authors analyzed the residents' checklist scores for response process validity, internal structure validity, and validity concerning the relationship to other variables.

Results: The authors established the checklist's content validity through its development based on a theoretical model, literature, and expert input. Response process validity evidence includes interrater reliability based on an intraclass correlation of 0.94. Internal structure validity evidence includes a generalizability analysis yielding a phi coefficient of 0.89 with three raters across two cases and a decision study yielding a phi coefficient of 0.8 with a single rater and three cases. Relationship to other variables includes correlations of 0.59 and 0.79 between mean scores on a global rating scale and on the checklist for the two simulated cases.

Conclusions: Multiple sources of validity evidence support the use of the 5Cs model as a tool to assess telephone consultations.