Practical Applications of Generalizability Theory for Designing, Evaluating, and Improving Psychological Assessments

In this article, we illustrate how generalizability theory (G-theory) can extend traditional assessment methods for designing, improving, and evaluating results from both objectively and subjectively scored measures of individual differences. Our illustrations include quantification of multiple sources of measurement error, derivation of unique indexes of consistency for norm- and criterion-referenced interpretations of scores, estimation of score consistency when changing a measurement procedure, and disattenuation of correlation coefficients for measurement error. We also expand G-theory analyses beyond the item level to include parcels and split measures and highlight linkages among G-theory, classical test theory, and structural equation modeling. Computer code and sample data are provided in online supplements to help readers apply the demonstrated techniques to their own assessments.