What Too Strict a Method Obscures About the Validity of Outcome Measures

To assess the outcome of psychotherapeutic treatments, psychotherapy researchers often compare pre- and post-treatment scores on self-report outcome measures. In this paper, the common assumption is challenged that pre-to-post decreasing and increasing outcome scores are indicative of successful and failed therapies, respectively. The outcome of 29 psychotherapeutic treatments was evaluated by means of quantitative analysis of pre- and post-treatment scores on commonly used outcome measures (such as the Symptom Checklist-90-R, the Inventory of Interpersonal Problems, and the General Health Questionnaire-12), as well as through consensual qualitative research. Overall, a moderate to low convergence between qualitative and quantitative evaluations of outcome was observed. Detailed analyses of six cases are presented in which pre-to-post comparisons of outcome measures proved misleading. It is concluded that psychotherapy outcome research might benefit from assessment strategies that are sensitive to the singularities of individual treatments and to the complexity of the phenomenon of therapeutic outcome. Furthermore, classical psychometric evaluations of the validity of outcome measures might be supplemented with less-systematic evaluations that take any contingent source of information on outcome into account.