Reliability of Observers' Subjective Impressions of Families: a Generalizability Theory Approach

Overview

Journal Psychother Res

Publisher Routledge

Specialties Pharmacology
Psychiatry
Psychology

Date 2012 Oct 17

PMID 23066691

Citations 6

Authors

Bent Stora

Knut A Hagtvet

Sonja Heyerdahl

Affiliations

Soon will be listed here.

Abstract

Parenting was observed in videotaped interactions in 30 families referred for child conduct problems. Generalizability coefficients and the impact of varying numbers of raters were estimated. Two measurement designs were compared: All raters observed all families ("crossed" design) and a different rater observed each family ("nested" design). The crossed design provided higher generalizability coefficients than a nested design, implying inflated generalizability estimates if a crossed estimation model is used for a nested data collection. Three and four raters were needed to obtain generalizability coefficients in the .70-.80 range for monitoring and discipline, respectively. One rater was sufficient for a corresponding estimate for positive involvement and for an estimate in .80-.90 range for problem-solving. Estimates for skill encouragement were non-acceptable.

Citing Articles

Development and preliminary validation of the chronic obstructive pulmonary disease scale quality of life instruments for chronic diseases-chronic obstructive pulmonary disease based on classical test theory and generalizability theory.

Wan C, Yang Z, Zhao Z, Quan P, Wu B, Yang Y Chron Respir Dis. 2022; 19:14799731221104099.

PMID: 36000309 PMC: 9421010. DOI: 10.1177/14799731221104099.

Development and validation of the peptic ulcer scale under the system of quality of life instruments for chronic diseases based on classical test theory and generalizability theory.

Wan C, Chen Y, Gao L, Zhang Q, Quan P, Sun X BMC Gastroenterol. 2020; 20(1):422.

PMID: 33317456 PMC: 7734778. DOI: 10.1186/s12876-020-01562-y.

Development and validation of the nasopharyngeal cancer scale among the system of quality of life instruments for cancer patients (QLICP-NA V2.0): combined classical test theory and generalizability theory.

Wu J, Hu L, Zhang G, Liang Q, Meng Q, Wan C Qual Life Res. 2016; 25(8):2087-100.

PMID: 26928910 DOI: 10.1007/s11136-016-1251-4.

Development and validation of the irritable bowel syndrome scale under the system of quality of life instruments for chronic diseases QLICD-IBS: combinations of classical test theory and generalizability theory.

Lei P, Lei G, Tian J, Zhou Z, Zhao M, Wan C Int J Colorectal Dis. 2014; 29(10):1245-55.

PMID: 25080147 DOI: 10.1007/s00384-014-1976-x.

Development and validation of the coronary heart disease scale under the system of quality of life instruments for chronic diseases QLICD-CHD: combinations of classical test theory and Generalizability Theory.

Wan C, Li H, Fan X, Yang R, Pan J, Chen W Health Qual Life Outcomes. 2014; 12:82.

PMID: 24894812 PMC: 4229878. DOI: 10.1186/1477-7525-12-82.

References

Prinz R, Foster S, Kent R, OLeary K . Multivariate assessment of conflict in distressed and nondistressed mother-adolescent dyads. J Appl Behav Anal. 1979; 12(4):691-700. PMC: 1311489. DOI: 10.1901/jaba.1979.12-691. View

Gardner F . Methodological issues in the direct observation of parent-child interaction: do observational findings reflect the natural behavior of participants?. Clin Child Fam Psychol Rev. 2001; 3(3):185-98. DOI: 10.1023/a:1009503409699. View

Ogden T, Hagen K . Treatment effectiveness of Parent Management Training in Norway: a randomized controlled trial of children with conduct problems. J Consult Clin Psychol. 2008; 76(4):607-621. DOI: 10.1037/0022-006X.76.4.607. View

DeGarmo D, Patterson G, Forgatch M . How do outcomes in a specified parent training intervention maintain or wane over time?. Prev Sci. 2004; 5(2):73-89. DOI: 10.1023/b:prev.0000023078.30191.e0. View

Snyder J, Reid J, Stoolmiller M, Howe G, Brown H, Dagne G . The role of behavior observation in measurement systems for randomized prevention trials. Prev Sci. 2006; 7(1):43-56. DOI: 10.1007/s11121-005-0020-3. View

Lakes K, Hoyt W . Applications of generalizability theory to clinical child and adolescent psychology research. J Clin Child Adolesc Psychol. 2009; 38(1):144-65. PMC: 3650138. DOI: 10.1080/15374410802575461. View

Martinez Jr C, Forgatch M . Preventing problems with boys' noncompliance: effects of a parent training intervention for divorcing mothers. J Consult Clin Psychol. 2001; 69(3):416-28. DOI: 10.1037//0022-006x.69.3.416. View

Forgatch M, DeGarmo D, Beldavs Z . An efficacious theory-based intervention for stepfamilies. Behav Ther. 2006; 36(4):357-65. PMC: 1464401. DOI: 10.1016/s0005-7894(05)80117-0. View

Wasserman R, Levy K, Loken E . Generalizability theory in psychotherapy research: the impact of multiple sources of variance on the dependability of psychotherapy process ratings. Psychother Res. 2009; 19(4-5):397-408. DOI: 10.1080/10503300802579156. View

10.

Martinez Jr C, Forgatch M . Adjusting to change: linking family structure transitions with parenting and boys' adjustment. J Fam Psychol. 2002; 16(2):107-17. DOI: 10.1037//0893-3200.16.2.107. View

11.

Aspland H, Gardner F . Observational Measures of Parent-Child Interaction: An Introductory Review. Child Adolesc Ment Health. 2020; 8(3):136-143. DOI: 10.1111/1475-3588.00061. View

12.

Mori L, Armendariz G . Analogue assessment of child behavior problems. Psychol Assess. 2001; 13(1):36-45. DOI: 10.1037//1040-3590.13.1.36. View

13.

Haynes S . Clinical applications of analogue behavioral observation: dimensions of psychometric evaluation. Psychol Assess. 2001; 13(1):73-85. DOI: 10.1037//1040-3590.13.1.73. View

14.

Roberts M . Clinic observations of structured parent-child interaction designed to evaluate externalizing disorders. Psychol Assess. 2001; 13(1):46-58. View

15.

Crits-Christoph P, Johnson J, Gallop R, Connolly Gibbons M, Ring-Kurtz S, Hamilton J . A generalizability theory analysis of group process ratings in the treatment of cocaine dependence. Psychother Res. 2011; 21(3):252-66. PMC: 3361025. DOI: 10.1080/10503307.2010.551429. View