When Intraclass Correlation Coefficients Go Awry: a Case Study from a School-based Smoking Prevention Study in South Africa

Objectives: We conducted a group randomized trial of 2 South African school-based smoking prevention programs and examined possible sources and implications of why our actual intraclass correlation coefficients (ICCs) were significantly higher than the ICC of 0.02 used to compute initial sample size requirements.

Methods: Thirty-six South African high schools were randomly assigned to 1 of 3 experimental groups. On 3 occasions, students completed questionnaires on tobacco and drug use attitudes and behaviors. We used mixed-effects models to partition individual and school-level variance components, with and without covariate adjustment.

Results: For 30-day smoking, unadjusted ICCs ranged from 0.12 to 0.17 across the 3 time points. For lifetime smoking, ICCs ranged from 0.18 to 0.22; for other drug use variables, 0.02 to 0.10; and for psychosocial variables, 0.09 to 0.23. Covariate adjustment substantially reduced most ICCs.

Conclusions: The unadjusted ICCs we observed for smoking behaviors were considerably higher than those previously reported. This effectively reduced our sample size by a factor of 17. Future studies that anticipate significant cluster-level racial homogeneity may consider using higher-value ICCs in sample-size calculations to ensure adequate statistical power.