Multiple Testing of Hypotheses in Comparing Two Groups

Researchers frequently encounter studies that compare two groups on many variables. We discourage the use of multiple tests of hypotheses on individual variables, an approach that ignores the correlation among the variables and increases the chance of a type I error. Instead of examining each variable separately, we recommend using multivariate procedures that integrate all measures on a person into a unified analysis of the differences between the two groups. We describe three multivariate procedures: Hotelling's T2, discriminant analysis, and logistic regression. We also discuss the use of Bonferroni's adjustment to preserve the overall chance of a type I error in conducting individual tests on each variable after doing the multivariate procedures. We review the underlying assumptions and relative merits and disadvantages of the three multivariate methods and recommend which method to use in various circumstances.