Post Hoc Power Estimation in Large-scale Multiple Testing Problems

Overview

Journal Bioinformatics

Publisher Oxford University Press

Specialty Biology

Date 2010 Mar 2

PMID 20189938

Citations 4

Authors

Sonja Zehetmayer

Martin Posch

Affiliations

Soon will be listed here.

Abstract

Background: The statistical power or multiple Type II error rate in large-scale multiple testing problems as, for example, in gene expression microarray experiments, depends on typically unknown parameters and is therefore difficult to assess a priori. However, it has been suggested to estimate the multiple Type II error rate post hoc, based on the observed data.

Methods: We consider a class of post hoc estimators that are functions of the estimated proportion of true null hypotheses among all hypotheses. Numerous estimators for this proportion have been proposed and we investigate the statistical properties of the derived multiple Type II error rate estimators in an extensive simulation study.

Results: The performance of the estimators in terms of the mean squared error depends sensitively on the distributional scenario. Estimators based on empirical distributions of the null hypotheses are superior in the presence of strongly correlated test statistics.

Availability: R-code to compute all considered estimators based on P-values and supplementary material is available on the authors web page http://statistics.msi.meduniwien.ac.at/index.php?page=pageszfnr.

Citing Articles

Impact of adaptive filtering on power and false discovery rate in RNA-seq experiments.

Zehetmayer S, Posch M, Graf A BMC Bioinformatics. 2022; 23(1):388.

PMID: 36153479 PMC: 9509565. DOI: 10.1186/s12859-022-04928-z.

Sample size reassessment for a two-stage design controlling the false discovery rate.

Zehetmayer S, Graf A, Posch M Stat Appl Genet Mol Biol. 2015; 14(5):429-42.

PMID: 26461844 PMC: 4789494. DOI: 10.1515/sagmb-2014-0025.

False discovery rate control in two-stage designs.

Zehetmayer S, Posch M BMC Bioinformatics. 2012; 13:81.

PMID: 22559038 PMC: 3496575. DOI: 10.1186/1471-2105-13-81.

Cross-platform comparison of microarray data using order restricted inference.

Klinglmueller F, Tuechler T, Posch M Bioinformatics. 2011; 27(7):953-60.

PMID: 21317143 PMC: 4766743. DOI: 10.1093/bioinformatics/btr066.

References

Pawitan Y, Michiels S, Koscielny S, Gusnanto A, Ploner A . False discovery rate, sensitivity and sample size for microarray studies. Bioinformatics. 2005; 21(13):3017-24. DOI: 10.1093/bioinformatics/bti448. View

Pounds S, Morris S . Estimating the occurrence of false positives and false negatives in microarray studies by approximating and partitioning the empirical distribution of p-values. Bioinformatics. 2003; 19(10):1236-42. DOI: 10.1093/bioinformatics/btg148. View

Dalmasso C, Broet P, Moreau T . A simple procedure for estimating the false discovery rate. Bioinformatics. 2004; 21(5):660-8. DOI: 10.1093/bioinformatics/bti063. View

Gruvberger S, Ringner M, Chen Y, Panavally S, Saal L, Borg A . Estrogen receptor status in breast cancer is associated with remarkably distinct gene expression patterns. Cancer Res. 2001; 61(16):5979-84. View

Senn S, Bretz F . Power and sample size when multiple endpoints are considered. Pharm Stat. 2007; 6(3):161-70. DOI: 10.1002/pst.301. View

Hsueh H, Chen J, Kodell R . Comparison of methods for estimating the number of true null hypotheses in multiplicity testing. J Biopharm Stat. 2003; 13(4):675-89. DOI: 10.1081/BIP-120024202. View

Huang Y, Prasad M, Lemon W, Hampel H, Wright F, Kornacker K . Gene expression in papillary thyroid carcinoma reveals highly consistent profiles. Proc Natl Acad Sci U S A. 2001; 98(26):15044-9. PMC: 64980. DOI: 10.1073/pnas.251547398. View

Storey J, Tibshirani R . Statistical significance for genomewide studies. Proc Natl Acad Sci U S A. 2003; 100(16):9440-5. PMC: 170937. DOI: 10.1073/pnas.1530509100. View

Strimmer K . A unified approach to false discovery rate estimation. BMC Bioinformatics. 2008; 9:303. PMC: 2475539. DOI: 10.1186/1471-2105-9-303. View

10.

Efron B . Correlated z-values and the accuracy of large-scale statistical estimates. J Am Stat Assoc. 2010; 105(491):1042-1055. PMC: 2967047. DOI: 10.1198/jasa.2010.tm09129. View

11.

Victor A, Hommel G . Combining adaptive designs with control of the false discovery rate--a generalized definition for a global p-value. Biom J. 2007; 49(1):94-106. DOI: 10.1002/bimj.200510311. View

12.

Broberg P . A comparative review of estimates of the proportion unchanged genes and the false discovery rate. BMC Bioinformatics. 2005; 6:199. PMC: 1199583. DOI: 10.1186/1471-2105-6-199. View

13.

Pounds S, Cheng C . Improving false discovery rate estimation. Bioinformatics. 2004; 20(11):1737-45. DOI: 10.1093/bioinformatics/bth160. View

14.

Zehetmayer S, Bauer P, Posch M . Two-stage designs for experiments with a large number of hypotheses. Bioinformatics. 2005; 21(19):3771-7. DOI: 10.1093/bioinformatics/bti604. View

15.

Delongchamp R, Bowyer J, Chen J, Kodell R . Multiple-testing strategy for analyzing cDNA array data on gene expression. Biometrics. 2004; 60(3):774-82. DOI: 10.1111/j.0006-341X.2004.00228.x. View

16.

Norris A, Kahn C . Analysis of gene expression in pathophysiological states: balancing false discovery and false negative rates. Proc Natl Acad Sci U S A. 2006; 103(3):649-53. PMC: 1334678. DOI: 10.1073/pnas.0510115103. View

17.

Zehetmayer S, Bauer P, Posch M . Optimized multi-stage designs controlling the false discovery or the family-wise error rate. Stat Med. 2008; 27(21):4145-60. DOI: 10.1002/sim.3300. View

18.

Pavlidis P, Li Q, Noble W . The effect of replication on gene expression microarray experiments. Bioinformatics. 2003; 19(13):1620-7. DOI: 10.1093/bioinformatics/btg227. View