Exaggerated False Positives by Popular Differential Expression Methods when Analyzing Human Population Samples

Overview

Journal Genome Biol

Specialties Biology
Genetics

Date 2022 Mar 16

PMID 35292087

Authors

Yumei Li

Xinzhou Ge

Fanglue Peng

Wei Li

Jingyi Jessica Li

Affiliations

Soon will be listed here.

Abstract

When identifying differentially expressed genes between two conditions using human population RNA-seq samples, we found a phenomenon by permutation analysis: two popular bioinformatics methods, DESeq2 and edgeR, have unexpectedly high false discovery rates. Expanding the analysis to limma-voom, NOISeq, dearseq, and Wilcoxon rank-sum test, we found that FDR control is often failed except for the Wilcoxon rank-sum test. Particularly, the actual FDRs of DESeq2 and edgeR sometimes exceed 20% when the target FDR is 5%. Based on these results, for population-level RNA-seq studies with large sample sizes, we recommend the Wilcoxon rank-sum test.

Citing Articles

Identifying Essential Hub Genes and circRNA-Regulated ceRNA Networks in Hepatocellular Carcinoma.

Yu X, Xu H, Xing Y, Sun D, Li D, Shi J Int J Mol Sci. 2025; 26(4).

PMID: 40003874 PMC: 11855757. DOI: 10.3390/ijms26041408.

Worm Perturb-Seq: massively parallel whole-animal RNAi and RNA-seq.

Zhang H, Li X, Song D, Yukselen O, Nanda S, Kucukural A bioRxiv. 2025; .

PMID: 39975282 PMC: 11838469. DOI: 10.1101/2025.02.02.636107.

Genome-wide transcriptome differences associated with perceived discrimination in an urban, community-dwelling middle-aged cohort.

Pacheco N, Noren Hooten N, Wu S, Mensah-Bonsu M, Zhang Y, Chitrala K FASEB J. 2025; 39(3):e70366.

PMID: 39887814 PMC: 11874777. DOI: 10.1096/fj.202402000R.

Categorization of 34 computational methods to detect spatially variable genes from spatially resolved transcriptomics data.

Yan G, Hua S, Li J Nat Commun. 2025; 16(1):1141.

PMID: 39880807 PMC: 11779979. DOI: 10.1038/s41467-025-56080-w.

Combination adjuvant improves influenza virus immunity by downregulation of immune homeostasis genes in lymphocytes.

Dollinger E, Hernandez-Davies J, Felgner J, Jain A, Hwang M, Strahsburger E Immunohorizons. 2025; 9(2).

PMID: 39849993 PMC: 11841980. DOI: 10.1093/immhor/vlae007.

References

Riaz N, Havel J, Makarov V, Desrichard A, Urba W, Sims J . Tumor and Microenvironment Evolution during Immunotherapy with Nivolumab. Cell. 2017; 171(4):934-949.e16. PMC: 5685550. DOI: 10.1016/j.cell.2017.09.028. View

Tang Z, Li C, Kang B, Gao G, Li C, Zhang Z . GEPIA: a web server for cancer and normal gene expression profiling and interactive analyses. Nucleic Acids Res. 2017; 45(W1):W98-W102. PMC: 5570223. DOI: 10.1093/nar/gkx247. View

Williams C, Baccarella A, Parrish J, Kim C . Empirical assessment of analysis workflows for differential expression analysis of human samples using RNA-Seq. BMC Bioinformatics. 2017; 18(1):38. PMC: 5240434. DOI: 10.1186/s12859-016-1457-z. View

Fay M, Proschan M . Wilcoxon-Mann-Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Stat Surv. 2010; 4:1-39. PMC: 2857732. DOI: 10.1214/09-SS051. View

Love M, Huber W, Anders S . Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biol. 2014; 15(12):550. PMC: 4302049. DOI: 10.1186/s13059-014-0550-8. View

Gide T, Quek C, Menzies A, Tasker A, Shang P, Holst J . Distinct Immune Cell Populations Define Response to Anti-PD-1 Monotherapy and Anti-PD-1/Anti-CTLA-4 Combined Therapy. Cancer Cell. 2019; 35(2):238-255.e6. DOI: 10.1016/j.ccell.2019.01.003. View

. The GTEx Consortium atlas of genetic regulatory effects across human tissues. Science. 2020; 369(6509):1318-1330. PMC: 7737656. DOI: 10.1126/science.aaz1776. View

Soneson C, Delorenzi M . A comparison of methods for differential expression analysis of RNA-seq data. BMC Bioinformatics. 2013; 14:91. PMC: 3608160. DOI: 10.1186/1471-2105-14-91. View

Weinstein J, Collisson E, Mills G, Mills Shaw K, Ozenberger B, Ellrott K . The Cancer Genome Atlas Pan-Cancer analysis project. Nat Genet. 2013; 45(10):1113-20. PMC: 3919969. DOI: 10.1038/ng.2764. View

10.

Tarazona S, Furio-Tari P, Turra D, Di Pietro A, Nueda M, Ferrer A . Data quality aware analysis of differential expression in RNA-seq with NOISeq R/Bioc package. Nucleic Acids Res. 2015; 43(21):e140. PMC: 4666377. DOI: 10.1093/nar/gkv711. View

11.

Seyednasrollah F, Laiho A, Elo L . Comparison of software packages for detecting differential expression in RNA-seq studies. Brief Bioinform. 2013; 16(1):59-70. PMC: 4293378. DOI: 10.1093/bib/bbt086. View

12.

Li Y, Ge X, Peng F, Li W, Li J . Exaggerated false positives by popular differential expression methods when analyzing human population samples. Genome Biol. 2022; 23(1):79. PMC: 8922736. DOI: 10.1186/s13059-022-02648-4. View

13.

Peng L, Bian X, Li D, Xu C, Wang G, Xia Q . Large-scale RNA-Seq Transcriptome Analysis of 4043 Cancers and 548 Normal Tissue Controls across 12 TCGA Cancer Types. Sci Rep. 2015; 5:13413. PMC: 4544034. DOI: 10.1038/srep13413. View

14.

Bullard J, Purdom E, Hansen K, Dudoit S . Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments. BMC Bioinformatics. 2010; 11:94. PMC: 2838869. DOI: 10.1186/1471-2105-11-94. View

15.

Kvam V, Liu P, Si Y . A comparison of statistical methods for detecting differentially expressed genes from RNA-seq data. Am J Bot. 2012; 99(2):248-56. DOI: 10.3732/ajb.1100340. View

16.

Schurch N, Schofield P, Gierlinski M, Cole C, Sherstnev A, Singh V . How many biological replicates are needed in an RNA-seq experiment and which differential expression tool should you use?. RNA. 2016; 22(6):839-51. PMC: 4878611. DOI: 10.1261/rna.053959.115. View

17.

Ritchie M, Phipson B, Wu D, Hu Y, Law C, Shi W . limma powers differential expression analyses for RNA-sequencing and microarray studies. Nucleic Acids Res. 2015; 43(7):e47. PMC: 4402510. DOI: 10.1093/nar/gkv007. View

18.

Baik B, Yoon S, Nam D . Benchmarking RNA-seq differential expression analysis methods using spike-in and simulation data. PLoS One. 2020; 15(4):e0232271. PMC: 7192453. DOI: 10.1371/journal.pone.0232271. View

19.

Ge X, Chen Y, Song D, McDermott M, Woyshner K, Manousopoulou A . Clipper: p-value-free FDR control on high-throughput data from two conditions. Genome Biol. 2021; 22(1):288. PMC: 8504070. DOI: 10.1186/s13059-021-02506-9. View

20.

Zhang Z, Jhaveri D, Marshall V, Bauer D, Edson J, Narayanan R . A comparative study of techniques for differential expression analysis on RNA-Seq data. PLoS One. 2014; 9(8):e103207. PMC: 4132098. DOI: 10.1371/journal.pone.0103207. View