Signature-scoring Methods Developed for Bulk Samples Are Not Adequate for Cancer Single-cell RNA Sequencing Data

Overview

Journal Elife

Specialty Biology

Date 2022 Feb 25

PMID 35212622

Authors

Nighat Noureen

Zhenqing Ye

Yidong Chen

Xiaojing Wang

Siyuan Zheng

Affiliations

Soon will be listed here.

Abstract

Quantifying the activity of gene expression signatures is common in analyses of single-cell RNA sequencing data. Methods originally developed for bulk samples are often used for this purpose without accounting for contextual differences between bulk and single-cell data. More broadly, few attempts have been made to benchmark these methods. Here, we benchmark five such methods, including single sample gene set enrichment analysis (ssGSEA), Gene Set Variation Analysis (GSVA), AUCell, Single Cell Signature Explorer (SCSE), and a new method we developed, Jointly Assessing Signature Mean and Inferring Enrichment (JASMINE). Using cancer as an example, we show cancer cells consistently express more genes than normal cells. This imbalance leads to bias in performance by bulk-sample-based ssGSEA in gold standard tests and down sampling experiments. In contrast, single-cell-based methods are less susceptible. Our results suggest caution should be exercised when using bulk-sample-based methods in single-cell data analyses, and cellular contexts should be taken into consideration when designing benchmarking strategies.

Citing Articles

Polyploid cancer cells reveal signatures of chemotherapy resistance.

Schmidt M, Naghdloo A, Prabakar R, Kamal M, Cadaneanu R, Garraway I Oncogene. 2024; 44(7):439-449.

PMID: 39578659 PMC: 11810791. DOI: 10.1038/s41388-024-03212-z.

Differential modulation of mutant CALR and JAK2 V617F-driven oncogenesis by HLA genotype in myeloproliferative neoplasms.

Shivarov V, Tsvetkova G, Micheva I, Hadjiev E, Petrova J, Ivanova A Front Immunol. 2024; 15:1427810.

PMID: 39351227 PMC: 11439724. DOI: 10.3389/fimmu.2024.1427810.

Comparative analysis of single-cell pathway scoring methods and a novel approach.

Wang R, Thakar J NAR Genom Bioinform. 2024; 6(3):lqae124.

PMID: 39318507 PMC: 11420841. DOI: 10.1093/nargab/lqae124.

Intratumoral radiation dose heterogeneity augments antitumor immunity in mice and primes responses to checkpoint blockade.

Jagodinsky J, Vera J, Jin W, Shea A, Clark P, Sriramaneni R Sci Transl Med. 2024; 16(765):eadk0642.

PMID: 39292804 PMC: 11522033. DOI: 10.1126/scitranslmed.adk0642.

Innovative aspects and applications of single cell technology for different diseases.

Ali A, Manzoor S, Ali T, Asim M, Muhammad G, Ahmad A Am J Cancer Res. 2024; 14(8):4028-4048.

PMID: 39267684 PMC: 11387862. DOI: 10.62347/VUFU1836.

References

Noureen N, Ye Z, Chen Y, Wang X, Zheng S . Signature-scoring methods developed for bulk samples are not adequate for cancer single-cell RNA sequencing data. Elife. 2022; 11. PMC: 8916770. DOI: 10.7554/eLife.71994. View

Pont F, Tosolini M, Fournie J . Single-Cell Signature Explorer for comprehensive visualization of single cell signatures across scRNA-seq datasets. Nucleic Acids Res. 2019; 47(21):e133. PMC: 6868346. DOI: 10.1093/nar/gkz601. View

Venteicher A, Tirosh I, Hebert C, Yizhak K, Neftel C, Filbin M . Decoupling genetics, lineages, and microenvironment in IDH-mutant gliomas by single-cell RNA-seq. Science. 2017; 355(6332). PMC: 5519096. DOI: 10.1126/science.aai8478. View

Bi K, He M, Bakouny Z, Kanodia A, Napolitano S, Wu J . Tumor and immune reprogramming during immunotherapy in advanced renal cell carcinoma. Cancer Cell. 2021; 39(5):649-661.e5. PMC: 8115394. DOI: 10.1016/j.ccell.2021.02.015. View

Subramanian A, Tamayo P, Mootha V, Mukherjee S, Ebert B, Gillette M . Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proc Natl Acad Sci U S A. 2005; 102(43):15545-50. PMC: 1239896. DOI: 10.1073/pnas.0506580102. View

Neftel C, Laffy J, Filbin M, Hara T, Shore M, Rahme G . An Integrative Model of Cellular States, Plasticity, and Genetics for Glioblastoma. Cell. 2019; 178(4):835-849.e21. PMC: 6703186. DOI: 10.1016/j.cell.2019.06.024. View

McCarthy D, Campbell K, Lun A, Wills Q . Scater: pre-processing, quality control, normalization and visualization of single-cell RNA-seq data in R. Bioinformatics. 2017; 33(8):1179-1186. PMC: 5408845. DOI: 10.1093/bioinformatics/btw777. View

Qiu P . Embracing the dropouts in single-cell RNA-seq analysis. Nat Commun. 2020; 11(1):1169. PMC: 7054558. DOI: 10.1038/s41467-020-14976-9. View

Hanzelmann S, Castelo R, Guinney J . GSVA: gene set variation analysis for microarray and RNA-seq data. BMC Bioinformatics. 2013; 14:7. PMC: 3618321. DOI: 10.1186/1471-2105-14-7. View

10.

Puram S, Tirosh I, Parikh A, Patel A, Yizhak K, Gillespie S . Single-Cell Transcriptomic Analysis of Primary and Metastatic Tumor Ecosystems in Head and Neck Cancer. Cell. 2017; 171(7):1611-1624.e24. PMC: 5878932. DOI: 10.1016/j.cell.2017.10.044. View

11.

Finak G, McDavid A, Yajima M, Deng J, Gersuk V, Shalek A . MAST: a flexible statistical framework for assessing transcriptional changes and characterizing heterogeneity in single-cell RNA sequencing data. Genome Biol. 2015; 16:278. PMC: 4676162. DOI: 10.1186/s13059-015-0844-5. View

12.

Holland C, Tanevski J, Perales-Paton J, Gleixner J, Kumar M, Mereu E . Robustness and applicability of transcription factor and pathway analysis tools on single-cell RNA-seq data. Genome Biol. 2020; 21(1):36. PMC: 7017576. DOI: 10.1186/s13059-020-1949-z. View

13.

Gulati G, Sikandar S, Wesche D, Manjunath A, Bharadwaj A, Berger M . Single-cell transcriptional diversity is a hallmark of developmental potential. Science. 2020; 367(6476):405-411. PMC: 7694873. DOI: 10.1126/science.aax0249. View

14.

Lee H, Hong Y, Etlioglu H, Cho Y, Pomella V, Van den Bosch B . Lineage-dependent gene expression programs influence the immune landscape of colorectal cancer. Nat Genet. 2020; 52(6):594-603. DOI: 10.1038/s41588-020-0636-z. View

15.

Darmanis S, Sloan S, Croote D, Mignardi M, Chernikova S, Samghababi P . Single-Cell RNA-Seq Analysis of Infiltrating Neoplastic Cells at the Migrating Front of Human Glioblastoma. Cell Rep. 2017; 21(5):1399-1410. PMC: 5810554. DOI: 10.1016/j.celrep.2017.10.030. View

16.

Aibar S, Gonzalez-Blas C, Moerman T, Huynh-Thu V, Imrichova H, Hulselmans G . SCENIC: single-cell regulatory network inference and clustering. Nat Methods. 2017; 14(11):1083-1086. PMC: 5937676. DOI: 10.1038/nmeth.4463. View

17.

Ma L, Hernandez M, Zhao Y, Mehta M, Tran B, Kelly M . Tumor Cell Biodiversity Drives Microenvironmental Reprogramming in Liver Cancer. Cancer Cell. 2019; 36(4):418-430.e6. PMC: 6801104. DOI: 10.1016/j.ccell.2019.08.007. View

18.

Butler A, Hoffman P, Smibert P, Papalexi E, Satija R . Integrating single-cell transcriptomic data across different conditions, technologies, and species. Nat Biotechnol. 2018; 36(5):411-420. PMC: 6700744. DOI: 10.1038/nbt.4096. View

19.

Hicks S, Townes F, Teng M, Irizarry R . Missing data and technical variability in single-cell RNA-sequencing experiments. Biostatistics. 2017; 19(4):562-578. PMC: 6215955. DOI: 10.1093/biostatistics/kxx053. View

20.

Chung W, Eum H, Lee H, Lee K, Lee H, Kim K . Single-cell RNA-seq enables comprehensive tumour and immune cell profiling in primary breast cancer. Nat Commun. 2017; 8:15081. PMC: 5424158. DOI: 10.1038/ncomms15081. View