A Benchmark of Batch-effect Correction Methods for Single-cell RNA Sequencing Data

Overview

Journal Genome Biol

Specialties Biology
Genetics

Date 2020 Jan 18

PMID 31948481

Citations 452

Authors

Hoa Thi Nhu Tran

Kok Siong Ang

Marion Chevrier

Xiaomeng Zhang

Nicole Yee Shin Lee

Michelle Goh

Jinmiao Chen

Affiliations

Soon will be listed here.

Abstract

Background: Large-scale single-cell transcriptomic datasets generated using different technologies contain batch-specific systematic variations that present a challenge to batch-effect removal and data integration. With continued growth expected in scRNA-seq data, achieving effective batch integration with available computational resources is crucial. Here, we perform an in-depth benchmark study on available batch correction methods to determine the most suitable method for batch-effect removal.

Results: We compare 14 methods in terms of computational runtime, the ability to handle large datasets, and batch-effect correction efficacy while preserving cell type purity. Five scenarios are designed for the study: identical cell types with different technologies, non-identical cell types, multiple batches, big data, and simulated data. Performance is evaluated using four benchmarking metrics including kBET, LISI, ASW, and ARI. We also investigate the use of batch-corrected data to study differential gene expression.

Conclusion: Based on our results, Harmony, LIGER, and Seurat 3 are the recommended methods for batch integration. Due to its significantly shorter runtime, Harmony is recommended as the first method to try, with the other methods as viable alternatives.

Citing Articles

A single-nucleus and spatial transcriptomic atlas of the COVID-19 liver reveals topological, functional, and regenerative organ disruption in patients.

Pita-Juarez Y, Karagkouni D, Kalavros N, Melms J, Niezen S, Delorey T Genome Biol. 2025; 26(1):56.

PMID: 40087773 DOI: 10.1186/s13059-025-03499-5.

Feature selection methods affect the performance of scRNA-seq data integration and querying.

Zappia L, Richter S, Ramirez-Suastegui C, Kfuri-Rubens R, Vornholz L, Wang W Nat Methods. 2025; .

PMID: 40082610 DOI: 10.1038/s41592-025-02624-3.

A dataset of single-cell transcriptomic atlas of Bama pig and potential marker genes across seven tissues.

Chen L, Tong X, Wu Y, Liu C, Tang C, Qi X BMC Genom Data. 2025; 26(1):16.

PMID: 40075302 PMC: 11899051. DOI: 10.1186/s12863-025-01308-3.

Composite quantile regression approach to batch effect correction in microbiome data.

Park J, Park T Front Microbiol. 2025; 16:1484183.

PMID: 40071205 PMC: 11893821. DOI: 10.3389/fmicb.2025.1484183.

WCSGNet: a graph neural network approach using weighted cell-specific networks for cell-type annotation in scRNA-seq.

Wang Y, Du P Front Genet. 2025; 16:1553352.

PMID: 40034748 PMC: 11872911. DOI: 10.3389/fgene.2025.1553352.

References

Zappia L, Phipson B, Oshlack A . Splatter: simulation of single-cell RNA sequencing data. Genome Biol. 2017; 18(1):174. PMC: 5596896. DOI: 10.1186/s13059-017-1305-0. View

Buttner M, Miao Z, Wolf F, Teichmann S, Theis F . A test metric for assessing single-cell RNA-seq batch correction. Nat Methods. 2018; 16(1):43-49. DOI: 10.1038/s41592-018-0254-1. View

Saunders A, Macosko E, Wysoker A, Goldman M, Krienen F, de Rivera H . Molecular Diversity and Specializations among the Cells of the Adult Mouse Brain. Cell. 2018; 174(4):1015-1030.e16. PMC: 6447408. DOI: 10.1016/j.cell.2018.07.028. View

Rosenberg A, Roco C, Muscat R, Kuchina A, Sample P, Yao Z . Single-cell profiling of the developing mouse brain and spinal cord with split-pool barcoding. Science. 2018; 360(6385):176-182. PMC: 7643870. DOI: 10.1126/science.aam8999. View

Korsunsky I, Millard N, Fan J, Slowikowski K, Zhang F, Wei K . Fast, sensitive and accurate integration of single-cell data with Harmony. Nat Methods. 2019; 16(12):1289-1296. PMC: 6884693. DOI: 10.1038/s41592-019-0619-0. View

Wang Y, Schug J, Won K, Liu C, Naji A, Avrahami D . Single-Cell Transcriptomics of the Human Endocrine Pancreas. Diabetes. 2016; 65(10):3028-38. PMC: 5033269. DOI: 10.2337/db16-0405. View

Kharchenko P, Silberstein L, Scadden D . Bayesian approach to single-cell differential expression analysis. Nat Methods. 2014; 11(7):740-2. PMC: 4112276. DOI: 10.1038/nmeth.2967. View

Hie B, Bryson B, Berger B . Efficient integration of heterogeneous single-cell transcriptomes using Scanorama. Nat Biotechnol. 2019; 37(6):685-691. PMC: 6551256. DOI: 10.1038/s41587-019-0113-3. View

Lin Y, Ghazanfar S, Wang K, Gagnon-Bartsch J, Lo K, Su X . scMerge leverages factor analysis, stable expression, and pseudoreplication to merge multiple single-cell RNA-seq datasets. Proc Natl Acad Sci U S A. 2019; 116(20):9775-9784. PMC: 6525515. DOI: 10.1073/pnas.1820006116. View

10.

Lun A, McCarthy D, Marioni J . A step-by-step workflow for low-level analysis of single-cell RNA-seq data with Bioconductor. F1000Res. 2016; 5:2122. PMC: 5112579. DOI: 10.12688/f1000research.9501.2. View

11.

Muraro M, Dharmadhikari G, Grun D, Groen N, Dielen T, Jansen E . A Single-Cell Transcriptome Atlas of the Human Pancreas. Cell Syst. 2016; 3(4):385-394.e3. PMC: 5092539. DOI: 10.1016/j.cels.2016.09.002. View

12.

Macosko E, Basu A, Satija R, Nemesh J, Shekhar K, Goldman M . Highly Parallel Genome-wide Expression Profiling of Individual Cells Using Nanoliter Droplets. Cell. 2015; 161(5):1202-1214. PMC: 4481139. DOI: 10.1016/j.cell.2015.05.002. View

13.

Johnson W, Li C, Rabinovic A . Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics. 2006; 8(1):118-27. DOI: 10.1093/biostatistics/kxj037. View

14.

Baron M, Veres A, Wolock S, Faust A, Gaujoux R, Vetere A . A Single-Cell Transcriptomic Map of the Human and Mouse Pancreas Reveals Inter- and Intra-cell Population Structure. Cell Syst. 2016; 3(4):346-360.e4. PMC: 5228327. DOI: 10.1016/j.cels.2016.08.011. View

15.

Xin Y, Kim J, Okamoto H, Ni M, Wei Y, Adler C . RNA Sequencing of Single Human Islet Cells Reveals Type 2 Diabetes Genes. Cell Metab. 2016; 24(4):608-615. DOI: 10.1016/j.cmet.2016.08.018. View

16.

Scrucca L, Fop M, Murphy T, Raftery A . mclust 5: Clustering, Classification and Density Estimation Using Gaussian Finite Mixture Models. R J. 2016; 8(1):289-317. PMC: 5096736. View

17.

Villani A, Satija R, Reynolds G, Sarkizova S, Shekhar K, Fletcher J . Single-cell RNA-seq reveals new types of human blood dendritic cells, monocytes, and progenitors. Science. 2017; 356(6335). PMC: 5775029. DOI: 10.1126/science.aah4573. View

18.

Shaham U, Stanton K, Zhao J, Li H, Raddassi K, Montgomery R . Removal of batch effects using distribution-matching residual networks. Bioinformatics. 2017; 33(16):2539-2546. PMC: 5870543. DOI: 10.1093/bioinformatics/btx196. View

19.

Wolf F, Angerer P, Theis F . SCANPY: large-scale single-cell gene expression data analysis. Genome Biol. 2018; 19(1):15. PMC: 5802054. DOI: 10.1186/s13059-017-1382-0. View

20.

Butler A, Hoffman P, Smibert P, Papalexi E, Satija R . Integrating single-cell transcriptomic data across different conditions, technologies, and species. Nat Biotechnol. 2018; 36(5):411-420. PMC: 6700744. DOI: 10.1038/nbt.4096. View