STEAM: Spatial Transcriptomics Evaluation Algorithm and Metric for Clustering Performance

Overview

Journal bioRxiv

Date 2025 Mar 3

PMID 40027655

Authors

Samantha Reynoso

Courtney Schiebout

Revanth Krishna

Fan Zhang

Affiliations

Soon will be listed here.

Abstract

Motivation: Spatial transcriptomic technologies allow researchers to explore the diversity and specificity of gene expression within their original tissue structure. Accurately identifying regions that are spatially coherent in both gene expression and physical tissue structures is an emerging topic, but challenging due to the lack of ground truth labels which renders complicating validation of clustering consistency and reproducibility. This highlights a need for a computational evaluation framework to rigorously and unbiasedly assess clustering performance.

Results: To address this gap, we propose STEAM (Spatial Transcriptomics Evaluation Algorithm and Metric), a user-friendly computational pipeline designed to evaluate the consistency and reliability of clustering results by leveraging machine learning classification and prediction methods, with the goal of maintaining the spatial proximity and gene expression patterns within clusters. We benchmarked STEAM on various public datasets, spanning multi-cell to single-cell resolution, as well as spatial transcriptomics and proteomics. The results highlighted its robustness and generalizability through comprehensive statistical evaluation metrics, such as Kappa score, F1 score, accuracy, and adjusted rand index. Notably, STEAM supports multi-sample training, enabling cross-replicate clustering consistency assessment. Moreover, STEAM provides practical guidance by comparing clustering results across multiple approaches; here, we evaluated four different methods, including spatial-aware and spatial-ignorant approaches. In summary, we believe that STEAM provides researchers a promising tool for evaluating clustering robustness and benchmarking clustering performance for spatial omics data, offering valuable insights to drive reproducible discoveries in spatial biology.

Availability And Implementation: Source code and the R software tool STEAM are available from https://github.com/fanzhanglab/STEAM.

References

Moses L, Pachter L . Museum of spatial transcriptomics. Nat Methods. 2022; 19(5):534-546. DOI: 10.1038/s41592-022-01409-2. View

Chen S, Zhu B, Huang S, Hickey J, Lin K, Snyder M . Integration of spatial and single-cell data across modalities with weakly linked features. Nat Biotechnol. 2023; 42(7):1096-1106. PMC: 11638971. DOI: 10.1038/s41587-023-01935-0. View

Zhao E, Stone M, Ren X, Guenthoer J, Smythe K, Pulliam T . Spatial transcriptomics at subspot resolution with BayesSpace. Nat Biotechnol. 2021; 39(11):1375-1384. PMC: 8763026. DOI: 10.1038/s41587-021-00935-2. View

Paradis E, Schliep K . ape 5.0: an environment for modern phylogenetics and evolutionary analyses in R. Bioinformatics. 2018; 35(3):526-528. DOI: 10.1093/bioinformatics/bty633. View

Hafemeister C, Satija R . Normalization and variance stabilization of single-cell RNA-seq data using regularized negative binomial regression. Genome Biol. 2019; 20(1):296. PMC: 6927181. DOI: 10.1186/s13059-019-1874-1. View

Korsunsky I, Millard N, Fan J, Slowikowski K, Zhang F, Wei K . Fast, sensitive and accurate integration of single-cell data with Harmony. Nat Methods. 2019; 16(12):1289-1296. PMC: 6884693. DOI: 10.1038/s41592-019-0619-0. View

Becht E, McInnes L, Healy J, Dutertre C, Kwok I, Ng L . Dimensionality reduction for visualizing single-cell data using UMAP. Nat Biotechnol. 2018; . DOI: 10.1038/nbt.4314. View

Maynard K, Collado-Torres L, Weber L, Uytingco C, Barry B, Williams S . Transcriptome-scale spatial gene expression in the human dorsolateral prefrontal cortex. Nat Neurosci. 2021; 24(3):425-436. PMC: 8095368. DOI: 10.1038/s41593-020-00787-0. View

Hao Y, Stuart T, Kowalski M, Choudhary S, Hoffman P, Hartman A . Dictionary learning for integrative, multimodal and scalable single-cell analysis. Nat Biotechnol. 2023; 42(2):293-304. PMC: 10928517. DOI: 10.1038/s41587-023-01767-y. View

10.

Larsson L, Frisen J, Lundeberg J . Spatially resolved transcriptomics adds a new dimension to genomics. Nat Methods. 2021; 18(1):15-18. DOI: 10.1038/s41592-020-01038-7. View

11.

Miller B, Bambah-Mukku D, Dulac C, Zhuang X, Fan J . Characterizing spatial gene expression heterogeneity in spatially resolved single-cell transcriptomic data with nonuniform cellular densities. Genome Res. 2021; 31(10):1843-1855. PMC: 8494224. DOI: 10.1101/gr.271288.120. View

12.

Guo N, Vargas J, Reynoso S, Fritz D, Krishna R, Wang C . Uncover spatially informed variations for single-cell spatial transcriptomics with STew. Bioinform Adv. 2024; 4(1):vbae064. PMC: 11142628. DOI: 10.1093/bioadv/vbae064. View

13.

Shang L, Zhou X . Spatially aware dimension reduction for spatial transcriptomics. Nat Commun. 2022; 13(1):7203. PMC: 9684472. DOI: 10.1038/s41467-022-34879-1. View

14.

Liu W, Liao X, Yang Y, Lin H, Yeong J, Zhou X . Joint dimension reduction and clustering analysis of single-cell RNA-seq and spatial transcriptomics data. Nucleic Acids Res. 2022; 50(12):e72. PMC: 9262606. DOI: 10.1093/nar/gkac219. View

15.

Hickey J, Becker W, Nevins S, Horning A, Perez A, Zhu C . Organization of the human intestine at single-cell resolution. Nature. 2023; 619(7970):572-584. PMC: 10356619. DOI: 10.1038/s41586-023-05915-x. View

16.

Ren H, Walker B, Cang Z, Nie Q . Identifying multicellular spatiotemporal organization of cells with SpaceFlow. Nat Commun. 2022; 13(1):4076. PMC: 9283532. DOI: 10.1038/s41467-022-31739-w. View

17.

Traag V, Waltman L, van Eck N . From Louvain to Leiden: guaranteeing well-connected communities. Sci Rep. 2019; 9(1):5233. PMC: 6435756. DOI: 10.1038/s41598-019-41695-z. View

18.

Yao Z, van Velthoven C, Kunst M, Zhang M, McMillen D, Lee C . A high-resolution transcriptomic and spatial atlas of cell types in the whole mouse brain. Nature. 2023; 624(7991):317-332. PMC: 10719114. DOI: 10.1038/s41586-023-06812-z. View

19.

Cable D, Murray E, Zou L, Goeva A, Macosko E, Chen F . Robust decomposition of cell type mixtures in spatial transcriptomics. Nat Biotechnol. 2021; 40(4):517-526. PMC: 8606190. DOI: 10.1038/s41587-021-00830-w. View