Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images

Overview

Journal Cell Rep

Publisher Cell Press

Specialties Biology
Cell Biology
Molecular Biology

Date 2018 Apr 5

PMID 29617659

Citations 418

Authors

Joel Saltz

Rajarsi Gupta

Le Hou

Tahsin Kurc

Pankaj Singh

Vu Nguyen

Dimitris Samaras

Kenneth R Shroyer

Tianhao Zhao

Rebecca Batiste

John Van Arnam

Ilya Shmulevich

Arvind U K Rao

Alexander J Lazar

Ashish Sharma

Vesteinn Thorsson

Affiliations

Soon will be listed here.

Abstract

Beyond sample curation and basic pathologic characterization, the digitized H&E-stained images of TCGA samples remain underutilized. To highlight this resource, we present mappings of tumor-infiltrating lymphocytes (TILs) based on H&E images from 13 TCGA tumor types. These TIL maps are derived through computational staining using a convolutional neural network trained to classify patches of images. Affinity propagation revealed local spatial structure in TIL patterns and correlation with overall survival. TIL map structural patterns were grouped using standard histopathological parameters. These patterns are enriched in particular T cell subpopulations derived from molecular measures. TIL densities and spatial structure were differentially enriched among tumor types, immune subtypes, and tumor molecular subtypes, implying that spatial infiltrate state could reflect particular tumor cell aberration states. Obtaining spatial lymphocytic patterns linked to the rich genomic characterization of TCGA samples demonstrates one use for the TCGA image archives with insights into the tumor-immune microenvironment.

Citing Articles

Quantifying and interpreting biologically meaningful spatial signatures within tumor microenvironments.

Jing S, Wang H, Lin P, Yuan J, Tang Z, Li H NPJ Precis Oncol. 2025; 9(1):68.

PMID: 40069556 PMC: 11897387. DOI: 10.1038/s41698-025-00857-1.

Automated deep learning-based assessment of tumour-infiltrating lymphocyte density determines prognosis in colorectal cancer.

Millward J, He Z, Nibali A, Mouradov D, Mielke L, Tran K J Transl Med. 2025; 23(1):298.

PMID: 40065354 PMC: 11892243. DOI: 10.1186/s12967-025-06254-3.

Colorectal cancer tumor grade segmentation: A new dataset and baseline results.

Arslan D, Sehlaver S, Guder E, Temena M, Bahcekapili A, Ozdemir U Heliyon. 2025; 11(4):e42467.

PMID: 40061913 PMC: 11889546. DOI: 10.1016/j.heliyon.2025.e42467.

A novel approach for estimating postmortem intervals under varying temperature conditions using pathology images and artificial intelligence models.

Liang X, Deng M, Zhu Z, Zhang W, Li Y, Luo J Int J Legal Med. 2025; .

PMID: 40019556 DOI: 10.1007/s00414-025-03447-9.

Inflammation in cancer: therapeutic opportunities from new insights.

Xie Y, Liu F, Wu Y, Zhu Y, Jiang Y, Wu Q Mol Cancer. 2025; 24(1):51.

PMID: 39994787 PMC: 11849313. DOI: 10.1186/s12943-025-02243-8.

References

Ren S, He K, Girshick R, Sun J . Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans Pattern Anal Mach Intell. 2016; 39(6):1137-1149. DOI: 10.1109/TPAMI.2016.2577031. View

Galon J, Costes A, Sanchez-Cabo F, Kirilovsky A, Mlecnik B, Lagorce-Pages C . Type, density, and location of immune cells within human colorectal tumors predict clinical outcome. Science. 2006; 313(5795):1960-4. DOI: 10.1126/science.1129139. View

Chen H, Qi X, Yu L, Dou Q, Qin J, Heng P . DCAN: Deep contour-aware networks for object instance segmentation from histology images. Med Image Anal. 2016; 36:135-146. DOI: 10.1016/j.media.2016.11.004. View

Sirinukunwattana K, Raza S, Tsang Y, Snead D, Cree I, Rajpoot N . Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images. IEEE Trans Med Imaging. 2016; 35(5):1196-1206. DOI: 10.1109/TMI.2016.2525803. View

Rutledge W, Kong J, Gao J, Gutman D, Cooper L, Appin C . Tumor-infiltrating lymphocytes in glioblastoma are associated with specific genomic alterations and related to transcriptional class. Clin Cancer Res. 2013; 19(18):4951-60. PMC: 3865611. DOI: 10.1158/1078-0432.CCR-13-0551. View

Li B, Severson E, Pignon J, Zhao H, Li T, Novak J . Comprehensive analyses of tumor immunity: implications for cancer immunotherapy. Genome Biol. 2016; 17(1):174. PMC: 4993001. DOI: 10.1186/s13059-016-1028-7. View

Cooper L, Demicco E, Saltz J, Powell R, Rao A, Lazar A . PanCancer insights from The Cancer Genome Atlas: the pathologist's perspective. J Pathol. 2017; 244(5):512-524. PMC: 6240356. DOI: 10.1002/path.5028. View

Kardos J, Chai S, Mose L, Selitsky S, Krishnan B, Saito R . Claudin-low bladder tumors are immune infiltrated and actively immune suppressed. JCI Insight. 2016; 1(3):e85902. PMC: 5033914. DOI: 10.1172/jci.insight.85902. View

Salgado R, Denkert C, Demaria S, Sirtaine N, Klauschen F, Pruneri G . The evaluation of tumor-infiltrating lymphocytes (TILs) in breast cancer: recommendations by an International TILs Working Group 2014. Ann Oncol. 2014; 26(2):259-71. PMC: 6267863. DOI: 10.1093/annonc/mdu450. View

10.

Murthy V, Hou L, Samaras D, Kurc T, Saltz J . Center-Focusing Multi-task CNN with Injected Features for Classification of Glioma Nuclear Images. IEEE Winter Conf Appl Comput Vis. 2018; 2017:834-841. PMC: 5988234. DOI: 10.1109/WACV.2017.98. View

11.

Saltz J, Sharma A, Iyer G, Bremer E, Wang F, Jasniewski A . A Containerized Software System for Generation, Management, and Exploration of Features from Whole Slide Tissue Images. Cancer Res. 2017; 77(21):e79-e82. PMC: 5987533. DOI: 10.1158/0008-5472.CAN-17-0316. View

12.

Frey B, Dueck D . Clustering by passing messages between data points. Science. 2007; 315(5814):972-6. DOI: 10.1126/science.1136800. View

13.

Broussard E, Disis M . TNM staging in colorectal cancer: T is for T cell and M is for memory. J Clin Oncol. 2011; 29(6):601-3. DOI: 10.1200/JCO.2010.32.9078. View

14.

Xie Y, Kong X, Xing F, Liu F, Su H, Yang L . Deep Voting: A Robust Approach Toward Nucleus Localization in Microscopy Images. Med Image Comput Comput Assist Interv. 2017; 9351:374-382. PMC: 5224767. DOI: 10.1007/978-3-319-24574-4_45. View

15.

. Integrated genomic analyses of ovarian carcinoma. Nature. 2011; 474(7353):609-15. PMC: 3163504. DOI: 10.1038/nature10166. View

16.

Hou L, Samaras D, Kurc T, Gao Y, Davis J, Saltz J . Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2016; 2016:2424-2433. PMC: 5085270. DOI: 10.1109/CVPR.2016.266. View

17.

Trinchieri G . Cancer and inflammation: an old intuition with rapidly evolving new concepts. Annu Rev Immunol. 2012; 30:677-706. DOI: 10.1146/annurev-immunol-020711-075008. View

18.

Iglesia M, Vincent B, Parker J, Hoadley K, Carey L, Perou C . Prognostic B-cell signatures using mRNA-seq in patients with subtype-specific breast and ovarian cancer. Clin Cancer Res. 2014; 20(14):3818-29. PMC: 4102637. DOI: 10.1158/1078-0432.CCR-13-3368. View

19.

Rooney M, Shukla S, Wu C, Getz G, Hacohen N . Molecular and genetic properties of tumors associated with local immune cytolytic activity. Cell. 2015; 160(1-2):48-61. PMC: 4856474. DOI: 10.1016/j.cell.2014.12.033. View

20.

Ciresan D, Giusti A, Gambardella L, Schmidhuber J . Mitosis detection in breast cancer histology images with deep neural networks. Med Image Comput Comput Assist Interv. 2014; 16(Pt 2):411-8. DOI: 10.1007/978-3-642-40763-5_51. View