A Deep-learning Framework to Predict Cancer Treatment Response from Histopathology Images Through Imputed Transcriptomics

Overview

Journal Nat Cancer

Specialty Oncology

Date 2024 Jul 3

PMID 38961276

Authors

Danh-Tai Hoang

Gal Dinstag

Eldad D Shulman

Leandro C Hermida

Doreen S Ben-Zvi

Efrat Elis

Katherine Caley

Stephen-John Sammut

Sanju Sinha

Neelam Sinha

Christopher H Dampier

Chani Stossel

Tejas Patil

Arun Rajan

Wiem Lassoued

Julius Strauss

Shania Bailey

Clint Allen

Jason Redman

Tuvik Beker

Peng Jiang

Talia Golan

Scott Wilkinson

Adam G Sowalsky

Sharon R Pine

Carlos Caldas

James L Gulley

Kenneth Aldape

Ranit Aharonov

Eric A Stone

Eytan Ruppin

Affiliations

Soon will be listed here.

Abstract

Advances in artificial intelligence have paved the way for leveraging hematoxylin and eosin-stained tumor slides for precision oncology. We present ENLIGHT-DeepPT, an indirect two-step approach consisting of (1) DeepPT, a deep-learning framework that predicts genome-wide tumor mRNA expression from slides, and (2) ENLIGHT, which predicts response to targeted and immune therapies from the inferred expression values. We show that DeepPT successfully predicts transcriptomics in all 16 The Cancer Genome Atlas cohorts tested and generalizes well to two independent datasets. ENLIGHT-DeepPT successfully predicts true responders in five independent patient cohorts involving four different treatments spanning six cancer types, with an overall odds ratio of 2.28 and a 39.5% increased response rate among predicted responders versus the baseline rate. Notably, its prediction accuracy, obtained without any training on the treatment data, is comparable to that achieved by directly predicting the response from the images, which requires specific training on the treatment evaluation cohorts.

Citing Articles

Hallmarks of artificial intelligence contributions to precision oncology.

Chang T, Park S, Schaffer A, Jiang P, Ruppin E Nat Cancer. 2025; .

PMID: 40055572 DOI: 10.1038/s43018-025-00917-2.

Ligand-receptor interactions combined with histopathology for improved prognostic modeling in HPV-negative head and neck squamous cell carcinoma.

Feng B, Zhao D, Zhang Z, Jia R, Schuler P, Hess J NPJ Precis Oncol. 2025; 9(1):57.

PMID: 40021759 PMC: 11871237. DOI: 10.1038/s41698-025-00844-6.

Invasion and metastasis in cancer: molecular insights and therapeutic targets.

Li Y, Liu F, Cai Q, Deng L, OuYang Q, Zhang X Signal Transduct Target Ther. 2025; 10(1):57.

PMID: 39979279 PMC: 11842613. DOI: 10.1038/s41392-025-02148-4.

Machine learning methods for histopathological image analysis: Updates in 2024.

Komura D, Ochi M, Ishikawa S Comput Struct Biotechnol J. 2025; 27:383-400.

PMID: 39897057 PMC: 11786909. DOI: 10.1016/j.csbj.2024.12.033.

Evolution of Artificial Intelligence in Medical Education From 2000 to 2024: Bibliometric Analysis.

Li R, Wu T Interact J Med Res. 2025; 14:e63775.

PMID: 39883926 PMC: 11826936. DOI: 10.2196/63775.

References

Golub T, Slonim D, Tamayo P, Huard C, Gaasenbeek M, Mesirov J . Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science. 1999; 286(5439):531-7. DOI: 10.1126/science.286.5439.531. View

Curtis C, Shah S, Chin S, Turashvili G, Rueda O, Dunning M . The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature. 2012; 486(7403):346-52. PMC: 3440846. DOI: 10.1038/nature10983. View

Doroshow D, Doroshow J . Genomics and the History of Precision Oncology. Surg Oncol Clin N Am. 2019; 29(1):35-49. PMC: 6878897. DOI: 10.1016/j.soc.2019.08.003. View

Rosenthal J, Carelli R, Omar M, Brundage D, Halbert E, Nyman J . Building Tools for Machine Learning and Artificial Intelligence in Cancer Research: Best Practices and a Case Study with the PathML Toolkit for Computational Pathology. Mol Cancer Res. 2021; 20(2):202-206. PMC: 9127877. DOI: 10.1158/1541-7786.MCR-21-0665. View

Strom P, Kartasalo K, Olsson H, Solorzano L, Delahunt B, Berney D . Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. Lancet Oncol. 2020; 21(2):222-232. DOI: 10.1016/S1470-2045(19)30738-7. View

Fu Y, Jung A, Torne R, Gonzalez S, Vohringer H, Shmatko A . Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nat Cancer. 2022; 1(8):800-810. DOI: 10.1038/s43018-020-0085-8. View

Noorbakhsh J, Farahmand S, Foroughi Pour A, Namburi S, Caruana D, Rimm D . Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images. Nat Commun. 2020; 11(1):6367. PMC: 7733499. DOI: 10.1038/s41467-020-20030-5. View

Kim J, Rustam S, Mosquera J, Randell S, Shaykhiev R, Rendeiro A . Unsupervised discovery of tissue architecture in multiplexed imaging. Nat Methods. 2022; 19(12):1653-1661. PMC: 11102857. DOI: 10.1038/s41592-022-01657-2. View

Coudray N, Ocampo P, Sakellaropoulos T, Narula N, Snuderl M, Fenyo D . Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med. 2018; 24(10):1559-1567. PMC: 9847512. DOI: 10.1038/s41591-018-0177-5. View

10.

Swiderska-Chadaj Z, Pinckaers H, van Rijthoven M, Balkenhol M, Melnikova M, Geessink O . Learning to detect lymphocytes in immunohistochemistry with deep learning. Med Image Anal. 2019; 58:101547. DOI: 10.1016/j.media.2019.101547. View

11.

Yu K, Wang F, Berry G, Re C, Altman R, Snyder M . Classifying non-small cell lung cancer types and transcriptomic subtypes using convolutional neural networks. J Am Med Inform Assoc. 2020; 27(5):757-769. PMC: 7309263. DOI: 10.1093/jamia/ocz230. View

12.

Couture H, Williams L, Geradts J, Nyante S, Butler E, Marron J . Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype. NPJ Breast Cancer. 2018; 4:30. PMC: 6120869. DOI: 10.1038/s41523-018-0079-1. View

13.

Campanella G, Hanna M, Geneslaw L, Miraflor A, Silva V, Busam K . Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med. 2019; 25(8):1301-1309. PMC: 7418463. DOI: 10.1038/s41591-019-0508-1. View

14.

Xu H, Clemenceau J, Park S, Choi J, Lee S, Hwang T . Spatial heterogeneity and organization of tumor mutation burden with immune infiltrates within tumors based on whole slide images correlated with patient survival in bladder cancer. J Pathol Inform. 2022; 13:100105. PMC: 9577053. DOI: 10.1016/j.jpi.2022.100105. View

15.

Kather J, Heij L, Grabsch H, Loeffler C, Echle A, Muti H . Pan-cancer image-based detection of clinically actionable genetic alterations. Nat Cancer. 2021; 1(8):789-799. PMC: 7610412. DOI: 10.1038/s43018-020-0087-6. View

16.

Qu H, Zhou M, Yan Z, Wang H, Rustgi V, Zhang S . Genetic mutation and biological pathway prediction based on whole slide images in breast carcinoma using deep learning. NPJ Precis Oncol. 2021; 5(1):87. PMC: 8460699. DOI: 10.1038/s41698-021-00225-9. View

17.

Tsou P, Wu C . Mapping Driver Mutations to Histopathological Subtypes in Papillary Thyroid Carcinoma: Applying a Deep Convolutional Neural Network. J Clin Med. 2019; 8(10). PMC: 6832421. DOI: 10.3390/jcm8101675. View

18.

Chang P, Grinband J, Weinberg B, Bardis M, Khy M, Cadena G . Deep-Learning Convolutional Neural Networks Accurately Classify Genetic Mutations in Gliomas. AJNR Am J Neuroradiol. 2018; 39(7):1201-1207. PMC: 6880932. DOI: 10.3174/ajnr.A5667. View

19.

Kather J, Pearson A, Halama N, Jager D, Krause J, Loosen S . Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat Med. 2019; 25(7):1054-1056. PMC: 7423299. DOI: 10.1038/s41591-019-0462-y. View

20.

Chen M, Zhang B, Topatana W, Cao J, Zhu H, Juengpanich S . Classification and mutation prediction based on histopathology H&E images in liver cancer using deep learning. NPJ Precis Oncol. 2020; 4:14. PMC: 7280520. DOI: 10.1038/s41698-020-0120-3. View