Deep Multi-Magnification Networks for Multi-class Breast Cancer Image Segmentation

Overview

Journal Comput Med Imaging Graph

Specialty Radiology

Date 2021 Jan 23

PMID 33485058

Citations 31

Authors

David Joon Ho

Dig V K Yarlagadda

Timothy M DAlfonso

Matthew G Hanna

Anne Grabenstetter

Peter Ntiamoah

Edi Brogi

Lee K Tan

Thomas J Fuchs

Affiliations

Soon will be listed here.

Abstract

Pathologic analysis of surgical excision specimens for breast carcinoma is important to evaluate the completeness of surgical excision and has implications for future treatment. This analysis is performed manually by pathologists reviewing histologic slides prepared from formalin-fixed tissue. In this paper, we present Deep Multi-Magnification Network trained by partial annotation for automated multi-class tissue segmentation by a set of patches from multiple magnifications in digitized whole slide images. Our proposed architecture with multi-encoder, multi-decoder, and multi-concatenation outperforms other single and multi-magnification-based architectures by achieving the highest mean intersection-over-union, and can be used to facilitate pathologists' assessments of breast cancer.

Citing Articles

Colorectal cancer tumor grade segmentation: A new dataset and baseline results.

Arslan D, Sehlaver S, Guder E, Temena M, Bahcekapili A, Ozdemir U Heliyon. 2025; 11(4):e42467.

PMID: 40061913 PMC: 11889546. DOI: 10.1016/j.heliyon.2025.e42467.

Context-guided segmentation for histopathologic cancer segmentation.

Juybari J, Hamilton J, Chen C, Khalil A, Zhu Y Sci Rep. 2025; 15(1):5404.

PMID: 39948139 PMC: 11825859. DOI: 10.1038/s41598-025-86428-7.

MUNet: a novel framework for accurate brain tumor segmentation combining UNet and mamba networks.

Yang L, Dong Q, Lin D, Tian C, Lu X Front Comput Neurosci. 2025; 19:1513059.

PMID: 39944950 PMC: 11814164. DOI: 10.3389/fncom.2025.1513059.

Out-of-distribution generalization for segmentation of lymph node metastasis in breast cancer.

Varnava Y, Jakate K, Garnett R, Androutsos D, Tyrrell P, Khademi A Sci Rep. 2025; 15(1):1127.

PMID: 39775089 PMC: 11707152. DOI: 10.1038/s41598-024-80495-y.

Invasive carcinoma segmentation in whole slide images using MS-ResMTUNet.

Liu Y, Shi H, He Q, Fu Y, Wang Y, He Y Heliyon. 2024; 10(4):e26413.

PMID: 39670062 PMC: 11636800. DOI: 10.1016/j.heliyon.2024.e26413.

References

Veta M, Pluim J, van Diest P, Viergever M . Breast cancer histopathology image analysis: a review. IEEE Trans Biomed Eng. 2014; 61(5):1400-11. DOI: 10.1109/TBME.2014.2303852. View

LeCun Y, Bengio Y, Hinton G . Deep learning. Nature. 2015; 521(7553):436-44. DOI: 10.1038/nature14539. View

Hou L, Samaras D, Kurc T, Gao Y, Davis J, Saltz J . Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2016; 2016:2424-2433. PMC: 5085270. DOI: 10.1109/CVPR.2016.266. View

Bejnordi B, Veta M, van Diest P, van Ginneken B, Karssemeijer N, Litjens G . Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer. JAMA. 2017; 318(22):2199-2210. PMC: 5820737. DOI: 10.1001/jama.2017.14585. View

DeSantis C, Ma J, Gaudet M, Newman L, Miller K, Sauer A . Breast cancer statistics, 2019. CA Cancer J Clin. 2019; 69(6):438-451. DOI: 10.3322/caac.21583. View

Nguyen K, Sarkar A, Jain A . Structure and context in prostatic gland segmentation and classification. Med Image Comput Comput Assist Interv. 2013; 15(Pt 1):115-23. DOI: 10.1007/978-3-642-33415-3_15. View

Cruz-Roa A, Gilmore H, Basavanhally A, Feldman M, Ganesan S, Shih N . Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent. Sci Rep. 2017; 7:46450. PMC: 5394452. DOI: 10.1038/srep46450. View

Moo T, Choi L, Culpepper C, Olcese C, Heerdt A, Sclafani L . Impact of margin assessment method on positive margin rate and total volume excised. Ann Surg Oncol. 2013; 21(1):86-92. PMC: 3923624. DOI: 10.1245/s10434-013-3257-2. View

Litjens G, Kooi T, Bejnordi B, Setio A, Ciompi F, Ghafoorian M . A survey on deep learning in medical image analysis. Med Image Anal. 2017; 42:60-88. DOI: 10.1016/j.media.2017.07.005. View

10.

Buda M, Maki A, Mazurowski M . A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw. 2018; 106:249-259. DOI: 10.1016/j.neunet.2018.07.011. View

11.

Badrinarayanan V, Kendall A, Cipolla R . SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Trans Pattern Anal Mach Intell. 2017; 39(12):2481-2495. DOI: 10.1109/TPAMI.2016.2644615. View

12.

Bandi P, Geessink O, Manson Q, van Dijk M, Balkenhol M, Hermsen M . From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge. IEEE Trans Med Imaging. 2019; 38(2):550-560. DOI: 10.1109/TMI.2018.2867350. View

13.

Gage I, Schnitt S, Nixon A, Silver B, Recht A, Troyan S . Pathologic margin involvement and the risk of recurrence in patients treated with breast-conserving therapy. Cancer. 1996; 78(9):1921-8. DOI: 10.1002/(sici)1097-0142(19961101)78:9<1921::aid-cncr12>3.0.co;2-#. View

14.

Gurcan M, Boucheron L, Can A, Madabhushi A, Rajpoot N, Yener B . Histopathological image analysis: a review. IEEE Rev Biomed Eng. 2010; 2:147-71. PMC: 2910932. DOI: 10.1109/RBME.2009.2034865. View

15.

Shaban M, Awan R, Fraz M, Azam A, Tsang Y, Snead D . Context-Aware Convolutional Neural Network for Grading of Colorectal Cancer Histology Images. IEEE Trans Med Imaging. 2020; 39(7):2395-2405. DOI: 10.1109/TMI.2020.2971006. View

16.

Ali S, Madabhushi A . An integrated region-, boundary-, shape-based active contour for multiple object overlap resolution in histological imagery. IEEE Trans Med Imaging. 2012; 31(7):1448-60. DOI: 10.1109/TMI.2012.2190089. View

17.

Campanella G, Hanna M, Geneslaw L, Miraflor A, Silva V, Busam K . Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med. 2019; 25(8):1301-1309. PMC: 7418463. DOI: 10.1038/s41591-019-0508-1. View

18.

Bejnordi B, Mullooly M, Pfeiffer R, Fan S, Vacek P, Weaver D . Using deep convolutional neural networks to identify and classify tumor-associated stroma in diagnostic breast biopsies. Mod Pathol. 2018; 31(10):1502-1512. PMC: 6752036. DOI: 10.1038/s41379-018-0073-z. View

19.

Petushi S, Garcia F, Haber M, Katsinis C, Tozeren A . Large-scale computations on histology images reveal grade-differentiating parameters for breast cancer. BMC Med Imaging. 2006; 6:14. PMC: 1634843. DOI: 10.1186/1471-2342-6-14. View

20.

Shelhamer E, Long J, Darrell T . Fully Convolutional Networks for Semantic Segmentation. IEEE Trans Pattern Anal Mach Intell. 2016; 39(4):640-651. DOI: 10.1109/TPAMI.2016.2572683. View