VSGD-Net: Virtual Staining Guided Melanocyte Detection on Histopathological Images

Overview

Journal IEEE Winter Conf Appl Comput Vis

Date 2023 Mar 3

PMID 36865487

Authors

Kechun Liu

Beibin Li

Wenjun Wu

Caitlin May

Oliver Chang

Stevan Knezevich

Lisa Reisch

Joann Elmore

Linda Shapiro

Affiliations

Soon will be listed here.

Abstract

Detection of melanocytes serves as a critical prerequisite in assessing melanocytic growth patterns when diagnosing melanoma and its precursor lesions on skin biopsy specimens. However, this detection is challenging due to the visual similarity of melanocytes to other cells in routine Hematoxylin and Eosin (H&E) stained images, leading to the failure of current nuclei detection methods. Stains such as Sox10 can mark melanocytes, but they require an additional step and expense and thus are not regularly used in clinical practice. To address these limitations, we introduce VSGD-Net, a novel detection network that learns melanocyte identification through virtual staining from H&E to Sox10. The method takes only routine H&E images during inference, resulting in a promising approach to support pathologists in the diagnosis of melanoma. To the best of our knowledge, this is the first study that investigates the detection problem using image synthesis features between two distinct pathology stainings. Extensive experimental results show that our proposed model outperforms state-of-the-art nuclei detection methods for melanocyte detection. The source code and pre-trained model are available at: https://github.com/kechunl/VSGD-Net.

References

Piepkorn M, Barnhill R, Elder D, Knezevich S, Carney P, Reisch L . The MPATH-Dx reporting schema for melanocytic proliferations and melanoma. J Am Acad Dermatol. 2013; 70(1):131-41. PMC: 3992990. DOI: 10.1016/j.jaad.2013.07.027. View

Liu S, Zhang B, Liu Y, Han A, Shi H, Guan T . Unpaired Stain Transfer Using Pathology-Consistent Constrained Generative Adversarial Networks. IEEE Trans Med Imaging. 2021; 40(8):1977-1989. DOI: 10.1109/TMI.2021.3069874. View

Knoll F, Holler M, Koesters T, Otazo R, Bredies K, Sodickson D . Joint MR-PET Reconstruction Using a Multi-Channel Image Regularizer. IEEE Trans Med Imaging. 2017; 36(1):1-16. PMC: 5218518. DOI: 10.1109/TMI.2016.2564989. View

Shelhamer E, Long J, Darrell T . Fully Convolutional Networks for Semantic Segmentation. IEEE Trans Pattern Anal Mach Intell. 2016; 39(4):640-651. DOI: 10.1109/TPAMI.2016.2572683. View

Graham S, Vu Q, Raza S, Azam A, Tsang Y, Kwak J . Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med Image Anal. 2019; 58:101563. DOI: 10.1016/j.media.2019.101563. View

Zhou Z, Siddiquee M, Tajbakhsh N, Liang J . UNet++: A Nested U-Net Architecture for Medical Image Segmentation. Deep Learn Med Image Anal Multimodal Learn Clin Decis Support (2018). 2020; 11045:3-11. PMC: 7329239. DOI: 10.1007/978-3-030-00889-5_1. View

Jin C, Kim H, Liu M, Jung W, Joo S, Park E . Deep CT to MR Synthesis Using Paired and Unpaired Data. Sensors (Basel). 2019; 19(10). PMC: 6566351. DOI: 10.3390/s19102361. View

Yang L, Ghosh R, Franklin J, Chen S, You C, Narayan R . NuSeT: A deep learning tool for reliably separating and analyzing crowded cells. PLoS Comput Biol. 2020; 16(9):e1008193. PMC: 7515182. DOI: 10.1371/journal.pcbi.1008193. View

Levy J, Azizgolshani N, Andersen Jr M, Suriawinata A, Liu X, Lisovsky M . A large-scale internal validation study of unsupervised virtual trichrome staining technologies on nonalcoholic steatohepatitis liver biopsies. Mod Pathol. 2020; 34(4):808-822. PMC: 7985027. DOI: 10.1038/s41379-020-00718-1. View

10.

Hofener H, Homeyer A, Weiss N, Molin J, Lundstrom C, Hahn H . Deep learning nuclei detection: A simple approach can deliver state-of-the-art results. Comput Med Imaging Graph. 2018; 70:43-52. DOI: 10.1016/j.compmedimag.2018.08.010. View

11.

Sohail A, Khan A, Wahab N, Zameer A, Khan S . A multi-phase deep CNN based mitosis detection framework for breast cancer histopathological images. Sci Rep. 2021; 11(1):6215. PMC: 7973714. DOI: 10.1038/s41598-021-85652-1. View

12.

He K, Gkioxari G, Dollar P, Girshick R . Mask R-CNN. IEEE Trans Pattern Anal Mach Intell. 2018; 42(2):386-397. DOI: 10.1109/TPAMI.2018.2844175. View

13.

Mahmood F, Borders D, Chen R, McKay G, Salimian K, Baras A . Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images. IEEE Trans Med Imaging. 2019; 39(11):3257-3267. PMC: 8588951. DOI: 10.1109/TMI.2019.2927182. View

14.

Zhao J, Li D, Kassam Z, Howey J, Chong J, Chen B . Tripartite-GAN: Synthesizing liver contrast-enhanced MRI to improve tumor detection. Med Image Anal. 2020; 63:101667. DOI: 10.1016/j.media.2020.101667. View

15.

Elmore J, Barnhill R, Elder D, Longton G, Pepe M, Reisch L . Pathologists' diagnosis of invasive melanoma and melanocytic proliferations: observer accuracy and reproducibility study. BMJ. 2017; 357:j2813. PMC: 5485913. DOI: 10.1136/bmj.j2813. View

16.

Siegel R, Miller K, Jemal A . Cancer statistics, 2019. CA Cancer J Clin. 2019; 69(1):7-34. DOI: 10.3322/caac.21551. View

17.

Nofallah S, Mehta S, Mercan E, Knezevich S, May C, Weaver D . Machine learning techniques for mitoses classification. Comput Med Imaging Graph. 2020; 87:101832. PMC: 7855641. DOI: 10.1016/j.compmedimag.2020.101832. View

18.

Raza S, Cheung L, Shaban M, Graham S, Epstein D, Pelengaris S . Micro-Net: A unified model for segmentation of various objects in microscopy images. Med Image Anal. 2018; 52:160-173. DOI: 10.1016/j.media.2018.12.003. View

19.

Wang S, Rong R, Yang D, Fujimoto J, Yan S, Cai L . Computational Staining of Pathology Images to Study the Tumor Microenvironment in Lung Cancer. Cancer Res. 2020; 80(10):2056-2066. PMC: 7919065. DOI: 10.1158/0008-5472.CAN-19-1629. View

20.

Zhao B, Chen X, Li Z, Yu Z, Yao S, Yan L . Triple U-net: Hematoxylin-aware nuclei segmentation with progressive dense feature aggregation. Med Image Anal. 2020; 65:101786. DOI: 10.1016/j.media.2020.101786. View