3D Deep Learning on Medical Images: A Review

Overview

Journal Sensors (Basel)

Publisher MDPI

Specialty Biotechnology

Date 2020 Sep 10

PMID 32906819

Citations 104

Authors

Satya P Singh

Lipo Wang

Sukrit Gupta

Haveesh Goli

Parasuraman Padmanabhan

Balazs Gulyas

Affiliations

Soon will be listed here.

Abstract

The rapid advancements in machine learning, graphics processing technologies and the availability of medical imaging data have led to a rapid increase in the use of deep learning models in the medical domain. This was exacerbated by the rapid advancements in convolutional neural network (CNN) based architectures, which were adopted by the medical imaging community to assist clinicians in disease diagnosis. Since the grand success of AlexNet in 2012, CNNs have been increasingly used in medical image analysis to improve the efficiency of human clinicians. In recent years, three-dimensional (3D) CNNs have been employed for the analysis of medical images. In this paper, we trace the history of how the 3D CNN was developed from its machine learning roots, we provide a brief mathematical description of 3D CNN and provide the preprocessing steps required for medical images before feeding them to 3D CNNs. We review the significant research in the field of 3D medical imaging analysis using 3D CNNs (and its variants) in different medical areas such as classification, segmentation, detection and localization. We conclude by discussing the challenges associated with the use of 3D CNNs in the medical imaging domain (and the use of deep learning models in general) and possible future trends in the field.

Citing Articles

A robust deep learning approach for segmenting cortical and trabecular bone from 3D high resolution µCT scans of mouse bone.

Lagzouli A, Pivonka P, Cooper D, Sansalone V, Othmani A Sci Rep. 2025; 15(1):8656.

PMID: 40082604 PMC: 11906900. DOI: 10.1038/s41598-025-92954-1.

From morphology to single-cell molecules: high-resolution 3D histology in biomedicine.

Xu X, Su J, Zhu R, Li K, Zhao X, Fan J Mol Cancer. 2025; 24(1):63.

PMID: 40033282 PMC: 11874780. DOI: 10.1186/s12943-025-02240-x.

End-to-end 3D instance segmentation of synthetic data and embryo microscopy images with a 3D Mask R-CNN.

David G, Faure E Front Bioinform. 2025; 4:1497539.

PMID: 39944885 PMC: 11814465. DOI: 10.3389/fbinf.2024.1497539.

Automating bone loss measurement on periapical radiographs for predicting the periodontitis stage and grade.

Ameli N, Gibson M, Kornerup I, Lagravere M, Gierl M, Lai H Front Dent Med. 2025; 5:1479380.

PMID: 39917648 PMC: 11797779. DOI: 10.3389/fdmed.2024.1479380.

Publication Counts in Context: Normalization Using Query and Reference Terms in PubMed.

Varghese J, Bickmann L, Strunker T, Neuhaus N, Tuttelmann F, Sandmann S J Med Internet Res. 2025; 27:e60616.

PMID: 39899831 PMC: 11833257. DOI: 10.2196/60616.

References

Iglehart J . Health insurers and medical-imaging policy--a work in progress. N Engl J Med. 2009; 360(10):1030-7. DOI: 10.1056/NEJMhpr0808703. View

Kamnitsas K, Ledig C, Newcombe V, Simpson J, Kane A, Menon D . Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal. 2016; 36:61-78. DOI: 10.1016/j.media.2016.10.004. View

Wernick M, Yang Y, Brankov J, Yourganov G, Strother S . Machine Learning in Medical Imaging. IEEE Signal Process Mag. 2014; 27(4):25-38. PMC: 4220564. DOI: 10.1109/MSP.2010.936730. View

Marin D, Nelson R, Schindera S, Richard S, Youngblood R, Yoshizumi T . Low-tube-voltage, high-tube-current multidetector abdominal CT: improved image quality and decreased radiation dose with adaptive statistical iterative reconstruction algorithm--initial clinical experience. Radiology. 2009; 254(1):145-53. DOI: 10.1148/radiol.09090094. View

Blendowski M, Heinrich M . Combining MRF-based deformable registration and deep binary 3D-CNN descriptors for large lung motion estimation in COPD patients. Int J Comput Assist Radiol Surg. 2018; 14(1):43-52. DOI: 10.1007/s11548-018-1888-2. View

Klein S, Staring M, Murphy K, Viergever M, Pluim J . elastix: a toolbox for intensity-based medical image registration. IEEE Trans Med Imaging. 2009; 29(1):196-205. DOI: 10.1109/TMI.2009.2035616. View

Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P . Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging. 1997; 16(2):187-98. DOI: 10.1109/42.563664. View

Pezeshk A, Hamidian S, Petrick N, Sahiner B . 3-D Convolutional Neural Networks for Automatic Detection of Pulmonary Nodules in Chest CT. IEEE J Biomed Health Inform. 2018; 23(5):2080-2090. DOI: 10.1109/JBHI.2018.2879449. View

Penney G, Weese J, Little J, Desmedt P, Hill D, Hawkes D . A comparison of similarity measures for use in 2-D-3-D medical image registration. IEEE Trans Med Imaging. 1998; 17(4):586-95. DOI: 10.1109/42.730403. View

10.

Hinton G, Osindero S, Teh Y . A fast learning algorithm for deep belief nets. Neural Comput. 2006; 18(7):1527-54. DOI: 10.1162/neco.2006.18.7.1527. View

11.

Maintz J, Viergever M . A survey of medical image registration. Med Image Anal. 2000; 2(1):1-36. DOI: 10.1016/s1361-8415(01)80026-8. View

12.

Gong L, Jiang S, Yang Z, Zhang G, Wang L . Automated pulmonary nodule detection in CT images using 3D deep squeeze-and-excitation networks. Int J Comput Assist Radiol Surg. 2019; 14(11):1969-1979. DOI: 10.1007/s11548-019-01979-1. View

13.

Doi K . Computer-aided diagnosis in medical imaging: historical review, current status and future potential. Comput Med Imaging Graph. 2007; 31(4-5):198-211. PMC: 1955762. DOI: 10.1016/j.compmedimag.2007.02.002. View

14.

Wang J, Schaffert R, Borsdorf A, Heigl B, Huang X, Hornegger J . Dynamic 2-D/3-D Rigid Registration Framework Using Point-To-Plane Correspondence Model. IEEE Trans Med Imaging. 2017; 36(9):1939-1954. DOI: 10.1109/TMI.2017.2702100. View

15.

Nie D, Zhang H, Adeli E, Liu L, Shen D . 3D Deep Learning for Multi-modal Imaging-Guided Survival Time Prediction of Brain Tumor Patients. Med Image Comput Comput Assist Interv. 2017; 9901:212-220. PMC: 5278791. DOI: 10.1007/978-3-319-46723-8_25. View

16.

Moher D, Liberati A, Tetzlaff J, Altman D . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009; 6(7):e1000097. PMC: 2707599. DOI: 10.1371/journal.pmed.1000097. View

17.

Wang S, Phillips P, Sui Y, Liu B, Yang M, Cheng H . Classification of Alzheimer's Disease Based on Eight-Layer Convolutional Neural Network with Leaky Rectified Linear Unit and Max Pooling. J Med Syst. 2018; 42(5):85. DOI: 10.1007/s10916-018-0932-7. View

18.

Torng W, Altman R . 3D deep convolutional neural networks for amino acid environment similarity analysis. BMC Bioinformatics. 2017; 18(1):302. PMC: 5472009. DOI: 10.1186/s12859-017-1702-0. View

19.

Jones D, Griffin L, Alexander D, Catani M, Horsfield M, Howard R . Spatial normalization and averaging of diffusion tensor MRI data sets. Neuroimage. 2002; 17(2):592-617. View

20.

Ha R, Chang P, Mema E, Mutasa S, Karcich J, Wynn R . Fully Automated Convolutional Neural Network Method for Quantification of Breast MRI Fibroglandular Tissue and Background Parenchymal Enhancement. J Digit Imaging. 2018; 32(1):141-147. PMC: 6382627. DOI: 10.1007/s10278-018-0114-7. View