A Review of Deep Learning Based Methods for Medical Image Multi-organ Segmentation

Overview

Journal Phys Med

Specialties Biophysics
General Medicine

Date 2021 May 16

PMID 33992856

Citations 47

Authors

Yabo Fu

Yang Lei

Tonghe Wang

Walter J Curran

Tian Liu

Xiaofeng Yang

Affiliations

Soon will be listed here.

Abstract

Deep learning has revolutionized image processing and achieved the-state-of-art performance in many medical image segmentation tasks. Many deep learning-based methods have been published to segment different parts of the body for different medical applications. It is necessary to summarize the current state of development for deep learning in the field of medical image segmentation. In this paper, we aim to provide a comprehensive review with a focus on multi-organ image segmentation, which is crucial for radiotherapy where the tumor and organs-at-risk need to be contoured for treatment planning. We grouped the surveyed methods into two broad categories which are 'pixel-wise classification' and 'end-to-end segmentation'. Each category was divided into subgroups according to their network design. For each type, we listed the surveyed works, highlighted important contributions and identified specific challenges. Following the detailed review, we discussed the achievements, shortcomings and future potentials of each category. To enable direct comparison, we listed the performance of the surveyed works that used thoracic and head-and-neck benchmark datasets.

Citing Articles

Sex-Based Bias in Artificial Intelligence-Based Segmentation Models in Clinical Oncology.

Doo F, Naranjo W, Kapouranis T, Thor M, Chao M, Yang X Clin Oncol (R Coll Radiol). 2025; 39:103758.

PMID: 39874747 PMC: 11850178. DOI: 10.1016/j.clon.2025.103758.

Multimodal Machine Learning for Predicting Post-Surgery Quality of Life in Colorectal Cancer Patients.

Rhanoui M, Mikram M, Amazian K, Ait-Abderrahim A, Yousfi S, Toughrai I J Imaging. 2024; 10(12).

PMID: 39728194 PMC: 11676117. DOI: 10.3390/jimaging10120297.

Cone-Beam CT to CT Image Translation Using a Transformer-Based Deep Learning Model for Prostate Cancer Adaptive Radiotherapy.

Koike Y, Takegawa H, Anetai Y, Nakamura S, Yoshida K, Yoshida A J Imaging Inform Med. 2024; .

PMID: 39511015 DOI: 10.1007/s10278-024-01312-6.

Automated abdominal organ segmentation algorithms for non-enhanced CT for volumetry and 3D radiomics analysis.

Park J, Joo I, Jeon S, Kim J, Park S, Yoon S Abdom Radiol (NY). 2024; 50(3):1448-1456.

PMID: 39299987 PMC: 11821665. DOI: 10.1007/s00261-024-04581-5.

A Systematic Review of Real-Time Deep Learning Methods for Image-Based Cancer Diagnostics.

Sriraman H, Badarudeen S, Vats S, Balasubramanian P J Multidiscip Healthc. 2024; 17:4411-4425.

PMID: 39281299 PMC: 11397255. DOI: 10.2147/JMDH.S446745.

References

Fu Y, Lei Y, Wang T, Curran W, Liu T, Yang X . Deep learning in medical image registration: a review. Phys Med Biol. 2020; 65(20):20TR01. PMC: 7759388. DOI: 10.1088/1361-6560/ab843e. View

Lei Y, Jeong J, Wang T, Shu H, Patel P, Tian S . MRI-based pseudo CT synthesis using anatomical signature and alternating random forest with iterative refinement model. J Med Imaging (Bellingham). 2019; 5(4):043504. PMC: 6280993. DOI: 10.1117/1.JMI.5.4.043504. View

Harms J, Lei Y, Wang T, McDonald M, Ghavidel B, Stokes W . Cone-beam CT-derived relative stopping power map generation via deep learning for proton radiotherapy. Med Phys. 2020; 47(9):4416-4427. PMC: 11650372. DOI: 10.1002/mp.14347. View

Liang S, Thung K, Nie D, Zhang Y, Shen D . Multi-View Spatial Aggregation Framework for Joint Localization and Segmentation of Organs at Risk in Head and Neck CT Images. IEEE Trans Med Imaging. 2020; 39(9):2794-2805. DOI: 10.1109/TMI.2020.2975853. View

Bae H, Kim C, Kim N, Park B, Kim N, Seo J . A Perlin Noise-Based Augmentation Strategy for Deep Learning with Small Data Samples of HRCT Images. Sci Rep. 2018; 8(1):17687. PMC: 6283833. DOI: 10.1038/s41598-018-36047-2. View

Zeng Q, Fu Y, Tian Z, Lei Y, Zhang Y, Wang T . Label-driven magnetic resonance imaging (MRI)-transrectal ultrasound (TRUS) registration using weakly supervised learning for MRI-guided prostate radiotherapy. Phys Med Biol. 2020; 65(13):135002. PMC: 7771987. DOI: 10.1088/1361-6560/ab8cd6. View

Liao Z, Lee J, Komaki R, Gomez D, OReilly M, Fossella F . Bayesian Adaptive Randomization Trial of Passive Scattering Proton Therapy and Intensity-Modulated Photon Radiotherapy for Locally Advanced Non-Small-Cell Lung Cancer. J Clin Oncol. 2018; 36(18):1813-1822. PMC: 6008104. DOI: 10.1200/JCO.2017.74.0720. View

Park J, Yun J, Kim N, Park B, Cho Y, Park H . Fully Automated Lung Lobe Segmentation in Volumetric Chest CT with 3D U-Net: Validation with Intra- and Extra-Datasets. J Digit Imaging. 2019; 33(1):221-230. PMC: 7064651. DOI: 10.1007/s10278-019-00223-1. View

Lei Y, Fu Y, Wang T, Liu Y, Patel P, Curran W . 4D-CT deformable image registration using multiscale unsupervised deep learning. Phys Med Biol. 2020; 65(8):085003. PMC: 7775640. DOI: 10.1088/1361-6560/ab79c4. View

10.

Zhou X, Takayama R, Wang S, Hara T, Fujita H . Deep learning of the sectional appearances of 3D CT images for anatomical structure segmentation based on an FCN voting method. Med Phys. 2017; 44(10):5221-5233. DOI: 10.1002/mp.12480. View

11.

K J, R G, A M . Fuzzy-C-Means Clustering Based Segmentation and CNN-Classification for Accurate Segmentation of Lung Nodules. Asian Pac J Cancer Prev. 2017; 18(7):1869-1874. PMC: 5648392. DOI: 10.22034/APJCP.2017.18.7.1869. View

12.

Molitoris J, Diwanji T, Snider 3rd J, Mossahebi S, Samanta S, Badiyan S . Advances in the use of motion management and image guidance in radiation therapy treatment for lung cancer. J Thorac Dis. 2018; 10(Suppl 21):S2437-S2450. PMC: 6123191. DOI: 10.21037/jtd.2018.01.155. View

13.

Wang T, Lei Y, Fu Y, Curran W, Liu T, Nye J . Machine learning in quantitative PET: A review of attenuation correction and low-count image reconstruction methods. Phys Med. 2020; 76:294-306. PMC: 7484241. DOI: 10.1016/j.ejmp.2020.07.028. View

14.

Liu Y, Lei Y, Wang T, Fu Y, Tang X, Curran W . CBCT-based synthetic CT generation using deep-attention cycleGAN for pancreatic adaptive radiotherapy. Med Phys. 2020; 47(6):2472-2483. PMC: 7762616. DOI: 10.1002/mp.14121. View

15.

Ren S, He K, Girshick R, Sun J . Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans Pattern Anal Mach Intell. 2016; 39(6):1137-1149. DOI: 10.1109/TPAMI.2016.2577031. View

16.

Yang X, Rossi P, Ogunleye T, Marcus D, Jani A, Mao H . Prostate CT segmentation method based on nonrigid registration in ultrasound-guided CT-based HDR prostate brachytherapy. Med Phys. 2014; 41(11):111915. PMC: 4241831. DOI: 10.1118/1.4897615. View

17.

Sun S, Bauer C, Beichel R . Automated 3-D segmentation of lungs with lung cancer in CT data using a novel robust active shape model approach. IEEE Trans Med Imaging. 2011; 31(2):449-60. PMC: 3657761. DOI: 10.1109/TMI.2011.2171357. View

18.

Yang X, Wu N, Cheng G, Zhou Z, Yu D, Beitler J . Automated segmentation of the parotid gland based on atlas registration and machine learning: a longitudinal MRI study in head-and-neck radiation therapy. Int J Radiat Oncol Biol Phys. 2014; 90(5):1225-33. PMC: 4362545. DOI: 10.1016/j.ijrobp.2014.08.350. View

19.

Son J, Park S, Jung K . Towards Accurate Segmentation of Retinal Vessels and the Optic Disc in Fundoscopic Images with Generative Adversarial Networks. J Digit Imaging. 2018; 32(3):499-512. PMC: 6499859. DOI: 10.1007/s10278-018-0126-3. View

20.

Chen X, Pan L . A Survey of Graph Cuts/Graph Search Based Medical Image Segmentation. IEEE Rev Biomed Eng. 2018; 11:112-124. DOI: 10.1109/RBME.2018.2798701. View