A Review of Deep Learning-based Three-dimensional Medical Image Registration Methods

Overview

Journal Quant Imaging Med Surg

Specialty Radiology

Date 2021 Dec 10

PMID 34888197

Citations 18

Authors

Haonan Xiao

Xinzhi Teng

Chenyang Liu

Tian Li

Ge Ren

Ruijie Yang

Dinggang Shen

Jing Cai

Affiliations

Soon will be listed here.

Abstract

Medical image registration is a vital component of many medical procedures, such as image-guided radiotherapy (IGRT), as it allows for more accurate dose-delivery and better management of side effects. Recently, the successful implementation of deep learning (DL) in various fields has prompted many research groups to apply DL to three-dimensional (3D) medical image registration. Several of these efforts have led to promising results. This review summarized the progress made in DL-based 3D image registration over the past 5 years and identify existing challenges and potential avenues for further research. The collected studies were statistically analyzed based on the region of interest (ROI), image modality, supervision method, and registration evaluation metrics. The studies were classified into three categories: deep iterative registration, supervised registration, and unsupervised registration. The studies are thoroughly reviewed and their unique contributions are highlighted. A summary is presented following a review of each category of study, discussing its advantages, challenges, and trends. Finally, the common challenges for all categories are discussed, and potential future research topics are identified.

Citing Articles

Artificial Intelligence-Empowered Radiology-Current Status and Critical Review.

Obuchowicz R, Lasek J, Wodzinski M, Piorkowski A, Strzelecki M, Nurzynska K Diagnostics (Basel). 2025; 15(3).

PMID: 39941212 PMC: 11816879. DOI: 10.3390/diagnostics15030282.

Enhancing unsupervised learning in medical image registration through scale-aware context aggregation.

Liu Y, Wang L, Ning X, Gao Y, Wang D iScience. 2025; 28(2):111734.

PMID: 39898031 PMC: 11787544. DOI: 10.1016/j.isci.2024.111734.

Development and validation of a multi-parametric MRI deep-learning model for preoperative lymphovascular invasion evaluation in rectal cancer.

Shi S, Singh A, Ma J, Nie X, Kong X, Xiao L Quant Imaging Med Surg. 2025; 15(1):427-439.

PMID: 39839029 PMC: 11744136. DOI: 10.21037/qims-24-789.

A survey on deep learning in medical image registration: New technologies, uncertainty, evaluation metrics, and beyond.

Chen J, Liu Y, Wei S, Bian Z, Subramanian S, Carass A Med Image Anal. 2024; 100():103385.

PMID: 39612808 PMC: 11730935. DOI: 10.1016/j.media.2024.103385.

Navigating the innovation policy dilemma: How subnational governments balance expenditure competition pressures and long-term innovation goals.

Song W, Zhao K Heliyon. 2024; 10(15):e34787.

PMID: 39145017 PMC: 11320305. DOI: 10.1016/j.heliyon.2024.e34787.

References

Onieva J, Marti-Fuster B, de la Puente M, San Jose Estepar R . Diffeomorphic Lung Registration Using Deep CNNs and Reinforced Learning. Image Anal Mov Organ Breast Thorac Images (2018). 2020; 11040:284-294. PMC: 7266290. DOI: 10.1007/978-3-030-00946-5_28. View

Vishnevskiy V, Gass T, Szekely G, Tanner C, Goksel O . Isotropic Total Variation Regularization of Displacements in Parametric Image Registration. IEEE Trans Med Imaging. 2016; 36(2):385-395. DOI: 10.1109/TMI.2016.2610583. View

Verellen D, De Ridder M, Linthout N, Tournel K, Soete G, Storme G . Innovations in image-guided radiotherapy. Nat Rev Cancer. 2007; 7(12):949-60. DOI: 10.1038/nrc2288. View

Zhu Z, Cao Y, Qin C, Rao Y, Lin D, Dou Q . Joint affine and deformable three-dimensional networks for brain MRI registration. Med Phys. 2020; 48(3):1182-1196. DOI: 10.1002/mp.14674. View

Marcus D, Wang T, Parker J, Csernansky J, Morris J, Buckner R . Open Access Series of Imaging Studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. J Cogn Neurosci. 2007; 19(9):1498-507. DOI: 10.1162/jocn.2007.19.9.1498. View

Fan J, Cao X, Wang Q, Yap P, Shen D . Adversarial learning for mono- or multi-modal registration. Med Image Anal. 2019; 58:101545. PMC: 7455790. DOI: 10.1016/j.media.2019.101545. View

Holmes A, Hollinshead M, OKeefe T, Petrov V, Fariello G, Wald L . Brain Genomics Superstruct Project initial data release with structural, functional, and behavioral measures. Sci Data. 2015; 2:150031. PMC: 4493828. DOI: 10.1038/sdata.2015.31. View

Huang Y, Ahmad S, Fan J, Shen D, Yap P . Difficulty-aware hierarchical convolutional neural networks for deformable registration of brain MR images. Med Image Anal. 2020; 67:101817. PMC: 7725910. DOI: 10.1016/j.media.2020.101817. View

Shen D, Davatzikos C . HAMMER: hierarchical attribute matching mechanism for elastic registration. IEEE Trans Med Imaging. 2003; 21(11):1421-39. DOI: 10.1109/TMI.2002.803111. View

10.

Klein S, Staring M, Murphy K, Viergever M, Pluim J . elastix: a toolbox for intensity-based medical image registration. IEEE Trans Med Imaging. 2009; 29(1):196-205. DOI: 10.1109/TMI.2009.2035616. View

11.

. The ADHD-200 Consortium: A Model to Advance the Translational Potential of Neuroimaging in Clinical Neuroscience. Front Syst Neurosci. 2012; 6:62. PMC: 3433679. DOI: 10.3389/fnsys.2012.00062. View

12.

Lei Y, Fu Y, Wang T, Liu Y, Patel P, Curran W . 4D-CT deformable image registration using multiscale unsupervised deep learning. Phys Med Biol. 2020; 65(8):085003. PMC: 7775640. DOI: 10.1088/1361-6560/ab79c4. View

13.

Castillo R, Castillo E, Guerra R, Johnson V, McPhail T, Garg A . A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. Phys Med Biol. 2009; 54(7):1849-70. DOI: 10.1088/0031-9155/54/7/001. View

14.

Li W, Li Y, Qin W, Liang X, Xu J, Xiong J . Magnetic resonance image (MRI) synthesis from brain computed tomography (CT) images based on deep learning methods for magnetic resonance (MR)-guided radiotherapy. Quant Imaging Med Surg. 2020; 10(6):1223-1236. PMC: 7276358. DOI: 10.21037/qims-19-885. View

15.

Cao X, Yang J, Zhang J, Nie D, Kim M, Wang Q . Deformable Image Registration based on Similarity-Steered CNN Regression. Med Image Comput Comput Assist Interv. 2017; 10433:300-308. PMC: 5731783. DOI: 10.1007/978-3-319-66182-7_35. View

16.

Vandemeulebroucke J, Rit S, Kybic J, Clarysse P, Sarrut D . Spatiotemporal motion estimation for respiratory-correlated imaging of the lungs. Med Phys. 2011; 38(1):166-78. DOI: 10.1118/1.3523619. View

17.

Fu Y, Lei Y, Wang T, Curran W, Liu T, Yang X . Deep learning in medical image registration: a review. Phys Med Biol. 2020; 65(20):20TR01. PMC: 7759388. DOI: 10.1088/1361-6560/ab843e. View

18.

Liang X, Yin F, Wang C, Cai J . A robust deformable image registration enhancement method based on radial basis function. Quant Imaging Med Surg. 2019; 9(7):1315-1325. PMC: 6685815. DOI: 10.21037/qims.2019.07.05. View

19.

Teng X, Chen Y, Zhang Y, Ren L . Respiratory deformation registration in 4D-CT/cone beam CT using deep learning. Quant Imaging Med Surg. 2021; 11(2):737-748. PMC: 7779910. DOI: 10.21037/qims-19-1058. View

20.

Ren G, Zhang J, Li T, Xiao H, Cheung L, Ho W . Deep Learning-Based Computed Tomography Perfusion Mapping (DL-CTPM) for Pulmonary CT-to-Perfusion Translation. Int J Radiat Oncol Biol Phys. 2021; 110(5):1508-1518. DOI: 10.1016/j.ijrobp.2021.02.032. View