Automatic Landmark Correspondence Detection in Medical Images with an Application to Deformable Image Registration

Overview

Journal J Med Imaging (Bellingham)

Specialty Radiology

Date 2023 Feb 28

PMID 36852414

Authors

Monika Grewal

Jan Wiersma

Henrike Westerveld

Peter A N Bosman

Tanja Alderliesten

Affiliations

Soon will be listed here.

Abstract

Purpose: Deformable image registration (DIR) can benefit from additional guidance using corresponding landmarks in the images. However, the benefits thereof are largely understudied, especially due to the lack of automatic landmark detection methods for three-dimensional (3D) medical images.

Approach: We present a deep convolutional neural network (DCNN), called DCNN-Match, that learns to predict landmark correspondences in 3D images in a self-supervised manner. We trained DCNN-Match on pairs of computed tomography (CT) scans containing simulated deformations. We explored five variants of DCNN-Match that use different loss functions and assessed their effect on the spatial density of predicted landmarks and the associated matching errors. We also tested DCNN-Match variants in combination with the open-source registration software Elastix to assess the impact of predicted landmarks in providing additional guidance to DIR.

Results: We tested our approach on lower abdominal CT scans from cervical cancer patients: 121 pairs containing simulated deformations and 11 pairs demonstrating clinical deformations. The results showed significant improvement in DIR performance when landmark correspondences predicted by DCNN-Match were used in the case of simulated ( ) as well as clinical deformations ( ). We also observed that the spatial density of the automatic landmarks with respect to the underlying deformation affect the extent of improvement in DIR. Finally, DCNN-Match was found to generalize to magnetic resonance imaging scans without requiring retraining, indicating easy applicability to other datasets.

Conclusions: DCNN-match learns to predict landmark correspondences in 3D medical images in a self-supervised manner, which can improve DIR performance.

Citing Articles

Bionic Artificial Neural Networks in Medical Image Analysis.

Wang S, Chen H, Zhang Y Biomimetics (Basel). 2023; 8(2).

PMID: 37218797 PMC: 10204455. DOI: 10.3390/biomimetics8020211.

References

Klein S, Staring M, Murphy K, Viergever M, Pluim J . elastix: a toolbox for intensity-based medical image registration. IEEE Trans Med Imaging. 2009; 29(1):196-205. DOI: 10.1109/TMI.2009.2035616. View

Shamonin D, Bron E, Lelieveldt B, Smits M, Klein S, Staring M . Fast parallel image registration on CPU and GPU for diagnostic classification of Alzheimer's disease. Front Neuroinform. 2014; 7:50. PMC: 3893567. DOI: 10.3389/fninf.2013.00050. View

Weistrand O, Svensson S . The ANACONDA algorithm for deformable image registration in radiotherapy. Med Phys. 2015; 42(1):40-53. DOI: 10.1118/1.4894702. View

Esteva A, Kuprel B, Novoa R, Ko J, Swetter S, Blau H . Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017; 542(7639):115-118. PMC: 8382232. DOI: 10.1038/nature21056. View

Ruhaak J, Polzin T, Heldmann S, Simpson I, Handels H, Modersitzki J . Estimation of Large Motion in Lung CT by Integrating Regularized Keypoint Correspondences into Dense Deformable Registration. IEEE Trans Med Imaging. 2017; 36(8):1746-1757. DOI: 10.1109/TMI.2017.2691259. View

Bhalodia R, Elhabian S, Kavan L, Whitaker R . Leveraging unsupervised image registration for discovery of landmark shape descriptor. Med Image Anal. 2021; 73:102157. PMC: 8489970. DOI: 10.1016/j.media.2021.102157. View

Han D, Gao Y, Wu G, Yap P, Shen D . Robust anatomical landmark detection with application to MR brain image registration. Comput Med Imaging Graph. 2015; 46 Pt 3:277-90. PMC: 4689658. DOI: 10.1016/j.compmedimag.2015.09.002. View

Rigaud B, Klopp A, Vedam S, Venkatesan A, Taku N, Simon A . Deformable image registration for dose mapping between external beam radiotherapy and brachytherapy images of cervical cancer. Phys Med Biol. 2019; 64(11):115023. DOI: 10.1088/1361-6560/ab1378. View

Yang D, Zhang M, Chang X, Fu Y, Liu S, Li H . A method to detect landmark pairs accurately between intra-patient volumetric medical images. Med Phys. 2017; 44(11):5859-5872. DOI: 10.1002/mp.12526. View

10.

Vercauteren T, Pennec X, Perchant A, Ayache N . Diffeomorphic demons: efficient non-parametric image registration. Neuroimage. 2008; 45(1 Suppl):S61-72. DOI: 10.1016/j.neuroimage.2008.10.040. View

11.

Han D, Gao Y, Wu G, Yap P, Shen D . Robust anatomical landmark detection for MR brain image registration. Med Image Comput Comput Assist Interv. 2014; 17(Pt 1):186-93. PMC: 4206085. DOI: 10.1007/978-3-319-10404-1_24. View

12.

Yan K, Cai J, Jin D, Miao S, Guo D, Harrison A . SAM: Self-Supervised Learning of Pixel-Wise Anatomical Embeddings in Radiological Images. IEEE Trans Med Imaging. 2022; 41(10):2658-2669. DOI: 10.1109/TMI.2022.3169003. View

13.

Thevenaz P, Unser M . Optimization of mutual information for multiresolution image registration. IEEE Trans Image Process. 2008; 9(12):2083-99. DOI: 10.1109/83.887976. View

14.

Kearney V, Chen S, Gu X, Chiu T, Liu H, Jiang L . Automated landmark-guided deformable image registration. Phys Med Biol. 2014; 60(1):101-16. DOI: 10.1088/0031-9155/60/1/101. View

15.

Ghose S, Holloway L, Lim K, Chan P, Veera J, Vinod S . A review of segmentation and deformable registration methods applied to adaptive cervical cancer radiation therapy treatment planning. Artif Intell Med. 2015; 64(2):75-87. DOI: 10.1016/j.artmed.2015.04.006. View

16.

Gulshan V, Peng L, Coram M, Stumpe M, Wu D, Narayanaswamy A . Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA. 2016; 316(22):2402-2410. DOI: 10.1001/jama.2016.17216. View

17.

Devine J, Aponte J, Katz D, Liu W, Lo Vercio L, Forkert N . A Registration and Deep Learning Approach to Automated Landmark Detection for Geometric Morphometrics. Evol Biol. 2021; 47(3):246-259. PMC: 7880197. DOI: 10.1007/s11692-020-09508-8. View

18.

Chetty I, Rosu-Bubulac M . Deformable Registration for Dose Accumulation. Semin Radiat Oncol. 2019; 29(3):198-208. DOI: 10.1016/j.semradonc.2019.02.002. View

19.

Chao M, Xie Y, Xing L . Auto-propagation of contours for adaptive prostate radiation therapy. Phys Med Biol. 2008; 53(17):4533-42. DOI: 10.1088/0031-9155/53/17/005. View

20.

Bondar L, Hoogeman M, Vasquez Osorio E, Heijmen B . A symmetric nonrigid registration method to handle large organ deformations in cervical cancer patients. Med Phys. 2010; 37(7):3760-72. DOI: 10.1118/1.3443436. View