Multimodal Registration of SD-OCT Volumes and Fundus Photographs Using Histograms of Oriented Gradients

Overview

Journal Biomed Opt Express

Specialty Radiology

Date 2016 Dec 27

PMID 28018740

Citations 8

Authors

Mohammad Saleh Miri

Michael D Abramoff

Young H Kwon

Mona K Garvin

Affiliations

Soon will be listed here.

Abstract

With availability of different retinal imaging modalities such as fundus photography and spectral domain optical coherence tomography (SD-OCT), having a robust and accurate registration scheme to enable utilization of this complementary information is beneficial. The few existing fundus-OCT registration approaches contain a vessel segmentation step, as the retinal blood vessels are the most dominant structures that are in common between the pair of images. However, errors in the vessel segmentation from either modality may cause corresponding errors in the registration. In this paper, we propose a feature-based registration method for registering fundus photographs and SD-OCT projection images that benefits from vasculature structural information without requiring blood vessel segmentation. In particular, after a preprocessing step, a set of control points (CPs) are identified by looking for the corners in the images. Next, each CP is represented by a feature vector which encodes the local structural information via computing the histograms of oriented gradients (HOG) from the neighborhood of each CP. The best matching CPs are identified by calculating the distance of their corresponding feature vectors. After removing the incorrect matches the best affine transform that registers fundus photographs to SD-OCT projection images is computed using the random sample consensus (RANSAC) method. The proposed method was tested on 44 pairs of fundus and SD-OCT projection images of glaucoma patients and the result showed that the proposed method successfully registers the multimodal images and produced a registration error of 25.34 ± 12.34 m (0.84 ± 0.41 pixels).

Citing Articles

Automated inter-device 3D OCT image registration using deep learning and retinal layer segmentation.

Rivas-Villar D, Motschi A, Pircher M, Hitzenberger C, Schranz M, Roberts P Biomed Opt Express. 2023; 14(7):3726-3747.

PMID: 37497506 PMC: 10368062. DOI: 10.1364/BOE.493047.

Methods for real-time feature-guided image fusion of intrasurgical volumetric optical coherence tomography with digital microscopy.

Trout R, Viehland C, Li J, Raynor W, Dhalla A, Vajzovic L Biomed Opt Express. 2023; 14(7):3308-3326.

PMID: 37497493 PMC: 10368056. DOI: 10.1364/BOE.488975.

Hybrid registration of retinal fluorescein angiography and optical coherence tomography images of patients with diabetic retinopathy.

Golkar E, Rabbani H, Dehghani A Biomed Opt Express. 2021; 12(3):1707-1724.

PMID: 33796382 PMC: 7984788. DOI: 10.1364/BOE.415939.

Artificial Intelligence for Automated Overlay of Fundus Camera and Scanning Laser Ophthalmoscope Images.

Cavichini M, An C, Bartsch D, Jhingan M, Amador-Patarroyo M, Long C Transl Vis Sci Technol. 2020; 9(2):56.

PMID: 33173612 PMC: 7594596. DOI: 10.1167/tvst.9.2.56.

Registration of fluorescein angiography and optical coherence tomography images of curved retina via scanning laser ophthalmoscopy photographs.

Almasi R, Vafaei A, Ghasemi Z, Ommani M, Dehghani A, Rabbani H Biomed Opt Express. 2020; 11(7):3455-3476.

PMID: 33014544 PMC: 7510895. DOI: 10.1364/BOE.395784.

References

Lee K, Niemeijer M, Garvin M, Kwon Y, Sonka M, Abramoff M . Segmentation of the optic disc in 3-D OCT scans of the optic nerve head. IEEE Trans Med Imaging. 2009; 29(1):159-68. PMC: 2911797. DOI: 10.1109/TMI.2009.2031324. View

Hu Z, Niemeijer M, Abramoff M, Garvin M . Multimodal retinal vessel segmentation from spectral-domain optical coherence tomography and fundus photography. IEEE Trans Med Imaging. 2012; 31(10):1900-11. PMC: 4049064. DOI: 10.1109/TMI.2012.2206822. View

Cattin P, Bay H, Gool L, Szekely G . Retina mosaicing using local features. Med Image Comput Comput Assist Interv. 2007; 9(Pt 2):185-92. DOI: 10.1007/11866763_23. View

Tsai C, Li C, Yang G, Lin K . The edge-driven dual-bootstrap iterative closest point algorithm for registration of multimodal fluorescein angiogram sequence. IEEE Trans Med Imaging. 2009; 29(3):636-49. DOI: 10.1109/TMI.2009.2030324. View

Abramoff M, Garvin M, Sonka M . Retinal imaging and image analysis. IEEE Rev Biomed Eng. 2012; 3:169-208. PMC: 3131209. DOI: 10.1109/RBME.2010.2084567. View

Chen J, Tian J, Lee N, Zheng J, Smith R, Laine A . A partial intensity invariant feature descriptor for multimodal retinal image registration. IEEE Trans Biomed Eng. 2010; 57(7):1707-18. PMC: 3030813. DOI: 10.1109/TBME.2010.2042169. View

Yang G, Stewart C, Sofka M, Tsai C . Registration of challenging image pairs: initialization, estimation, and decision. IEEE Trans Pattern Anal Mach Intell. 2007; 29(11):1973-89. DOI: 10.1109/TPAMI.2007.1116. View

Miri M, Robles V, Abramoff M, Kwon Y, Garvin M . Incorporation of gradient vector flow field in a multimodal graph-theoretic approach for segmenting the internal limiting membrane from glaucomatous optic nerve head-centered SD-OCT volumes. Comput Med Imaging Graph. 2016; 55:87-94. PMC: 5219948. DOI: 10.1016/j.compmedimag.2016.06.007. View

Zheng J, Tian J, Deng K, Dai X, Zhang X, Xu M . Salient feature region: a new method for retinal image registration. IEEE Trans Inf Technol Biomed. 2010; 15(2):221-32. DOI: 10.1109/TITB.2010.2091145. View

10.

Peli E, Augliere R, Timberlake G . Feature-based registration of retinal images. IEEE Trans Med Imaging. 1987; 6(3):272-8. DOI: 10.1109/TMI.1987.4307837. View

11.

Alward W, Longmuir S, Miri M, Garvin M, Kwon Y . Movement of Retinal Vessels to Optic Nerve Head with Intraocular Pressure Elevation in a Child. Ophthalmology. 2015; 122(7):1532-4. DOI: 10.1016/j.ophtha.2015.01.028. View

12.

Matsopoulos G, Asvestas P, Mouravliansky N, Delibasis K . Multimodal registration of retinal images using self organizing maps. IEEE Trans Med Imaging. 2004; 23(12):1557-63. DOI: 10.1109/TMI.2004.836547. View

13.

Schuman J . Spectral domain optical coherence tomography for glaucoma (an AOS thesis). Trans Am Ophthalmol Soc. 2009; 106:426-58. PMC: 2646438. View

14.

Li Y, Gregori G, Knighton R, Lujan B, Rosenfeld P . Registration of OCT fundus images with color fundus photographs based on blood vessel ridges. Opt Express. 2011; 19(1):7-16. PMC: 3368356. DOI: 10.1364/OE.19.000007. View

15.

Chauhan B, Burgoyne C . From clinical examination of the optic disc to clinical assessment of the optic nerve head: a paradigm change. Am J Ophthalmol. 2013; 156(2):218-227.e2. PMC: 3720683. DOI: 10.1016/j.ajo.2013.04.016. View

16.

Garvin M, Abramoff M, Wu X, Russell S, Burns T, Sonka M . Automated 3-D intraretinal layer segmentation of macular spectral-domain optical coherence tomography images. IEEE Trans Med Imaging. 2009; 28(9):1436-47. PMC: 2911837. DOI: 10.1109/TMI.2009.2016958. View

17.

Miri M, Abramoff M, Lee K, Niemeijer M, Wang J, Kwon Y . Multimodal Segmentation of Optic Disc and Cup From SD-OCT and Color Fundus Photographs Using a Machine-Learning Graph-Based Approach. IEEE Trans Med Imaging. 2015; 34(9):1854-66. PMC: 4560662. DOI: 10.1109/TMI.2015.2412881. View

18.

Bogunovic H, Sonka M, Kwon Y, Kemp P, Abramoff M, Wu X . Multi-surface and multi-field co-segmentation of 3-D retinal optical coherence tomography. IEEE Trans Med Imaging. 2014; 33(12):2242-53. PMC: 4326334. DOI: 10.1109/TMI.2014.2336246. View