Head-Mounted Display-Based Augmented Reality for Image-Guided Media Delivery to the Heart: A Preliminary Investigation of Perceptual Accuracy

Overview

Journal J Imaging

Publisher MDPI

Specialty Radiology

Date 2022 Feb 24

PMID 35200735

Authors

Mitchell Doughty

Nilesh R Ghugre

Affiliations

Soon will be listed here.

Abstract

By aligning virtual augmentations with real objects, optical see-through head-mounted display (OST-HMD)-based augmented reality (AR) can enhance user-task performance. Our goal was to compare the perceptual accuracy of several visualization paradigms involving an adjacent monitor, or the Microsoft HoloLens 2 OST-HMD, in a targeted task, as well as to assess the feasibility of displaying imaging-derived virtual models aligned with the injured porcine heart. With 10 participants, we performed a user study to quantify and compare the accuracy, speed, and subjective workload of each paradigm in the completion of a point-and-trace task that simulated surgical targeting. To demonstrate the clinical potential of our system, we assessed its use for the visualization of magnetic resonance imaging (MRI)-based anatomical models, aligned with the surgically exposed heart in a motion-arrested open-chest porcine model. Using the HoloLens 2 with alignment of the ground truth target and our display calibration method, users were able to achieve submillimeter accuracy (0.98 mm) and required 1.42 min for calibration in the point-and-trace task. In the porcine study, we observed good spatial agreement between the MRI-models and target surgical site. The use of an OST-HMD led to improved perceptual accuracy and task-completion times in a simulated targeting task.

Citing Articles

Is Overlain Display a Right Choice for AR Navigation? A Qualitative Study of Head-Mounted Augmented Reality Surgical Navigation on Accuracy for Large-Scale Clinical Deployment.

Ye J, Chen Q, Zhong T, Liu J, Gao H CNS Neurosci Ther. 2025; 31(1):e70217.

PMID: 39817491 PMC: 11736426. DOI: 10.1111/cns.70217.

Microsoft HoloLens 2 in Medical and Healthcare Context: State of the Art and Future Prospects.

Palumbo A Sensors (Basel). 2022; 22(20).

PMID: 36298059 PMC: 9611914. DOI: 10.3390/s22207709.

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery.

Doughty M, Ghugre N, Wright G J Imaging. 2022; 8(7).

PMID: 35877647 PMC: 9318659. DOI: 10.3390/jimaging8070203.

HMD-EgoPose: head-mounted display-based egocentric marker-less tool and hand pose estimation for augmented surgical guidance.

Doughty M, Ghugre N Int J Comput Assist Radiol Surg. 2022; 17(12):2253-2262.

PMID: 35701681 DOI: 10.1007/s11548-022-02688-y.

References

Perkins S, Daniel B, Hargreaves B . MR imaging of magnetic ink patterns via off-resonance sensitivity. Magn Reson Med. 2018; 80(5):2017-2023. PMC: 6107406. DOI: 10.1002/mrm.27187. View

Meola A, Cutolo F, Carbone M, Cagnazzo F, Ferrari M, Ferrari V . Augmented reality in neurosurgery: a systematic review. Neurosurg Rev. 2016; 40(4):537-548. PMC: 6155988. DOI: 10.1007/s10143-016-0732-9. View

Roche E, Hastings C, Lewin S, Shvartsman D, Brudno Y, Vasilyev N . Comparison of biomaterial delivery vehicles for improving acute retention of stem cells in the infarcted heart. Biomaterials. 2014; 35(25):6850-6858. PMC: 4051834. DOI: 10.1016/j.biomaterials.2014.04.114. View

Eckert M, Volmerg J, Friedrich C . Augmented Reality in Medicine: Systematic and Bibliographic Review. JMIR Mhealth Uhealth. 2019; 7(4):e10967. PMC: 6658230. DOI: 10.2196/10967. View

Jud L, Fotouhi J, Andronic O, Aichmair A, Osgood G, Navab N . Applicability of augmented reality in orthopedic surgery - A systematic review. BMC Musculoskelet Disord. 2020; 21(1):103. PMC: 7023780. DOI: 10.1186/s12891-020-3110-2. View

Bar A, Cohen S . Inducing Endogenous Cardiac Regeneration: Can Biomaterials Connect the Dots?. Front Bioeng Biotechnol. 2020; 8:126. PMC: 7056668. DOI: 10.3389/fbioe.2020.00126. View

Bernhardt S, Nicolau S, Soler L, Doignon C . The status of augmented reality in laparoscopic surgery as of 2016. Med Image Anal. 2017; 37:66-90. DOI: 10.1016/j.media.2017.01.007. View

Chong J, Yang X, Don C, Minami E, Liu Y, Weyers J . Human embryonic-stem-cell-derived cardiomyocytes regenerate non-human primate hearts. Nature. 2014; 510(7504):273-7. PMC: 4154594. DOI: 10.1038/nature13233. View

Fitzpatrick J, West J, Maurer Jr C . Predicting error in rigid-body point-based registration. IEEE Trans Med Imaging. 1999; 17(5):694-702. DOI: 10.1109/42.736021. View

10.

Nousiainen K, Makela T . Measuring geometric accuracy in magnetic resonance imaging with 3D-printed phantom and nonrigid image registration. MAGMA. 2019; 33(3):401-410. PMC: 7230057. DOI: 10.1007/s10334-019-00788-6. View

11.

Dastidar A, Rodrigues J, Baritussio A, Bucciarelli-Ducci C . MRI in the assessment of ischaemic heart disease. Heart. 2016; 102(3):239-52. DOI: 10.1136/heartjnl-2014-306963. View

12.

Grubert J, Itoh Y, Moser K, Swan J . A Survey of Calibration Methods for Optical See-Through Head-Mounted Displays. IEEE Trans Vis Comput Graph. 2017; 24(9):2649-2662. DOI: 10.1109/TVCG.2017.2754257. View

13.

Condino S, Carbone M, Piazza R, Ferrari M, Ferrari V . Perceptual Limits of Optical See-Through Visors for Augmented Reality Guidance of Manual Tasks. IEEE Trans Biomed Eng. 2019; 67(2):411-419. DOI: 10.1109/TBME.2019.2914517. View

14.

Qian L, Song T, Unberath M, Kazanzides P . AR-Loupe: Magnified Augmented Reality by Combining an Optical See-Through Head-Mounted Display and a Loupe. IEEE Trans Vis Comput Graph. 2020; 28(7):2550-2562. DOI: 10.1109/TVCG.2020.3037284. View

15.

Linte C, White J, Eagleson R, Guiraudon G, Peters T . Virtual and augmented medical imaging environments: enabling technology for minimally invasive cardiac interventional guidance. IEEE Rev Biomed Eng. 2012; 3:25-47. DOI: 10.1109/RBME.2010.2082522. View

16.

Kim Y, Kim H, Kim Y . Virtual Reality and Augmented Reality in Plastic Surgery: A Review. Arch Plast Surg. 2017; 44(3):179-187. PMC: 5447526. DOI: 10.5999/aps.2017.44.3.179. View

17.

Kramida G . Resolving the Vergence-Accommodation Conflict in Head-Mounted Displays. IEEE Trans Vis Comput Graph. 2015; 22(7):1912-31. DOI: 10.1109/TVCG.2015.2473855. View

18.

Lemma M, Mangini A, Redaelli A, Acocella F . Do cardiac stabilizers really stabilize? Experimental quantitative analysis of mechanical stabilization. Interact Cardiovasc Thorac Surg. 2007; 4(3):222-6. DOI: 10.1510/icvts.2004.098517. View

19.

Rahman R, Wood M, Qian L, Price C, Johnson A, Osgood G . Head-Mounted Display Use in Surgery: A Systematic Review. Surg Innov. 2019; 27(1):88-100. DOI: 10.1177/1553350619871787. View

20.

Cleland J, Teerlink J, Senior R, Nifontov E, Mc Murray J, Lang C . The effects of the cardiac myosin activator, omecamtiv mecarbil, on cardiac function in systolic heart failure: a double-blind, placebo-controlled, crossover, dose-ranging phase 2 trial. Lancet. 2011; 378(9792):676-83. DOI: 10.1016/S0140-6736(11)61126-4. View