CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions

Overview

Journal Proc IEEE Inst Electr Electron Eng

Date 2020 Jan 11

PMID 31920208

Citations 19

Authors

Tom Vercauteren

Mathias Unberath

Nicolas Padoy

Nassir Navab

Affiliations

Soon will be listed here.

Abstract

Data-driven computational approaches have evolved to enable extraction of information from medical images with a reliability, accuracy and speed which is already transforming their interpretation and exploitation in clinical practice. While similar benefits are longed for in the field of interventional imaging, this ambition is challenged by a much higher heterogeneity. Clinical workflows within interventional suites and operating theatres are extremely complex and typically rely on poorly integrated intra-operative devices, sensors, and support infrastructures. Taking stock of some of the most exciting developments in machine learning and artificial intelligence for computer assisted interventions, we highlight the crucial need to take context and human factors into account in order to address these challenges. Contextual artificial intelligence for computer assisted intervention, or CAI4CAI, arises as an emerging opportunity feeding into the broader field of surgical data science. Central challenges being addressed in CAI4CAI include how to integrate the ensemble of prior knowledge and instantaneous sensory information from experts, sensors and actuators; how to create and communicate a faithful and actionable shared representation of the surgery among a mixed human-AI actor team; how to design interventional systems and associated cognitive shared control schemes for online uncertainty-aware collaborative decision making ultimately producing more precise and reliable interventions.

Citing Articles

LoViT: Long Video Transformer for surgical phase recognition.

Liu Y, Boels M, Garcia-Peraza-Herrera L, Vercauteren T, Dasgupta P, Granados A Med Image Anal. 2024; 99:103366.

PMID: 39418831 PMC: 11876726. DOI: 10.1016/j.media.2024.103366.

Artificial intelligence automated surgical phases recognition in intraoperative videos of laparoscopic pancreatoduodenectomy.

You J, Cai H, Wang Y, Bian A, Cheng K, Meng L Surg Endosc. 2024; 38(9):4894-4905.

PMID: 38958719 DOI: 10.1007/s00464-024-10916-6.

SDA-CLIP: surgical visual domain adaptation using video and text labels.

Li Y, Jia S, Song G, Wang P, Jia F Quant Imaging Med Surg. 2023; 13(10):6989-7001.

PMID: 37869278 PMC: 10585553. DOI: 10.21037/qims-23-376.

Procedural Software Toolkit in the Armamentarium of Interventional Therapies: A Review of Additive Usefulness and Current Evidence.

Al-Sharydah A, BinShaiq F, Aloraifi R, Almefleh A, Alessa S, Alobud A Diagnostics (Basel). 2023; 13(4).

PMID: 36832254 PMC: 9955934. DOI: 10.3390/diagnostics13040765.

Translation of Medical AR Research into Clinical Practice.

Seibold M, Spirig J, Esfandiari H, Farshad M, Furnstahl P J Imaging. 2023; 9(2).

PMID: 36826963 PMC: 9961816. DOI: 10.3390/jimaging9020044.

References

Zia A, Sharma Y, Bettadapura V, Sarin E, Essa I . Video and accelerometer-based motion analysis for automated surgical skills assessment. Int J Comput Assist Radiol Surg. 2018; 13(3):443-455. DOI: 10.1007/s11548-018-1704-z. View

Lalys F, Jannin P . Surgical process modelling: a review. Int J Comput Assist Radiol Surg. 2013; 9(3):495-511. DOI: 10.1007/s11548-013-0940-5. View

Sarikaya D, Corso J, Guru K . Detection and Localization of Robotic Tools in Robot-Assisted Surgery Videos Using Deep Neural Networks for Region Proposal and Detection. IEEE Trans Med Imaging. 2017; 36(7):1542-1549. DOI: 10.1109/TMI.2017.2665671. View

Issenhuth T, Srivastav V, Gangi A, Padoy N . Face detection in the operating room: comparison of state-of-the-art methods and a self-supervised approach. Int J Comput Assist Radiol Surg. 2019; 14(6):1049-1058. DOI: 10.1007/s11548-019-01944-y. View

Boudissa M, Courvoisier A, Chabanas M, Tonetti J . Computer assisted surgery in preoperative planning of acetabular fracture surgery: state of the art. Expert Rev Med Devices. 2017; 15(1):81-89. DOI: 10.1080/17434440.2017.1413347. View

Quellec G, Lamard M, Cochener B, Cazuguel G . Real-time segmentation and recognition of surgical tasks in cataract surgery videos. IEEE Trans Med Imaging. 2014; 33(12):2352-60. DOI: 10.1109/TMI.2014.2340473. View

Pluim J, Fitzpatrick J . Image registration. IEEE Trans Med Imaging. 2003; 22(11):1341-3. DOI: 10.1109/tmi.2003.819272. View

Hu Y, Modat M, Gibson E, Li W, Ghavami N, Bonmati E . Weakly-supervised convolutional neural networks for multimodal image registration. Med Image Anal. 2018; 49:1-13. PMC: 6742510. DOI: 10.1016/j.media.2018.07.002. View

Luchies A, Byram B . Deep Neural Networks for Ultrasound Beamforming. IEEE Trans Med Imaging. 2018; 37(9):2010-2021. PMC: 6109603. DOI: 10.1109/TMI.2018.2809641. View

10.

Deganello A, Sellars M, Yusuf G, Sidhu P . How Much Should I Record During a CEUS Examination? Practical Aspects of the "Real-Time" Feature of a Contrast Ultrasound Study. Ultraschall Med. 2018; 39(5):484-486. DOI: 10.1055/a-0717-9582. View

11.

Twinanda A, Alkan E, Gangi A, De Mathelin M, Padoy N . Data-driven spatio-temporal RGBD feature encoding for action recognition in operating rooms. Int J Comput Assist Radiol Surg. 2015; 10(6):737-47. DOI: 10.1007/s11548-015-1186-1. View

12.

Mahmood F, Chen R, Durr N . Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training. IEEE Trans Med Imaging. 2018; 37(12):2572-2581. DOI: 10.1109/TMI.2018.2842767. View

13.

Maier-Hein L, Vedula S, Speidel S, Navab N, Kikinis R, Park A . Surgical data science for next-generation interventions. Nat Biomed Eng. 2019; 1(9):691-696. DOI: 10.1038/s41551-017-0132-7. View

14.

Wang G, Li W, Zuluaga M, Pratt R, Patel P, Aertsen M . Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning. IEEE Trans Med Imaging. 2018; 37(7):1562-1573. PMC: 6051485. DOI: 10.1109/TMI.2018.2791721. View

15.

Twinanda A, Shehata S, Mutter D, Marescaux J, De Mathelin M, Padoy N . EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos. IEEE Trans Med Imaging. 2016; 36(1):86-97. DOI: 10.1109/TMI.2016.2593957. View

16.

Ruhan Sa , Owens W, Wiegand R, Studin M, Capoferri D, Barooha K . Intervertebral disc detection in X-ray images using faster R-CNN. Annu Int Conf IEEE Eng Med Biol Soc. 2017; 2017:564-567. DOI: 10.1109/EMBC.2017.8036887. View

17.

Visentini-Scarzanella M, Sugiura T, Kaneko T, Koto S . Deep monocular 3D reconstruction for assisted navigation in bronchoscopy. Int J Comput Assist Radiol Surg. 2017; 12(7):1089-1099. DOI: 10.1007/s11548-017-1609-2. View

18.

Vakharia V, Sparks R, Li K, OKeeffe A, Miserocchi A, McEvoy A . Automated trajectory planning for laser interstitial thermal therapy in mesial temporal lobe epilepsy. Epilepsia. 2018; 59(4):814-824. PMC: 5901027. DOI: 10.1111/epi.14034. View

19.

Ravi D, Szczotka A, Shakir D, Pereira S, Vercauteren T . Effective deep learning training for single-image super-resolution in endomicroscopy exploiting video-registration-based reconstruction. Int J Comput Assist Radiol Surg. 2018; 13(6):917-924. PMC: 5973979. DOI: 10.1007/s11548-018-1764-0. View

20.

Jannin P, Morandi X . Surgical models for computer-assisted neurosurgery. Neuroimage. 2007; 37(3):783-91. DOI: 10.1016/j.neuroimage.2007.05.034. View