Improved Generative Adversarial Networks Using the Total Gradient Loss for the Resolution Enhancement of Fluorescence Images

Overview

Journal Biomed Opt Express

Specialty Radiology

Date 2019 Oct 1

PMID 31565522

Citations 3

Authors

Chong Zhang

Kun Wang

Yu An

Kunshan He

Tong Tong

Jie Tian

Affiliations

Soon will be listed here.

Abstract

Because of the optical properties of medical fluorescence images (FIs) and hardware limitations, light scattering and diffraction constrain the image quality and resolution. In contrast to device-based approaches, we developed a post-processing method for FI resolution enhancement by employing improved generative adversarial networks. To overcome the drawback of fake texture generation, we proposed total gradient loss for network training. Fine-tuning training procedure was applied to further improve the network architecture. Finally, a more agreeable network for resolution enhancement was applied to actual FIs to produce sharper and clearer boundaries than in the original images.

Citing Articles

Blood perfusion assessment by near-infrared fluorescence angiography of epiploic appendages in prevention of anastomotic leakage after laparoscopic intersphincteric resection for ultra-low rectal cancer: a case-matched study.

Qiu W, Liu J, He K, Hu G, Mei S, Guan X Surg Endosc. 2024; 38(9):5446-5456.

PMID: 39090199 DOI: 10.1007/s00464-024-11085-2.

A review of the application of machine learning in molecular imaging.

Yin L, Cao Z, Wang K, Tian J, Yang X, Zhang J Ann Transl Med. 2021; 9(9):825.

PMID: 34268438 PMC: 8246214. DOI: 10.21037/atm-20-5877.

Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning.

Pradhan P, Meyer T, Vieth M, Stallmach A, Waldner M, Schmitt M Biomed Opt Express. 2021; 12(4):2280-2298.

PMID: 33996229 PMC: 8086483. DOI: 10.1364/BOE.415962.

References

Troyan S, Kianzad V, Gibbs-Strauss S, Gioux S, Matsui A, Oketokoun R . The FLARE intraoperative near-infrared fluorescence imaging system: a first-in-human clinical trial in breast cancer sentinel lymph node mapping. Ann Surg Oncol. 2009; 16(10):2943-52. PMC: 2772055. DOI: 10.1245/s10434-009-0594-2. View

Hyun H, Park M, Owens E, Wada H, Henary M, Handgraaf H . Structure-inherent targeting of near-infrared fluorophores for parathyroid and thyroid gland imaging. Nat Med. 2015; 21(2):192-7. PMC: 4319985. DOI: 10.1038/nm.3728. View

Nishino H, Hatano E, Seo S, Nitta T, Saito T, Nakamura M . Real-time Navigation for Liver Surgery Using Projection Mapping With Indocyanine Green Fluorescence: Development of the Novel Medical Imaging Projection System. Ann Surg. 2017; 267(6):1134-1140. DOI: 10.1097/SLA.0000000000002172. View

Tipirneni K, Warram J, Moore L, Prince A, de Boer E, Jani A . Oncologic Procedures Amenable to Fluorescence-guided Surgery. Ann Surg. 2017; 266(1):36-47. PMC: 6280663. DOI: 10.1097/SLA.0000000000002127. View

Arici T, Dikbas S, Altunbasak Y . A histogram modification framework and its application for image contrast enhancement. IEEE Trans Image Process. 2009; 18(9):1921-35. DOI: 10.1109/TIP.2009.2021548. View

He K, Zhou J, Yang F, Chi C, Li H, Mao Y . Near-infrared Intraoperative Imaging of Thoracic Sympathetic Nerves: From Preclinical Study to Clinical Trial. Theranostics. 2018; 8(2):304-313. PMC: 5743549. DOI: 10.7150/thno.22369. View

Gioux S, Choi H, Frangioni J . Image-guided surgery using invisible near-infrared light: fundamentals of clinical translation. Mol Imaging. 2010; 9(5):237-55. PMC: 3105445. View

Vahrmeijer A, Hutteman M, van der Vorst J, van de Velde C, Frangioni J . Image-guided cancer surgery using near-infrared fluorescence. Nat Rev Clin Oncol. 2013; 10(9):507-18. PMC: 3755013. DOI: 10.1038/nrclinonc.2013.123. View

Jang B, Park J, Tung C, Kim I, Choi Y . Gold nanorod-photosensitizer complex for near-infrared fluorescence imaging and photodynamic/photothermal therapy in vivo. ACS Nano. 2011; 5(2):1086-94. DOI: 10.1021/nn102722z. View

10.

Leblond F, Davis S, Valdes P, Pogue B . Pre-clinical whole-body fluorescence imaging: Review of instruments, methods and applications. J Photochem Photobiol B. 2009; 98(1):77-94. PMC: 3678966. DOI: 10.1016/j.jphotobiol.2009.11.007. View

11.

Soltesz E, Kim S, Laurence R, Degrand A, Parungo C, Dor D . Intraoperative sentinel lymph node mapping of the lung using near-infrared fluorescent quantum dots. Ann Thorac Surg. 2004; 79(1):269-77. PMC: 1421510. DOI: 10.1016/j.athoracsur.2004.06.055. View

12.

van Oosterom M, den Houting D, van de Velde C, Leeuwen F . Navigating surgical fluorescence cameras using near-infrared optical tracking. J Biomed Opt. 2018; 23(5):1-10. DOI: 10.1117/1.JBO.23.5.056003. View

13.

Ashitate Y, Stockdale A, Choi H, Laurence R, Frangioni J . Real-time simultaneous near-infrared fluorescence imaging of bile duct and arterial anatomy. J Surg Res. 2011; 176(1):7-13. PMC: 3212656. DOI: 10.1016/j.jss.2011.06.027. View

14.

Dong C, Loy C, He K, Tang X . Image Super-Resolution Using Deep Convolutional Networks. IEEE Trans Pattern Anal Mach Intell. 2016; 38(2):295-307. DOI: 10.1109/TPAMI.2015.2439281. View

15.

Li Y, Xu F, Zhang F, Xu P, Zhang M, Fan M . DLBI: deep learning guided Bayesian inference for structure reconstruction of super-resolution fluorescence microscopy. Bioinformatics. 2018; 34(13):i284-i294. PMC: 6022599. DOI: 10.1093/bioinformatics/bty241. View

16.

Wang H, Rivenson Y, Jin Y, Wei Z, Gao R, Gunaydin H . Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat Methods. 2018; 16(1):103-110. PMC: 7276094. DOI: 10.1038/s41592-018-0239-0. View

17.

Frangioni J . In vivo near-infrared fluorescence imaging. Curr Opin Chem Biol. 2003; 7(5):626-34. DOI: 10.1016/j.cbpa.2003.08.007. View

18.

Zdankowski P, McGloin D, Swedlow J . Full volume super-resolution imaging of thick mitotic spindle using 3D AO STED microscope. Biomed Opt Express. 2019; 10(4):1999-2009. PMC: 6484978. DOI: 10.1364/BOE.10.001999. View

19.

DSouza A, Lin H, Henderson E, Samkoe K, Pogue B . Review of fluorescence guided surgery systems: identification of key performance capabilities beyond indocyanine green imaging. J Biomed Opt. 2016; 21(8):80901. PMC: 4985715. DOI: 10.1117/1.JBO.21.8.080901. View

20.

Wang Z, Bovik A, Sheikh H, Simoncelli E . Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process. 2004; 13(4):600-12. DOI: 10.1109/tip.2003.819861. View