Similarity and Quality Metrics for MR Image-to-image Translation

Overview

Journal Sci Rep

Date 2025 Jan 31

PMID 39890963

Authors

Melanie Dohmen

Mark A Klemens

Ivo M Baltruschat

Tuan Truong

Matthias Lenga

Affiliations

Soon will be listed here.

Abstract

Image-to-image translation can create large impact in medical imaging, as images can be synthetically transformed to other modalities, sequence types, higher resolutions or lower noise levels. To ensure patient safety, these methods should be validated by human readers, which requires a considerable amount of time and costs. Quantitative metrics can effectively complement such studies and provide reproducible and objective assessment of synthetic images. If a reference is available, the similarity of MR images is frequently evaluated by SSIM and PSNR metrics, even though these metrics are not or too sensitive regarding specific distortions. When reference images to compare with are not available, non-reference quality metrics can reliably detect specific distortions, such as blurriness. To provide an overview on distortion sensitivity, we quantitatively analyze 11 similarity (reference) and 12 quality (non-reference) metrics for assessing synthetic images. We additionally include a metric on a downstream segmentation task. We investigate the sensitivity regarding 11 kinds of distortions and typical MR artifacts, and analyze the influence of different normalization methods on each metric and distortion. Finally, we derive recommendations for effective usage of the analyzed similarity and quality metrics for evaluation of image-to-image translation models.

Citing Articles

Validation of ten federated learning strategies for multi-contrast image-to-image MRI data synthesis from heterogeneous sources.

Fiszer J, Ciupek D, Malawski M, Pieciak T bioRxiv. 2025; .

PMID: 39990397 PMC: 11844418. DOI: 10.1101/2025.02.09.637305.

References

Zhang L, Zhang L, Mou X, Zhang D . FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process. 2011; 20(8):2378-86. DOI: 10.1109/TIP.2011.2109730. View

Mallio C, Radbruch A, Deike-Hofmann K, van der Molen A, Dekkers I, Zaharchuk G . Artificial Intelligence to Reduce or Eliminate the Need for Gadolinium-Based Contrast Agents in Brain and Cardiac MRI: A Literature Review. Invest Radiol. 2023; 58(10):746-753. DOI: 10.1097/RLI.0000000000000983. View

Jayachandran Preetha C, Meredig H, Brugnara G, Mahmutoglu M, Foltyn M, Isensee F . Deep-learning-based synthesis of post-contrast T1-weighted MRI for tumour response assessment in neuro-oncology: a multicentre, retrospective cohort study. Lancet Digit Health. 2021; 3(12):e784-e794. DOI: 10.1016/S2589-7500(21)00205-3. View

Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby J . Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci Data. 2017; 4:170117. PMC: 5685212. DOI: 10.1038/sdata.2017.117. View

Khunte M, Chae A, Wang R, Jain R, Sun Y, Sollee J . Trends in clinical validation and usage of US Food and Drug Administration-cleared artificial intelligence algorithms for medical imaging. Clin Radiol. 2023; 78(2):123-129. DOI: 10.1016/j.crad.2022.09.122. View

Mittal A, Moorthy A, Bovik A . No-reference image quality assessment in the spatial domain. IEEE Trans Image Process. 2012; 21(12):4695-708. DOI: 10.1109/TIP.2012.2214050. View

Kebaili A, Lapuyade-Lahorgue J, Ruan S . Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review. J Imaging. 2023; 9(4). PMC: 10144738. DOI: 10.3390/jimaging9040081. View

Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P . Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging. 1997; 16(2):187-98. DOI: 10.1109/42.563664. View

Haase R, Pinetz T, Kobler E, Paech D, Effland A, Radbruch A . Artificial Contrast: Deep Learning for Reducing Gadolinium-Based Contrast Agents in Neuroradiology. Invest Radiol. 2023; 58(8):539-547. DOI: 10.1097/RLI.0000000000000963. View

10.

Lu J, Ofverstedt J, Lindblad J, Sladoje N . Is image-to-image translation the panacea for multimodal image registration? A comparative study. PLoS One. 2022; 17(11):e0276196. PMC: 9704666. DOI: 10.1371/journal.pone.0276196. View

11.

Yurt M, Dar S, Erdem A, Erdem E, Oguz K, Cukur T . mustGAN: multi-stream Generative Adversarial Networks for MR Image Synthesis. Med Image Anal. 2021; 70:101944. DOI: 10.1016/j.media.2020.101944. View

12.

Mason A, Rioux J, Clarke S, Costa A, Schmidt M, Keough V . Comparison of Objective Image Quality Metrics to Expert Radiologists' Scoring of Diagnostic Quality of MR Images. IEEE Trans Med Imaging. 2019; 39(4):1064-1072. DOI: 10.1109/TMI.2019.2930338. View

13.

Onofrey J, Casetti-Dinescu D, Lauritzen A, Sarkar S, Venkataraman R, Fan R . GENERALIZABLE MULTI-SITE TRAINING AND TESTING OF DEEP NEURAL NETWORKS USING IMAGE NORMALIZATION. Proc IEEE Int Symp Biomed Imaging. 2020; 2019:348-351. PMC: 7457546. DOI: 10.1109/isbi.2019.8759295. View

14.

Narvekar N, Karam L . A no-reference image blur metric based on the cumulative probability of blur detection (CPBD). IEEE Trans Image Process. 2011; 20(9):2678-83. DOI: 10.1109/TIP.2011.2131660. View

15.

Tellez D, Litjens G, Bandi P, Bulten W, Bokhorst J, Ciompi F . Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Med Image Anal. 2019; 58:101544. DOI: 10.1016/j.media.2019.101544. View

16.

Pianykh O, Pospelova K, Kamboj N . Modeling Human Perception of Image Quality. J Digit Imaging. 2018; 31(6):768-775. PMC: 6261189. DOI: 10.1007/s10278-018-0096-5. View

17.

Bonardel G, Dupont A, Decazes P, Queneau M, Modzelewski R, Coulot J . Clinical and phantom validation of a deep learning based denoising algorithm for F-18-FDG PET images from lower detection counting in comparison with the standard acquisition. EJNMMI Phys. 2022; 9(1):36. PMC: 9095795. DOI: 10.1186/s40658-022-00465-z. View

18.

Weigert M, Schmidt U, Boothe T, Muller A, Dibrov A, Jain A . Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat Methods. 2018; 15(12):1090-1097. DOI: 10.1038/s41592-018-0216-7. View

19.

McCormick M, Liu X, Jomier J, Marion C, Ibanez L . ITK: enabling reproducible research and open science. Front Neuroinform. 2014; 8:13. PMC: 3929840. DOI: 10.3389/fninf.2014.00013. View

20.

Haase R, Pinetz T, Bendella Z, Kobler E, Paech D, Block W . Reduction of Gadolinium-Based Contrast Agents in MRI Using Convolutional Neural Networks and Different Input Protocols: Limited Interchangeability of Synthesized Sequences With Original Full-Dose Images Despite Excellent Quantitative Performance. Invest Radiol. 2023; 58(6):420-430. DOI: 10.1097/RLI.0000000000000955. View