Comparison of Inter-Method Agreement and Reliability for Automatic Brain Volumetry Using Three Different Clinically Available Software Packages

Overview

Journal Medicina (Kaunas)

Publisher MDPI

Specialty General Medicine

Date 2024 May 25

PMID 38792912

Authors

Kwang Ho Choi

Young Jin Heo

Hye Jin Baek

Jun-Ho Kim

Jeong Yoon Jang

Affiliations

Soon will be listed here.

Abstract

No comparative study has evaluated the inter-method agreement and reliability between Heuron AD and other clinically available brain volumetric software packages. Hence, we aimed to investigate the inter-method agreement and reliability of three clinically available brain volumetric software packages: FreeSurfer (FS), NeuroQuant (NQ), and Heuron AD (HAD). In this study, we retrospectively included 78 patients who underwent conventional three-dimensional (3D) T1-weighed imaging (T1WI) to evaluate their memory impairment, including 21 with normal objective cognitive function, 24 with mild cognitive impairment, and 33 with Alzheimer's disease (AD). All 3D T1WI scans were analyzed using three different volumetric software packages. Repeated-measures analysis of variance, intraclass correlation coefficient, effect size measurements, and Bland-Altman analysis were used to evaluate the inter-method agreement and reliability. The measured volumes demonstrated substantial to almost perfect agreement for most brain regions bilaterally, except for the bilateral globi pallidi. However, the volumes measured using the three software packages showed significant mean differences for most brain regions, with consistent systematic biases and wide limits of agreement in the Bland-Altman analyses. The pallidum showed the largest effect size in the comparisons between NQ and FS (5.20-6.93) and between NQ and HAD (2.01-6.17), while the cortical gray matter showed the largest effect size in the comparisons between FS and HAD (0.79-1.91). These differences and variations between the software packages were also observed in the subset analyses of 45 patients without AD and 33 patients with AD. Despite their favorable reliability, the software-based brain volume measurements showed significant differences and systematic biases in most regions. Thus, these volumetric measurements should be interpreted based on the type of volumetric software used, particularly for smaller structures. Moreover, users should consider the replaceability-related limitations when using these packages in real-world practice.

References

Yim Y, Lee J, Oh S, Chung M, Park J, Moon Y . Comparison of Automated Brain Volume Measures by NeuroQuant vs. Freesurfer in Patients with Mild Cognitive Impairment: Effect of Slice Thickness. Yonsei Med J. 2021; 62(3):255-261. PMC: 7934099. DOI: 10.3349/ymj.2021.62.3.255. View

Ochs A, Ross D, Zannoni M, Abildskov T, Bigler E . Comparison of Automated Brain Volume Measures obtained with NeuroQuant and FreeSurfer. J Neuroimaging. 2015; 25(5):721-7. DOI: 10.1111/jon.12229. View

Kanda T, Nakai Y, Aoki S, Oba H, Toyoda K, Kitajima K . Contribution of metals to brain MR signal intensity: review articles. Jpn J Radiol. 2016; 34(4):258-66. DOI: 10.1007/s11604-016-0532-8. View

Heo Y, Baek H, Skare S, Lee H, Kim D, Kim J . Automated Brain Volumetry in Patients With Memory Impairment: Comparison of Conventional and Ultrafast 3D T1-Weighted MRI Sequences Using Two Software Packages. AJR Am J Roentgenol. 2022; 218(6):1062-1073. DOI: 10.2214/AJR.21.27043. View

Song H, Lee S, Jo S, Chang S, Lim Y, Yoo Y . Agreement and Reliability between Clinically Available Software Programs in Measuring Volumes and Normative Percentiles of Segmented Brain Regions. Korean J Radiol. 2022; 23(10):959-975. PMC: 9523231. DOI: 10.3348/kjr.2022.0067. View

Reid M, Hannemann N, York G, Ritter J, Kini J, Lewis J . Comparing Two Processing Pipelines to Measure Subcortical and Cortical Volumes in Patients with and without Mild Traumatic Brain Injury. J Neuroimaging. 2017; 27(4):365-371. DOI: 10.1111/jon.12431. View

Giorgio A, De Stefano N . Clinical use of brain volumetry. J Magn Reson Imaging. 2012; 37(1):1-14. DOI: 10.1002/jmri.23671. View

Lee J, Lee J, Oh S, Chung M, Park J, Moon Y . Evaluation of Reproducibility of Brain Volumetry between Commercial Software, Inbrain and Established Research Purpose Method, FreeSurfer. J Clin Neurol. 2021; 17(2):307-316. PMC: 8053534. DOI: 10.3988/jcn.2021.17.2.307. View

Guenette J, Stern R, Tripodis Y, Chua A, Schultz V, Sydnor V . Automated versus manual segmentation of brain region volumes in former football players. Neuroimage Clin. 2018; 18:888-896. PMC: 5988230. DOI: 10.1016/j.nicl.2018.03.026. View

10.

Klauschen F, Goldman A, Barra V, Meyer-Lindenberg A, Lundervold A . Evaluation of automated brain MR image segmentation and volumetry methods. Hum Brain Mapp. 2008; 30(4):1310-27. PMC: 6870639. DOI: 10.1002/hbm.20599. View

11.

Shin D, Heo H, Song S, Shin N, Nam Y, Yoo S . Automated assessment of the substantia nigra on susceptibility map-weighted imaging using deep convolutional neural networks for diagnosis of Idiopathic Parkinson's disease. Parkinsonism Relat Disord. 2021; 85:84-90. DOI: 10.1016/j.parkreldis.2021.03.004. View

12.

Brewer J, Magda S, Airriess C, Smith M . Fully-automated quantification of regional brain volumes for improved detection of focal atrophy in Alzheimer disease. AJNR Am J Neuroradiol. 2008; 30(3):578-80. PMC: 5947999. DOI: 10.3174/ajnr.A1402. View

13.

Cohen J . A power primer. Psychol Bull. 2009; 112(1):155-9. DOI: 10.1037//0033-2909.112.1.155. View

14.

Ross D, Ochs A, Tate D, Tokac U, Seabaugh J, Abildskov T . High correlations between MRI brain volume measurements based on NeuroQuant and FreeSurfer. Psychiatry Res Neuroimaging. 2018; 278:69-76. DOI: 10.1016/j.pscychresns.2018.05.007. View

15.

Lee J, Park J, Chung M, Oh S, Moon W . [Expert Opinions and Recommendations for the Clinical Use of Quantitative Analysis Software for MRI-Based Brain Volumetry]. Taehan Yongsang Uihakhoe Chi. 2022; 82(5):1124-1139. PMC: 9432367. DOI: 10.3348/jksr.2020.0174. View

16.

Morey R, Petty C, Xu Y, Hayes J, Wagner 2nd H, Lewis D . A comparison of automated segmentation and manual tracing for quantifying hippocampal and amygdala volumes. Neuroimage. 2009; 45(3):855-66. PMC: 2714773. DOI: 10.1016/j.neuroimage.2008.12.033. View

17.

Nestor S, Gibson E, Gao F, Kiss A, Black S . A direct morphometric comparison of five labeling protocols for multi-atlas driven automatic segmentation of the hippocampus in Alzheimer's disease. Neuroimage. 2012; 66:50-70. PMC: 3606906. DOI: 10.1016/j.neuroimage.2012.10.081. View

18.

Renvall V, Witzel T, Wald L, Polimeni J . Automatic cortical surface reconstruction of high-resolution T1 echo planar imaging data. Neuroimage. 2016; 134:338-354. PMC: 5234090. DOI: 10.1016/j.neuroimage.2016.04.004. View

19.

Sherfey J, Soplata A, Ardid S, Roberts E, Stanley D, Pittman-Polletta B . DynaSim: A MATLAB Toolbox for Neural Modeling and Simulation. Front Neuroinform. 2018; 12:10. PMC: 5862864. DOI: 10.3389/fninf.2018.00010. View

20.

Ross D, Seabaugh J, Seabaugh J, Barcelona J, Seabaugh D, Wright K . Updated Review of the Evidence Supporting the Medical and Legal Use of NeuroQuant and NeuroGage in Patients With Traumatic Brain Injury. Front Hum Neurosci. 2022; 16:715807. PMC: 9027332. DOI: 10.3389/fnhum.2022.715807. View