Manual Prostate MRI Segmentation by Readers with Different Experience: a Study of the Learning Progress

Overview

Journal Eur Radiol

Specialty Radiology

Date 2024 Jan 2

PMID 38165432

Authors

Fredrik Langkilde

Patrick Masaba

Lars Edenbrandt

Magnus Gren

Airin Halil

Mikael Hellstrom

Mans Larsson

Ameer Ali Naeem

Jonas Wallstrom

Stephan E Maier

Fredrik Jaderling

Affiliations

Soon will be listed here.

Abstract

Objective: To evaluate the learning progress of less experienced readers in prostate MRI segmentation.

Materials And Methods: One hundred bi-parametric prostate MRI scans were retrospectively selected from the Göteborg Prostate Cancer Screening 2 Trial (single center). Nine readers with varying degrees of segmentation experience were involved: one expert radiologist, two experienced radiology residents, two inexperienced radiology residents, and four novices. The task was to segment the whole prostate gland. The expert's segmentations were used as reference. For all other readers except three novices, the 100 MRI scans were divided into five rounds (cases 1-10, 11-25, 26-50, 51-76, 76-100). Three novices segmented only 50 cases (three rounds). After each round, a one-on-one feedback session between the expert and the reader was held, with feedback on systematic errors and potential improvements for the next round. Dice similarity coefficient (DSC) > 0.8 was considered accurate.

Results: Using DSC > 0.8 as the threshold, the novices had a total of 194 accurate segmentations out of 250 (77.6%). The residents had a total of 397/400 (99.2%) accurate segmentations. In round 1, the novices had 19/40 (47.5%) accurate segmentations, in round 2 41/60 (68.3%), and in round 3 84/100 (84.0%) indicating learning progress.

Conclusions: Radiology residents, regardless of prior experience, showed high segmentation accuracy. Novices showed larger interindividual variation and lower segmentation accuracy than radiology residents. To prepare datasets for artificial intelligence (AI) development, employing radiology residents seems safe and provides a good balance between cost-effectiveness and segmentation accuracy. Employing novices should only be considered on an individual basis.

Clinical Relevance Statement: Employing radiology residents for prostate MRI segmentation seems safe and can potentially reduce the workload of expert radiologists. Employing novices should only be considered on an individual basis.

Key Points: • Using less experienced readers for prostate MRI segmentation is cost-effective but may reduce quality. • Radiology residents provided high accuracy segmentations while novices showed large inter-reader variability. • To prepare datasets for AI development, employing radiology residents seems safe and might provide a good balance between cost-effectiveness and segmentation accuracy while novices should only be employed on an individual basis.

Citing Articles

Impact of uncertainty quantification through conformal prediction on volume assessment from deep learning-based MRI prostate segmentation.

Gade M, Nguyen K, Gedde S, Fernandez-Quilez A Insights Imaging. 2024; 15(1):286.

PMID: 39613981 PMC: 11607187. DOI: 10.1186/s13244-024-01863-w.

MRI radiomics based on deep learning automated segmentation to predict early recurrence of hepatocellular carcinoma.

Wei H, Zheng T, Zhang X, Wu Y, Chen Y, Zheng C Insights Imaging. 2024; 15(1):120.

PMID: 38763975 PMC: 11102894. DOI: 10.1186/s13244-024-01679-8.

References

Ahmed H, Brown L, Kaplan R, Parker C, Emberton M . Diagnostic accuracy of the PROMIS study - Authors' reply. Lancet. 2017; 390(10092):362. DOI: 10.1016/S0140-6736(17)31595-7. View

Turkbey B, Brown A, Sankineni S, Wood B, Pinto P, Choyke P . Multiparametric prostate magnetic resonance imaging in the evaluation of prostate cancer. CA Cancer J Clin. 2015; 66(4):326-36. PMC: 7511979. DOI: 10.3322/caac.21333. View

Gaffney C, Cai P, Li D, Margolis D, Sedrakyan A, Hu J . Increasing Utilization of MRI Before Prostate Biopsy in Black and Non-Black Men: An Analysis of the SEER-Medicare Cohort. AJR Am J Roentgenol. 2021; 217(2):389-394. DOI: 10.2214/AJR.20.23462. View

Giganti F, Rosenkrantz A, Villeirs G, Panebianco V, Stabile A, Emberton M . The Evolution of MRI of the Prostate: The Past, the Present, and the Future. AJR Am J Roentgenol. 2019; 213(2):384-396. DOI: 10.2214/AJR.18.20796. View

Sun Y, Reynolds H, Parameswaran B, Wraith D, Finnegan M, Williams S . Multiparametric MRI and radiomics in prostate cancer: a review. Australas Phys Eng Sci Med. 2019; 42(1):3-25. DOI: 10.1007/s13246-019-00730-z. View

Klingebiel M, Arsov C, Ullrich T, Quentin M, Al-Monajjed R, Mally D . Reasons for missing clinically significant prostate cancer by targeted magnetic resonance imaging/ultrasound fusion-guided biopsy. Eur J Radiol. 2021; 137:109587. DOI: 10.1016/j.ejrad.2021.109587. View

Moller A, Olsson H, Gronberg H, Eklund M, Aly M, Nordstrom T . The Stockholm3 blood-test predicts clinically-significant cancer on biopsy: independent validation in a multi-center community cohort. Prostate Cancer Prostatic Dis. 2018; 22(1):137-142. DOI: 10.1038/s41391-018-0082-5. View

Litjens G, Toth R, Van de Ven W, Hoeks C, Kerkstra S, van Ginneken B . Evaluation of prostate segmentation algorithms for MRI: the PROMISE12 challenge. Med Image Anal. 2014; 18(2):359-73. PMC: 4137968. DOI: 10.1016/j.media.2013.12.002. View

Shahedi M, Cool D, Bauman G, Bastian-Jordan M, Fenster A, Ward A . Accuracy Validation of an Automated Method for Prostate Segmentation in Magnetic Resonance Imaging. J Digit Imaging. 2017; 30(6):782-795. PMC: 5681466. DOI: 10.1007/s10278-017-9964-7. View

10.

Shahedi M, Cool D, Romagnoli C, Bauman G, Bastian-Jordan M, Gibson E . Spatially varying accuracy and reproducibility of prostate segmentation in magnetic resonance images using manual and semiautomated methods. Med Phys. 2014; 41(11):113503. DOI: 10.1118/1.4899182. View

11.

Dornfeld K, Hopkins S, Simmons J, Spitz D, Menda Y, Graham M . Posttreatment FDG-PET uptake in the supraglottic and glottic larynx correlates with decreased quality of life after chemoradiotherapy. Int J Radiat Oncol Biol Phys. 2008; 71(2):386-92. PMC: 2699674. DOI: 10.1016/j.ijrobp.2007.09.052. View

12.

Burel J, El Ayoubi M, Baste J, Garnier M, Montagne F, Dacher J . Surgery for lung cancer: postoperative changes and complications-what the Radiologist needs to know. Insights Imaging. 2021; 12(1):116. PMC: 8360261. DOI: 10.1186/s13244-021-01047-w. View

13.

Becker A, Chaitanya K, Schawkat K, Muehlematter U, Hotker A, Konukoglu E . Variability of manual segmentation of the prostate in axial T2-weighted MRI: A multi-reader study. Eur J Radiol. 2019; 121:108716. DOI: 10.1016/j.ejrad.2019.108716. View

14.

Cuocolo R, Cipullo M, Stanzione A, Romeo V, Green R, Cantoni V . Machine learning for the identification of clinically significant prostate cancer on MRI: a meta-analysis. Eur Radiol. 2020; 30(12):6877-6887. DOI: 10.1007/s00330-020-07027-w. View

15.

Belue M, Turkbey B . Tasks for artificial intelligence in prostate MRI. Eur Radiol Exp. 2022; 6(1):33. PMC: 9339059. DOI: 10.1186/s41747-022-00287-9. View

16.

Binder N, Dette H, Franz J, Zoller D, Suarez-Ibarrola R, Gratzke C . Corrigendum to "Data Mining in Urology: Understanding Real-world Treatment Pathways for Lower Urinary Tract Systems via Exploration of Big Data" [Eur Urol Focus 2022;8:391-3]. Eur Urol Focus. 2022; 9(1):219. DOI: 10.1016/j.euf.2022.07.001. View

17.

Jameson M, Holloway L, Vial P, Vinod S, Metcalfe P . A review of methods of analysis in contouring studies for radiation oncology. J Med Imaging Radiat Oncol. 2010; 54(5):401-10. DOI: 10.1111/j.1754-9485.2010.02192.x. View

18.

Kohestani K, Mansson M, Arnsrud Godtman R, Stranne J, Wallstrom J, Carlsson S . The GÖTEBORG prostate cancer screening 2 trial: a prospective, randomised, population-based prostate cancer screening trial with prostate-specific antigen testing followed by magnetic resonance imaging of the prostate. Scand J Urol. 2021; 55(2):116-124. PMC: 8376217. DOI: 10.1080/21681805.2021.1881612. View

19.

Tragardh E, Borrelli P, Kaboteh R, Gillberg T, Ulen J, Enqvist O . RECOMIA-a cloud-based platform for artificial intelligence research in nuclear medicine and radiology. EJNMMI Phys. 2020; 7(1):51. PMC: 7403290. DOI: 10.1186/s40658-020-00316-9. View

20.

Taha A, Hanbury A . Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging. 2015; 15:29. PMC: 4533825. DOI: 10.1186/s12880-015-0068-x. View