AI for Interpreting Screening Mammograms: Implications for Missed Cancer in Double Reading Practices and Challenging-to-locate Lesions

Overview

Journal Sci Rep

Specialty Science

Date 2024 May 24

PMID 38789575

Authors

Zhengqiang Jiang

Ziba Gandomkar

Phuong Dung Yun Trieu

Seyedamir Tavakoli Taba

Melissa L Barron

Sarah J Lewis

Affiliations

Soon will be listed here.

Abstract

Although the value of adding AI as a surrogate second reader in various scenarios has been investigated, it is unknown whether implementing an AI tool within double reading practice would capture additional subtle cancers missed by both radiologists who independently assessed the mammograms. This paper assesses the effectiveness of two state-of-the-art Artificial Intelligence (AI) models in detecting retrospectively-identified missed cancers within a screening program employing double reading practices. The study also explores the agreement between AI and radiologists in locating the lesions, considering various levels of concordance among the radiologists in locating the lesions. The Globally-aware Multiple Instance Classifier (GMIC) and Global-Local Activation Maps (GLAM) models were fine-tuned for our dataset. We evaluated the sensitivity of both models on missed cancers retrospectively identified by a panel of three radiologists who reviewed prior examinations of 729 cancer cases detected in a screening program with double reading practice. Two of these experts annotated the lesions, and based on their concordance levels, cases were categorized as 'almost perfect,' 'substantial,' 'moderate,' and 'poor.' We employed Similarity or Histogram Intersection (SIM) and Kullback-Leibler Divergence (KLD) metrics to compare saliency maps of malignant cases from the AI model with annotations from radiologists in each category. In total, 24.82% of cancers were labeled as "missed." The performance of GMIC and GLAM on the missed cancer cases was 82.98% and 79.79%, respectively, while for the true screen-detected cancers, the performances were 89.54% and 87.25%, respectively (p-values for the difference in sensitivity < 0.05). As anticipated, SIM and KLD from saliency maps were best in 'almost perfect,' followed by 'substantial,' 'moderate,' and 'poor.' Both GMIC and GLAM (p-values < 0.05) exhibited greater sensitivity at higher concordance. Even in a screening program with independent double reading, adding AI could potentially identify missed cancers. However, the challenging-to-locate lesions for radiologists impose a similar challenge for AI.

References

Wu N, Phang J, Park J, Shen Y, Huang Z, Zorin M . Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening. IEEE Trans Med Imaging. 2019; 39(4):1184-1194. PMC: 7427471. DOI: 10.1109/TMI.2019.2945514. View

Bylinskii Z, Judd T, Oliva A, Torralba A, Durand F . What Do Different Evaluation Metrics Tell Us About Saliency Models?. IEEE Trans Pattern Anal Mach Intell. 2018; 41(3):740-757. DOI: 10.1109/TPAMI.2018.2815601. View

Fiorica J . Breast Cancer Screening, Mammography, and Other Modalities. Clin Obstet Gynecol. 2016; 59(4):688-709. DOI: 10.1097/GRF.0000000000000246. View

Sechopoulos I, Teuwen J, Mann R . Artificial intelligence for breast cancer detection in mammography and digital breast tomosynthesis: State of the art. Semin Cancer Biol. 2020; 72:214-225. DOI: 10.1016/j.semcancer.2020.06.002. View

Akinyelu A, Zaccagna F, Grist J, Castelli M, Rundo L . Brain Tumor Diagnosis Using Machine Learning, Convolutional Neural Networks, Capsule Neural Networks and Vision Transformers, Applied to MRI: A Survey. J Imaging. 2022; 8(8). PMC: 9331677. DOI: 10.3390/jimaging8080205. View

Ribli D, Horvath A, Unger Z, Pollner P, Csabai I . Detecting and classifying lesions in mammograms with Deep Learning. Sci Rep. 2018; 8(1):4165. PMC: 5854668. DOI: 10.1038/s41598-018-22437-z. View

Siviengphanom S, Lewis S, Brennan P, Gandomkar Z . Computer-extracted global radiomic features can predict the radiologists' first impression about the abnormality of a screening mammogram. Br J Radiol. 2024; 97(1153):168-179. PMC: 11027311. DOI: 10.1093/bjr/tqad025. View

Othman E, Mahmoud M, Dhahri H, Abdulkader H, Mahmood A, Ibrahim M . Automatic Detection of Liver Cancer Using Hybrid Pre-Trained Models. Sensors (Basel). 2022; 22(14). PMC: 9322134. DOI: 10.3390/s22145429. View

Yang Z, Cao Z, Zhang Y, Tang Y, Lin X, Ouyang R . MommiNet-v2: Mammographic multi-view mass identification networks. Med Image Anal. 2021; 73:102204. DOI: 10.1016/j.media.2021.102204. View

10.

Shen L, Margolies L, Rothstein J, Fluder E, McBride R, Sieh W . Deep Learning to Improve Breast Cancer Detection on Screening Mammography. Sci Rep. 2019; 9(1):12495. PMC: 6715802. DOI: 10.1038/s41598-019-48995-4. View

11.

Jung H, Kim B, Lee I, Yoo M, Lee J, Ham S . Detection of masses in mammograms using a one-stage object detector based on a deep convolutional neural network. PLoS One. 2018; 13(9):e0203355. PMC: 6143189. DOI: 10.1371/journal.pone.0203355. View

12.

Yoon J, Strand F, Baltzer P, Conant E, Gilbert F, Lehman C . Standalone AI for Breast Cancer Detection at Screening Digital Mammography and Digital Breast Tomosynthesis: A Systematic Review and Meta-Analysis. Radiology. 2023; 307(5):e222639. PMC: 10315526. DOI: 10.1148/radiol.222639. View

13.

Chiu H, Chao H, Chen Y . Application of Artificial Intelligence in Lung Cancer. Cancers (Basel). 2022; 14(6). PMC: 8946647. DOI: 10.3390/cancers14061370. View

14.

Dembrower K, Crippa A, Colon E, Eklund M, Strand F . Artificial intelligence for breast cancer detection in screening mammography in Sweden: a prospective, population-based, paired-reader, non-inferiority study. Lancet Digit Health. 2023; 5(10):e703-e711. DOI: 10.1016/S2589-7500(23)00153-X. View

15.

Sung H, Ferlay J, Siegel R, Laversanne M, Soerjomataram I, Jemal A . Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin. 2021; 71(3):209-249. DOI: 10.3322/caac.21660. View

16.

Salim M, Dembrower K, Eklund M, Lindholm P, Strand F . Range of Radiologist Performance in a Population-based Screening Cohort of 1 Million Digital Mammography Examinations. Radiology. 2020; 297(1):33-39. DOI: 10.1148/radiol.2020192212. View

17.

Shen Y, Wu N, Phang J, Park J, Liu K, Tyagi S . An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization. Med Image Anal. 2020; 68:101908. PMC: 7828643. DOI: 10.1016/j.media.2020.101908. View

18.

Gandomkar Z, Siviengphanom S, Ekpo E, Suleiman M, Tavakoli Taba S, Li T . Global processing provides malignancy evidence complementary to the information captured by humans or machines following detailed mammogram inspection. Sci Rep. 2021; 11(1):20122. PMC: 8505651. DOI: 10.1038/s41598-021-99582-5. View

19.

Yap M, Pons G, Marti J, Ganau S, Sentis M, Zwiggelaar R . Automated Breast Ultrasound Lesions Detection Using Convolutional Neural Networks. IEEE J Biomed Health Inform. 2017; 22(4):1218-1226. DOI: 10.1109/JBHI.2017.2731873. View

20.

McKinney S, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H . International evaluation of an AI system for breast cancer screening. Nature. 2020; 577(7788):89-94. DOI: 10.1038/s41586-019-1799-6. View