Interreader Scoring Variability in an Observer Study Using Dual-modality Imaging for Breast Cancer Detection in Women with Dense Breasts

Overview

Journal Acad Radiol

Specialty Radiology

Date 2013 Apr 23

PMID 23601952

Citations 5

Authors

Karen Drukker

Karla J Horsch

Lorenzo L Pesce

Maryellen L Giger

Affiliations

Soon will be listed here.

Abstract

Rationale And Objectives: To evaluate variability in the clinical assessment of breast images, we evaluated scoring behavior of radiologists in a retrospective reader study combining x-ray mammography (XRM) and three-dimensional automated breast ultrasound (ABUS) for breast cancer detection in women with dense breasts.

Methods: The study involved 17 breast radiologists in a sequential study design with readers first interpreting XRM-alone followed by an interpretation of combined XRM + ABUS. Each interpretation included a forced Breast Imaging Reporting and Data System scale and a likelihood that the woman had breast cancer. The analysis included 164 asymptomatic patients, including 31 breast cancer patients, with dense breasts and a negative screening XRM. Of interest were interreader scoring variability for XRM-alone, XRM + ABUS, and the sequential effect. In addition, a simulated double reading by pairs of readers of XRM + ABUS was investigated. Performance analysis included receiver operating characteristic analysis, percentile analysis, and κ statistics. Bootstrapping was used to determine statistical significance.

Results: The median change in area under the receiver operating characteristic curve after ABUS interpretation was 0.12 (range 0.04-0.19). Reader agreement was fair with the median interreader κ being 0.26 (0.05-0.48) for XRM-alone and 0.34 (0.11-0.55) for XRM + ABUS (95% confidence interval for the difference in κ, 0.06-0.11). Simulated double reading of XRM + ABUS demonstrated tradeoffs in sensitivity and specificity, but conservative simulated double reading resulted in a significant improvement in both sensitivity (16.7%) and specificity (7.6%) with respect to XRM-alone.

Conclusion: A modest, but statistically significant, increase in interreader agreement was observed after interpretation of ABUS.

Citing Articles

Repeat Breast Ultrasound Demonstrates Utility with Added Cancer Detection in Patients following Breast Imaging Second Opinion Recommendations.

Weinfurtner R, Mallory M, Bermudez D Breast J. 2022; 2022:1561455.

PMID: 35711880 PMC: 9187284. DOI: 10.1155/2022/1561455.

Segmentation of breast masses on dedicated breast computed tomography and three-dimensional breast ultrasound images.

Kuo H, Giger M, Reiser I, Drukker K, Boone J, Lindfors K J Med Imaging (Bellingham). 2020; 1(1):014501.

PMID: 32855995 PMC: 7439764. DOI: 10.1117/1.JMI.1.1.014501.

Foundation and methodologies in computer-aided diagnosis systems for breast cancer detection.

Jalalian A, Mashohor S, Mahmud R, Karasfi B, Saripan M, Ramli A EXCLI J. 2017; 16:113-137.

PMID: 28435432 PMC: 5379115. DOI: 10.17179/excli2016-701.

Impact of lesion segmentation metrics on computer-aided diagnosis/detection in breast computed tomography.

Kuo H, Giger M, Reiser I, Drukker K, Boone J, Lindfors K J Med Imaging (Bellingham). 2015; 1(3):031012.

PMID: 26158052 PMC: 4478751. DOI: 10.1117/1.JMI.1.3.031012.

Computerized detection of breast cancer on automated breast ultrasound imaging of women with dense breasts.

Drukker K, Sennett C, Giger M Med Phys. 2014; 41(1):012901.

PMID: 24387528 PMC: 3874062. DOI: 10.1118/1.4837196.

References

Pesce L, Metz C . Reliable and computationally efficient maximum-likelihood estimation of "proper" binormal ROC curves. Acad Radiol. 2007; 14(7):814-29. PMC: 2693394. DOI: 10.1016/j.acra.2007.03.012. View

Roe C, Metz C . Dorfman-Berbaum-Metz method for statistical analysis of multireader, multimodality receiver operating characteristic data: validation with computer simulation. Acad Radiol. 1997; 4(4):298-303. DOI: 10.1016/s1076-6332(97)80032-3. View

Waldmann A, Kapsimalakou S, Katalinic A, Grande-Nagel I, Stoeckelhuber B, Fischer D . Benefits of the quality assured double and arbitration reading of mammograms in the early diagnosis of breast cancer in symptomatic women. Eur Radiol. 2011; 22(5):1014-22. DOI: 10.1007/s00330-011-2334-9. View

Berg W, Blume J, Cormack J, Mendelson E, Lehrer D, Bohm-Velez M . Combined screening with ultrasound and mammography vs mammography alone in women at elevated risk of breast cancer. JAMA. 2008; 299(18):2151-63. PMC: 2718688. DOI: 10.1001/jama.299.18.2151. View

Martin L, Melnichouk O, Guo H, Chiarelli A, Gregory Hislop T, Yaffe M . Family history, mammographic density, and risk of breast cancer. Cancer Epidemiol Biomarkers Prev. 2010; 19(2):456-63. DOI: 10.1158/1055-9965.EPI-09-0881. View

Boyd N, Guo H, Martin L, Sun L, Stone J, Fishell E . Mammographic density and the risk and detection of breast cancer. N Engl J Med. 2007; 356(3):227-36. DOI: 10.1056/NEJMoa062790. View

Hillis S, Berbaum K, Metz C . Recent developments in the Dorfman-Berbaum-Metz procedure for multireader ROC study analysis. Acad Radiol. 2008; 15(5):647-61. PMC: 2737462. DOI: 10.1016/j.acra.2007.12.015. View

Corsetti V, Ferrari A, Ghirardi M, BERGONZINI R, Bellarosa S, Angelini O . Role of ultrasonography in detecting mammographically occult breast carcinoma in women with dense breasts. Radiol Med. 2006; 111(3):440-8. DOI: 10.1007/s11547-006-0040-5. View

Corsetti V, Houssami N, Ghirardi M, Ferrari A, Speziani M, Bellarosa S . Evidence of the effect of adjunct ultrasound screening in women with mammography-negative dense breasts: interval breast cancers at 1 year follow-up. Eur J Cancer. 2011; 47(7):1021-6. DOI: 10.1016/j.ejca.2010.12.002. View

10.

Samuelson F, Gallas B, Myers K, Petrick N, Pinsky P, Sahiner B . The importance of ROC data. Acad Radiol. 2011; 18(2):257-8. DOI: 10.1016/j.acra.2010.10.016. View

11.

Bock K, Borisch B, Cawson J, Damtjernhaug B, de Wolf C, Dean P . Effect of population-based screening on breast cancer mortality. Lancet. 2011; 378(9805):1775-6. DOI: 10.1016/S0140-6736(11)61766-2. View

12.

Ghosh K, Brandt K, Sellers T, Reynolds C, Scott C, Maloney S . Association of mammographic density with the pathology of subsequent breast cancer among postmenopausal women. Cancer Epidemiol Biomarkers Prev. 2008; 17(4):872-9. PMC: 2705947. DOI: 10.1158/1055-9965.EPI-07-0559. View

13.

Beam C, Sullivan D, Layde P . Effect of human variability on independent double reading in screening mammography. Acad Radiol. 1996; 3(11):891-7. DOI: 10.1016/s1076-6332(96)80296-0. View

14.

Boyd N, Martin L, Yaffe M, Minkin S . Mammographic density and breast cancer risk: current understanding and future prospects. Breast Cancer Res. 2011; 13(6):223. PMC: 3326547. DOI: 10.1186/bcr2942. View

15.

Berry C . The kappa statistic. JAMA. 1992; 268(18):2513-4. DOI: 10.1001/jama.268.18.2513. View

16.

Duijm L, Groenewoud J, Hendriks J, de Koning H . Independent double reading of screening mammograms in The Netherlands: effect of arbitration following reader disagreements. Radiology. 2004; 231(2):564-70. DOI: 10.1148/radiol.2312030665. View

17.

Corsetti V, Houssami N, Ferrari A, Ghirardi M, Bellarosa S, Angelini O . Breast screening with ultrasound in women with mammography-negative dense breasts: evidence on incremental cancer detection and false positives, and associated cost. Eur J Cancer. 2008; 44(4):539-44. DOI: 10.1016/j.ejca.2008.01.009. View

18.

Mandelson M, Oestreicher N, Porter P, White D, Finder C, Taplin S . Breast density as a predictor of mammographic detection: comparison of interval- and screen-detected cancers. J Natl Cancer Inst. 2000; 92(13):1081-7. DOI: 10.1093/jnci/92.13.1081. View

19.

Tabar L, Fagerberg C, Gad A, Baldetorp L, Holmberg L, GRONTOFT O . Reduction in mortality from breast cancer after mass screening with mammography. Randomised trial from the Breast Cancer Screening Working Group of the Swedish National Board of Health and Welfare. Lancet. 1985; 1(8433):829-32. DOI: 10.1016/s0140-6736(85)92204-4. View

20.

Gruszauskas N, Drukker K, Giger M, Sennett C, Pesce L . Performance of breast ultrasound computer-aided diagnosis: dependence on image selection. Acad Radiol. 2008; 15(10):1234-45. PMC: 2567418. DOI: 10.1016/j.acra.2008.04.016. View