Improving Traumatic Fracture Detection on Radiographs with Artificial Intelligence Support: a Multi-reader Study

Overview

Journal BJR Open

Publisher Oxford University Press

Specialty Radiology

Date 2024 May 17

PMID 38757067

Authors

Rikke Bachmann

Gozde Gunes

Stine Hangaard

Andreas Nexmann

Pavel Lisouski

Mikael Boesen

Michael Lundemann

Scott G Baginski

Affiliations

Soon will be listed here.

Abstract

Objectives: The aim of this study was to evaluate the diagnostic performance of nonspecialist readers with and without the use of an artificial intelligence (AI) support tool to detect traumatic fractures on radiographs of the appendicular skeleton.

Methods: The design was a retrospective, fully crossed multi-reader, multi-case study on a balanced dataset of patients (≥2 years of age) with an AI tool as a diagnostic intervention. Fifteen readers assessed 340 radiographic exams, with and without the AI tool in 2 different sessions and the time spent was automatically recorded. Reference standard was established by 3 consultant radiologists. Sensitivity, specificity, and false positives per patient were calculated.

Results: Patient-wise sensitivity increased from 72% to 80% ( < .05) and patient-wise specificity increased from 81% to 85% ( < .05) in exams aided by the AI tool compared to the unaided exams. The increase in sensitivity resulted in a relative reduction of missed fractures of 29%. The average rate of false positives per patient decreased from 0.16 to 0.14, corresponding to a relative reduction of 21%. There was no significant difference in average reading time spent per exam. The largest gain in fracture detection performance, with AI support, across all readers, was on nonobvious fractures with a significant increase in sensitivity of 11 percentage points (pp) (60%-71%).

Conclusions: The diagnostic performance for detection of traumatic fractures on radiographs of the appendicular skeleton improved among nonspecialist readers tested AI fracture detection support tool showed an overall reader improvement in sensitivity and specificity when supported by an AI tool. Improvement was seen in both sensitivity and specificity without negatively affecting the interpretation time.

Advances In Knowledge: The division and analysis of obvious and nonobvious fractures are novel in AI reader comparison studies like this.

Citing Articles

Artificial Intelligence-Empowered Radiology-Current Status and Critical Review.

Obuchowicz R, Lasek J, Wodzinski M, Piorkowski A, Strzelecki M, Nurzynska K Diagnostics (Basel). 2025; 15(3).

PMID: 39941212 PMC: 11816879. DOI: 10.3390/diagnostics15030282.

Artificial intelligence in fracture detection on radiographs: a literature review.

Lo Mastro A, Grassi E, Berritto D, Russo A, Reginelli A, Guerra E Jpn J Radiol. 2024; .

PMID: 39538068 DOI: 10.1007/s11604-024-01702-4.

References

Oppenheimer J, Luken S, Hamm B, Niehues S . A Prospective Approach to Integration of AI Fracture Detection Software in Radiographs into Clinical Workflow. Life (Basel). 2023; 13(1). PMC: 9864518. DOI: 10.3390/life13010223. View

Anderson P, Baum G, Keathley N, Sicular S, Venkatesh S, Sharma A . Deep Learning Assistance Closes the Accuracy Gap in Fracture Detection Across Clinician Types. Clin Orthop Relat Res. 2022; 481(3):580-588. PMC: 9928835. DOI: 10.1097/CORR.0000000000002385. View

Soun J, Chow D, Nagamine M, Takhtawala R, Filippi C, Yu W . Artificial Intelligence and Acute Stroke Imaging. AJNR Am J Neuroradiol. 2020; 42(1):2-11. PMC: 7814792. DOI: 10.3174/ajnr.A6883. View

Zheng Q, Yang L, Zeng B, Li J, Guo K, Liang Y . Artificial intelligence performance in detecting tumor metastasis from medical radiology imaging: A systematic review and meta-analysis. EClinicalMedicine. 2021; 31:100669. PMC: 7773591. DOI: 10.1016/j.eclinm.2020.100669. View

Hayashi D, Kompel A, Ventre J, Ducarouge A, Nguyen T, Regnard N . Automated detection of acute appendicular skeletal fractures in pediatric patients using deep learning. Skeletal Radiol. 2022; 51(11):2129-2139. DOI: 10.1007/s00256-022-04070-0. View

Do Y, Jang M, Yun B, Shin S, Kim B, Kim S . Diagnostic Performance of Artificial Intelligence-Based Computer-Aided Diagnosis for Breast Microcalcification on Mammography. Diagnostics (Basel). 2021; 11(8). PMC: 8392744. DOI: 10.3390/diagnostics11081409. View

Hall M, Kea B, Wang R . Recognising Bias in Studies of Diagnostic Tests Part 1: Patient Selection. Emerg Med J. 2019; 36(7):431-434. PMC: 6738935. DOI: 10.1136/emermed-2019-208446. View

Kranz R, Cosson P . Anatomical and/or pathological predictors for the "incorrect" classification of red dot markers on wrist radiographs taken following trauma. Br J Radiol. 2014; 88(1046):20140503. PMC: 4614237. DOI: 10.1259/bjr.20140503. View

Weinick R, Burns R, Mehrotra A . Many emergency department visits could be managed at urgent care centers and retail clinics. Health Aff (Millwood). 2010; 29(9):1630-6. PMC: 3412873. DOI: 10.1377/hlthaff.2009.0748. View

10.

Guly H . Diagnostic errors in an accident and emergency department. Emerg Med J. 2001; 18(4):263-9. PMC: 1725632. DOI: 10.1136/emj.18.4.263. View

11.

Dupuis M, Delbos L, Veil R, Adamsbaum C . External validation of a commercially available deep learning algorithm for fracture detection in children. Diagn Interv Imaging. 2021; 103(3):151-159. DOI: 10.1016/j.diii.2021.10.007. View

12.

Mattijssen-Horstink L, Langeraar J, Mauritz G, van der Stappen W, Baggelaar M, Tan E . Radiologic discrepancies in diagnosis of fractures in a Dutch teaching emergency department: a retrospective analysis. Scand J Trauma Resusc Emerg Med. 2020; 28(1):38. PMC: 7222339. DOI: 10.1186/s13049-020-00727-8. View

13.

Gergenti L, Olympia R . Etiology and disposition associated with radiology discrepancies on emergency department patients. Am J Emerg Med. 2019; 37(11):2015-2019. DOI: 10.1016/j.ajem.2019.02.027. View

14.

Bossuyt P, Reitsma J, Bruns D, Gatsonis C, Glasziou P, Irwig L . STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015; 351:h5527. PMC: 4623764. DOI: 10.1136/bmj.h5527. View

15.

Cheng C, Chen C, Cheng F, Chen H, Su Y, Yeh C . A Human-Algorithm Integration System for Hip Fracture Detection on Plain Radiography: System Development and Validation Study. JMIR Med Inform. 2020; 8(11):e19416. PMC: 7732715. DOI: 10.2196/19416. View

16.

Obuchowski N, Bullen J . Multireader Diagnostic Accuracy Imaging Studies: Fundamentals of Design and Analysis. Radiology. 2022; 303(1):26-34. DOI: 10.1148/radiol.211593. View

17.

Court-Brown C, Caesar B . Epidemiology of adult fractures: A review. Injury. 2006; 37(8):691-7. DOI: 10.1016/j.injury.2006.04.130. View

18.

Li W, Stimec J, Camp M, Pusic M, Herman J, Boutis K . Pediatric Musculoskeletal Radiographs: Anatomy and Fractures Prone to Diagnostic Error Among Emergency Physicians. J Emerg Med. 2022; 62(4):524-533. DOI: 10.1016/j.jemermed.2021.12.021. View

19.

Parikh J, Wolfman D, Bender C, Arleo E . Radiologist Burnout According to Surveyed Radiology Practice Leaders. J Am Coll Radiol. 2019; 17(1 Pt A):78-81. DOI: 10.1016/j.jacr.2019.07.008. View

20.

York T, Franklin C, Reynolds K, Munro G, Jenney H, Harland W . Reporting errors in plain radiographs for lower limb trauma-a systematic review and meta-analysis. Skeletal Radiol. 2021; 51(1):171-182. PMC: 8626392. DOI: 10.1007/s00256-021-03821-9. View