Published In

Authors

Abstract

Purpose: 

Missed fractures on radiographs are not an uncommon cause of diagnostic discrepancy between initial interpretation and the final read by board-certified radiologists, leading to delay in care and preventable harm to the patient. Aim of the study was to assess the effect of assistance by artificial intelligence (AI) on diagnostic performances of physicians for fractures on radiographs.

Methods and Materials: 

Our study is a retrospective diagnostic study that follows the Multiple Readers Multiple Cases methodology. The dataset was composed of 480 exams with at least 60 exams per body region (foot/ankle, knee/leg, hip/pelvis, hand/wrist, elbow/arm, shoulder/clavicle, rib cage, thoracolumbar spine) with a prevalence of fractures set at 50%; 25% obvious and 25% non-obvious. The ground truth was determined by two expert musculoskeletal radiologists, with discrepancies solved by a third. Twenty-four readers of diverse specialties (radiologists, orthopedists, emergency medicine physicians and physician assistants, rheumatologists, family physicians) and level of experience were presented, in random order, the whole validation dataset of 480 cases, with and without AI assistance, with a minimum washout period of one month. The primary analysis had to demonstrate superiority of sensitivity by patient (SEPW) and the non-inferiority of specificity by patient (SPEPW) at −3% margin with AI aid.

Results: 

A total of 480 patients were included (327 women, mean (SD) age 58.3 (17.9) years). The SEPW was 10.4% higher (95% confidence interval (CI): 6.9 to 13.9%, p<.0001 for superiority) with AI aid (75.2%) than without AI (64.8%). The SPEPW with AI aid (95.6%) was non-inferior to that without AI aid (90.6%), with a difference of +5.0% (95%CI: +2.0 to +8.0%, p<.0001 for non-inferiority). The AI shortened the average reading time by 6.3 seconds per examination (95% CI: 0.1 to 12.5, p=.046). The SEPW gain was significant in all regions but shoulder/clavicle and thoracolumbar spine.

Conclusions: 

Radiographic AI assistance improves both sensitivity and specificity of fracture detection by radiologists and non-radiologists of variable expertise involving various anatomical locations. It also slightly reduces the time needed to interpret radiographs.

Clinical Relevance/Application: 

Artificial intelligence assistance for searching skeletal fractures on radiographs improves the sensitivity and specificity of readers and shortens their reading time. We used an external multicenter dataset from the United States including multi-vendor radiographic acquisition systems that were not related to the development set originating from Europe, providing the robust generalization capacity of the model.