Introduction

Fractures take place in people of all age groups. The episode depends on the type of trauma, location, and associated injuries. The incidence of fractures ranges between 733 and 4017 per 100000 patient-years.^1–3 Traumatic fractures are the major cause of morbidity and mortality, and in one study, 23,917 individuals sustained 27,169 fractures, with 64.5% of the fractures occurring in women.¹ The epidemiological data for fractures and dislocations in Saudi Arabia are not available.^4,5 It is expected that the number of fractures and dislocations will increase due to population growth.

Figure 1 PRISMA flowchart Showing the Final Selection of Analyzed studies.

Figure 2 Comparison between AI Model and Clinicians for Accuracy, Sensitivity and Specificity.

The reported incidence of missed diagnosis of fractures or dislocations by plain radiographs ranges between 3% and 10%,^6–8 and this inversely affects the final outcome of the recovery. The majority of the errors take place in the emergency room, where the radiographs are wrongly elucidated as some injuries might be tenuous, and in the majority, conspicuous injuries are missed due to improper training with sub-standard techniques employed in radiological evaluation.⁹ This could be more common in the junior residents under training in the emergency room and orthopedics and traumatology. Unfortunately, this is not uncommon in trained radiologists as well. In the USA, radiologists were at the 6th position in malpractice claims,^10–14 even though they make up about 3.1% of the 892 million physicians.¹⁵ It becomes mandatory to find ways to reduce this discrepancy at both fronts at the training levels and the trained level, and one such tenet is to bring the utilization of AI in the field of diagnosis of fractures and dislocations.

AI, which is part of computer science, can perform tasks that are usually performed by humans to humans. AI requires a high level of input from different images and then can use different algorithms using machine learning, deep learning, and convolutional neural networks to extricate high-level information from the input of images.¹⁶ Recent studies have suggested convincing accuracy of diagnosis of fractures and dislocations using AI algorithms, and with the objective to assess the accuracy, sensitivity, and specificity of AI algorithms in the diagnosis of fractures using plain radiographs, this review was carried out.

Methods

We searched all related electronic databases for English language literature between January 2015 and July 2023, Pub Med, Scopus, Web of Science, Cochrane Central Ovid Medline, Ovid Embase, EBSCO Cumulative Index to Allied Health Literature, Web of Science, and Cochrane Central with keywords of Artificial Intelligence, fractures, dislocations, X-rays, radiographs, missed diagnosis. All articles that fulfilled the following inclusion criteria: primary research using validated AI algorithms for fracture detection and Only studies with a comparative study between AI algorithms and clinicians were included in the analysis. Only studies with a comparative study between AI algorithms and clinicians were included in the analysis. All other publications and data were excluded, including reports by letter to the editor, conference presentations, and systematic reviews. EndNoteTM 39 was used to tabulate the references and delete any duplicates.

Data Extraction

We extracted available information from included studies fitting our inclusion criteria. The data extracted included a number of patients/images studied, site of fractures analyzed, algorithms used, the accuracy of the report based on the algorithm, sensitivity and specificity, area under the curve (AUC), comparison between the algorithm, junior orthopedic resident, emergency physicians, and board certified radiologists.

Statistical Analysis

The diagnostic prediction of the fractures of different algorithms was analyzed using contingency tables for validation. Regression analysis was performed between the different sites of fractures and the influence of the algorithms. A p-value of <0.05 was accepted as statistically significant at a 95% confidence interval (CI). SPSS (Statistical Package for Social Sciences) Inc., which is a statistical software developed by IBM for data management, advanced analytics, multivariate analysis, and business intelligence version 29, was used.

Results

We identified 2049 studies retrieved in which 347 were duplicates, and 1651 publications were excluded due to inclusion and exclusion criteria. Fifty-one studies were reviewed in depth as they nearly fulfilled the inclusion criteria, and only 27 publications fulfilled our objectives to be analyzed in detail and were included in this study (Figure 1). Eighty-eight thousand, nine hundred and ninety-six images were analyzed for fractures (Table 1), which showed that the overall accuracy of the correct diagnosis was 90.35±6.88 (73.59–98) percent, sensitivity 90.08±8.2 (73.8–99) percent, specificity 90.16±7 (72–100) and AUC was 0.931±0.06 (0.72–0.994). The fractures analyzed were common fractures from the wrist, upper and lower limbs, and spine. All studies had internally and externally validated algorithms for Diffusion-convolutional neural networks (DCNN). The majority of the studies limited their analysis for diagnoses based on a single view of the radiograph.

Table 1 Characteristics of Studies, Number of Images Analyzed, Site of Fractures, Algorithms Used, Accuracy, Sensitivity, Specificity and Area Under Curve

Table 2 shows the analysis of 214950 images where a comparison was made between the AI algorithm versus a junior resident in training. The accuracy of the AI model was 94.24±4.19, and that of orthopedic resident was 85.18±7.01 (P value of <0.0001), with sensitivity 92.15±7.12 versus 86.38±7.6 (P<0.0001) and specificity of 93.77±4.03 versus 87.05±12.9 (P<0.0001). Yamada et al (2020) 40 compared the AI model versus orthopedic residents and board-certified radiologists and found the accuracy to be 98% versus 87% and 92% (P value of <0.0001). Figure 2 shows the comparison between the AI model and the clinician for accuracy, sensitivity, and specificity.

Table 2 Comparative Data Between the AI Models and Clinicians

Discussion

This review shows that accuracy in the diagnosis of fractures using AI algorithms surpasses that of the trained and trainee residents. Secondly, the use of AI helped the trainees and trained radiologists in improving the accuracy, sensitivity, and specificity of fracture diagnosis. In this study, the AI with different models showed that the overall accuracy of the correct diagnosis was 90.35±6.88%, sensitivity 90.08±8.2%, specificity 90.16±7 and AUC was 0.931±0.06. These results were based on plain radiographs and included all limb and vertebral fractures.

In the recent past, there has been a consequential increase in different AI models, particularly CNNs, in the arena of trauma and orthopedics. Individual models have conclusively shown that AI models are accurate in the diagnosis of fractures, which are better than junior residents and, if not better, but at par with the senior radiologist. One aspect that needs to be questioned is that most of the reported data comes from retrospective testing, and few only are based prospectively on clinical practice. The accuracy of diagnosis of fractures varied at different sites of fractures. Murphy et al (2022)⁴⁴ reported an analysis of hip fractures, comparing the AI model with two trained and expert clinicians, and found that the AI model was 19% more accurate than the physicians. Another report suggested that the sensitivity of the correct diagnosis increases by over 10%. Lindsey et al (2018)³³ reported that the physician’s average sensitivity in the diagnosis of fractures improved from 80.8% to 91.5% (95% CI, 89.3–92.9%), and specificity was 87.5% to 93.9% (95% CI, 92.9–94.9%) when they were aided with Deep convolutional neural network and added to this the physicians experienced a reduction in misreading around 47.0%. Duron et al (2021)⁴² further concurred after their review that emergency room physicians improved their results after AI assistance from 61.3% to 74.3% (up 13.0%), and the trained radiologists enhanced their diagnosis from 80.2% to 84.6% (up 4.3%). Distal radius fractures, which amount to over 20% of all fractures, were studied using an ensemble model of AI between three groups: AI, orthopedic surgeons, and radiologists, and it was reported statistically significant between the three groups. The accuracy, sensitivity, and specificity between the attending orthopedic surgeons and radiologists showed significant differences: 93.69%, 91.94%, and 95.44% compared to 92.53%, 90.44%, and 94.62%. When the physician’s groups were compared to the AI ensemble tool, it was a highly significant score of 97.75%, 97.13%, and 98.37% by the AI tool.⁴³

Missed extremity fracture diagnosis in trauma practice has always been an issue and is the second most injuries to be misdiagnosed.⁴⁵ The most common malpractice claims against radiologists involve inaccuracies in the reporting of extremity fractures.^10,46,47 Orthopaedic residents are not immune to making misinterpretations of radiographs in extremity fractures. One such study from the United Kingdom highlights that Senior Orthopaedic Residents on plain radiographs missed 4% of fractures, 7.8% made a wrong diagnosis, and 12.6%, a fracture was diagnosed when there was none.⁴⁸ Report indicates that over the years, the number of claims against orthopadicians has increased, but complaints have remained comparatively the same.⁴⁹ In the present belligerent and litigation-oriented society, it is imperative that junior orthopedic residents have all the help in making a correct fracture diagnosis and not miss even a meager injury. AI and its algorithms can never replace human doctors but can unquestionably enhance and complement in improving the accuracy of fracture diagnosis.³⁷ Moreover, adequate and timely training of trainee residents in radiographic interpretation is paramount. It was reported that junior residents till 3rd of training level are more vulnerable to making errors in radiographic interpretation.⁹

Our review has limitations due to the number of studies we have included in the analysis, as there are a number of publications that are increasing by the day, and it is possible that we have not included the most recent literature. Secondly, we could not add the data of comparative accuracy between the unaided and aided AI tools in the fracture diagnosis. Lastly, we are basing the conclusion on the retrospective studies, and there were no prospective studies to compare with. The strength of the study is we have compared a large dataset, which suggests that the different AI models are more accurate than the physicians.

In conclusion, this review highlights with unbiased evaluations recommend that the use of AI models can definitely help residents in training by increasing the accuracy of fracture diagnosis and reducing the errors in diagnosis of fractures. AI has developed cutting edge tools, which need to be further evaluated so that procurement authorities in hospitals could integrate AI into healthcare and help physicians at all levels to improve correctness in fracture diagnosis, to prevent complications of delayed diagnosis.