- An AI bot took the test that radiologists in the UK must pass before completing their training.
- The AI candidate passed two out of 10 mock exams, while the human passed four out of 10.
- Although the bot didn’t outperform most of the humans who tested it, it was highly accurate.
- Do you already know our Instagram account? Follow us
A good artificial intelligence (AI) may be just what the doctor ordered, or at least the day before.
An AI system recently passed a radiology exam with an average accuracy of 79.5% across 10 tests, researchers in the UK found. That’s not bad, considering that the average accuracy for human radiologists is 84.8%.
researchers, who published their study in british medical journalIt wanted to determine whether an AI bot could pass the “quick report” part of the Fellowship of the Royal College of Radiologists (FRCD) exam, which radiologists in the UK must pass to complete their training.
The Quick Report section of the three-part exam requires candidates to interpret 30 radiographs in 35 minutes. To pass, candidates must report at least 90% of radiographs correctly. The AI scored over 90% on two of the 10 tests it took.
“This part of the exam is designed to ‘stress test’ candidates for speed and accuracy, providing a mix of challenging common and unusual cases commonly encountered in clinical practice by the general and emergency department are referred for radiological interpretation,” the researchers said.
AI had ‘relatively high’ accuracy, but still needs to train further to reach the skills of a qualified radiologist
The researchers used 10 simulated FRCR Rapid Report tests for the study. The AI candidate was a Smartergence tool developed by the French AI company Millview which is commercially available. The study also included 26 radiologists who had taken and passed the actual FRCR exam within the past year.
When no non-interpretable images were used, which were not recorded by the AI at all, the AI candidate had an average overall accuracy of 79.5% and was able to pass two out of 10 mock tests. Radiologists were 84.8% accurate and were able to pass four out of 10 mock exams on an average.
AI diagnosed images that radiologists typically diagnose correctly 91% of the time. For images that were misdiagnosed by most radiologists, the AI was wrong only 50% of the time; Correct diagnosis of hands, carpal bones and feet.
The AI candidate needs more training analyzing areas that are considered ‘non-descriptive’ such as the abdomen and axial skeleton.
While the AI candidate had “relatively high” accuracy, it only scored the highest on one mock test.
“The AI candidate will still need further training to achieve the same level of performance and skill as the average FRCR newly qualified radiologist,” the researchers said, adding that they still need training on “subtle musculoskeletal abnormalities.” .
now read: This is how a content creator faked a month of his life using AI
Read also: 3 predictions about practical artificial intelligence for 2023