The pitfalls of using AI in medical imaging

AI could be a big help for healthcare professionals and researchers who need to interpret diagnostic images. A radiologist can spot fractures and other abnormalities in X-rays. Still, AI models can see things that the human eye can’t, and that’s a golden opportunity to improve medical imaging. But a study in the journal Scientific Reports reveals a hidden problem with using AI in medical imaging research — a phenomenon called “shortcut learning,” which can produce highly accurate results but can be misleading.

The researchers analyzed more than 25,000 knee X-rays from the National Institutes of Health-funded Osteoarthritis Project. The result was that the AI ​​models were able to “predict” seemingly unrelated and bizarre things, such as whether patients would eat refried beans or beer. While these predictions had nothing to do with medicine, the models were able to achieve stunning accuracy by exploiting subtle, unintended patterns in the data.
A radiologist looks at X-ray images in a modern, state-of-the-art medical lab.
Radiologists are using AI to improve medical diagnoses.

“AI has the potential to revolutionize medical imaging, but we have to be very careful,” says Dr. Peter Schilling, who led the study and is an orthopedic surgeon at Dartmouth Hitchcock Medical Center and professor of orthopedics at Dartmouth University’s Geisel School of Medicine. “These models can find patterns that humans can’t see, but not all the patterns they find are correct or reliable,” he continues. “It’s important to understand these risks to prevent misleading conclusions and keep science healthy.”

The researchers examined how AI algorithms often rely on things that make mistakes — like differences in radiology equipment or clinical symptoms — rather than on important medical features.
The challenges facing AI in medical research

Efforts to eliminate these biases have only been partially successful because AI models are simply “learning” hidden patterns in other data. “This is a much bigger issue than biases that come from race or gender,” says Brandon Hill, a co-author of the study and a machine learning scientist at Dartmouth Hitchcock University. “We found that the algorithm could even predict the year the x-ray was taken,” he continues. “That’s dangerous, because when you stop learning one thing, the algorithm goes on to something else that it wasn’t paying attention to before.” This risk can lead to false claims, and researchers need to be aware that this can happen all too easily when using this technology.

A close-up of an AI algorithm with abstract patterns and complex connections against a dark background.

AI algorithms are learning from complex medical data. The findings show how important it is to have rigorous standards of evaluation in medical research that uses AI. If we rely on standard algorithms without careful scrutiny, we could end up with incorrect diagnoses and treatments. “When we’re going to use models to find new patterns in medicine, we need to be very confident that they’re working,” says Hill. "Part of the problem is that we have our own biases," he adds. "It's so easy to think that the model looks the same way we do. But in reality, that's not the case."