Responsible Steps to Implementing AI in Breast Screening
AI interpretation of mammography may be the ultimate assistant for radiologists
During Women's Health Month, RSNA News highlights the pivotal role radiology plays in support of women's health. This is the first of three stories. Read parts two and three.
Radiology leads the pack in AI development and clinical trials for breast screening assistance, but the move to clinical practice should be mindful of patient safety, reader variability and national regulatory pathways.
“There is so much great research on AI and breast imaging to share, but in widespread routine clinical implementation of AI, we are still relatively early in the process,” said Constance D. Lehman, MD, PhD, professor of radiology at Harvard Medical School and co-director of the Breast Imaging Research Center at Massachusetts General Hospital in Boston.
Results from reader studies of AI are unlikely to translate directly into community practices, according to Dr. Lehman, since reader studies are by design somewhat artificial, using collections of mammograms enriched with exams with cancers and typically read by radiologists with experience in reading with and without computer assistance.
In the “real world” of community practices, cancer rates in screening mammography programs are much lower than in reader studies and there is variation in how radiologists apply AI tools in their interpretations of mammograms.
In addition, mammographic readings vary widely—among radiologists across the globe and even within the same clinic. In many European studies, recall rates for screening mammograms may be as low as 2% to 5%.
“In the U.S., we’re very comfortable with our recall rates of 10% to 15%,” she said. “So, the implementation of innovative technologies, such as AI, is going to be different with these global variations. Nevertheless, we can also leverage our differences as opportunities to learn from each other.”
Approved Technologies as a Guide
Radiologists are in an excellent position to advance AI-applied mammographic interpretation, according to Dr. Lehman, because of the lessons learned through CAD.
In 2015, she and her colleagues in the Breast Cancer Surveillance Consortium published a study demonstrating that, although the U.S. Food and Drug Administration (FDA) had long since cleared CAD for clinical use, the reality was that CAD didn’t seem to improve radiologists’ interpretations of mammograms in routine practice. There were neither differences in cancer detection rates nor, performance improvements across readers. In fact, CAD decreased sensitivity in the subset of radiologists who interpreted studies with and without it.
These findings allowed the industry to revisit and refine CAD systems and software, exploring advancements like deep learning technologies and employing validation techniques that reflect real-world screening practices.
There is so much great research on AI and breast imaging to share, but in widespread routine clinical implementation of AI, we are still relatively early in the process
CONSTANCE D. LEHMAN, MD, PHD
FDA Review of AI Products for Breast Interpretation
The FDA is following the 510(k) review process for AI technologies, which requires demonstration of substantial equivalence to another legally U.S. marketed device. This is the same process that it followed for CAD.
“At first blush, this might cause concern—if we’re using the same strategy for clearance by the FDA that we used for prior CAD products, and those CAD products didn’t have the impact we were hoping for in our community practices, aren’t we destined to repeat the mistakes of the past?” Dr. Lehman asked. “I don’t think so. I couldn’t be more excited about what we are seeing in the new AI products that are coming out to assist in mammographic interpretation. But our prior experience also means we will be paying close attention to post market performance and effective methods to teach radiologists how to use the AI tools well.”
Currently, federal regulations require that all mammograms must be interpreted by a qualified physician and that’s a barrier to the use of AI in an autonomous manner, according to Etta D. Pisano, MD, chief research officer of the American College of Radiology and professor in residence in the Department of Radiology at Beth Israel Deaconess Medical School in Boston.
For standalone software performance, the FDA will require huge datasets. But these datasets must also represent the patient population they will serve, Dr. Pisano emphasized. For example, a study performed in the mostly white population of a Swedish center would not meet FDA requirements for representing the diverse populations of the U.S. For studies with datasets in the range of 200 to 400 cases, it’s not uncommon to have as many as 25 readers. In general, the radiologist will read studies with and without the assistance of AI, usually with a washout interval of a few weeks so that readers won’t remember specific cases.
“Obviously we get individual performance data from these kinds of studies, but we also get group performance data—and these reader studies are the most common way the FDA evaluates a product for approval,” Dr. Pisano said. “They’ll also want to see representation in the full range of breast density and in the full range of lesion types, sizes and features,” Dr. Pisano continued. “You don’t want lots of big cancers; you want the kinds of cases that are in a screening population, subtle cancers and, of course, rare conditions that should be detected by either the human reader or the AI.”
In addition, FDA validation will scrutinize not only the potential for missed cancers, but the risk for introducing false positives as well, she said. For Dr. Lehman, the current climate in mammography represents a supremely exciting era for refining and revolutionizing AI.
“We are just scratching the surface of the field of computer vision, and we are early in the application of AI for products to increase the potential of radiology to improve human health. The impact for radiologists will be extraordinary,” Dr. Lehman said. “There is no fear that AI will replace radiologists. In fact, our work will be so much more compelling, so much more impactful because we will have stronger tools to provide more patients access to higher quality, higher value care."
For More Information
Access the JAMA Internal Medicine study at jamanetwork.com/journals/jamainternalmedicine.
Read previous RSNA News stories on breast cancer detection: