How Well Does AI Detect Invasive Breast Cancers?
Researchers examine cancer subtypes and imaging features that AI is most likely to miss on screening mammograms
AI has rapidly become a pivotal force in radiology, transforming the way experts screen and interpret complex imaging like CT scans. Its influence is so profound that it has caused imaging professionals to debate whether AI could redefine, or even replace, some of the core tasks traditionally performed by radiologists.
Sung Eun Song, MD, is not concerned that AI will eclipse radiologists. However, she believes it’s crucial for radiologists to learn about and use the technology.
“I strongly agree with the statement Dr. Langlotz made several years ago, ‘Radiologists who use AI will replace those who don’t,’” said Dr. Song, a professor in the Department of Radiology at Korea University Anam Hospital in Seoul, South Korea.
She and colleagues recently published a study in Radiology that evaluated the false-negative rate (FNR) of AI screening mammograms according to molecular subtype. Their goal was to learn more about the types of cancers AI misses, and why it fails to detect them.
“To become a breast radiologist who uses AI effectively, I felt it was important to understand which types of breast cancers AI tends to miss and how radiologists themselves perform in those same areas,” she said.
Assessing AI’s Performance
The retrospective review looked at 1,082 consecutive women diagnosed with 1,097 cancers between January 2014 and December 2020.
The researchers used commercial AI software to read the mammograms and acquire an abnormality score (AS). They identified AI-missed cancers as those that didn’t identify a precise location that matched the reference standard.
The researchers calculated the false-negative rate by counting AI-missed cancers according to the molecular subtype, such as hormone receptor (HR)-positive cancers versus human epidermal growth factor receptor 2 (HER2)-positive cancers.
In the study, AI missed 14% of cancers, with the highest false-negative rate occurring in HR-positive cancers. The technology was more likely to miss cancers that were smaller, lower grade, in dense breast tissue or located outside typical mammary zones. Most of these missed cancers (62%) were deemed actionable by radiologists.
“HR-positive cancers account for about 70% of all breast cancers,” Dr. Song said. “HER2-positive cancers are often accompanied by calcifications, and triple-negative cancers tend to have more volumetric features, both of which make them easier to detect on mammography.”
In contrast, Dr. Song said, HR-positive cancers often present with more subtle findings, which may explain why AI had the highest false-negative rates in this group.
The findings suggest that relying solely on AI could lead to overlooked early-stage but clinically significant cancers, especially in younger women and those with dense breasts.
Radiologists Key to Catching AI Misses
According to Dr. Song, the findings carry particular significance because the investigators excluded ductal carcinoma in situ, focusing only on invasive cancers.
“Missing early-stage disease can have critical consequences,” she said. “I believe the best approach is to first review AI outputs and then carefully re-examine the areas emphasized by our study—such as dense breasts, architectural distortion, amorphous microcalcifications and non-mammary zones—when interpreting mammography.”
In a related Radiology editorial, Lisa A. Mullen, MD, an associate professor in the Breast Imaging Division in the Department of Radiology and Radiological Sciences at the Johns Hopkins University School of Medicine in Baltimore, noted that the study adds to our understanding of AI-missed invasive breast cancers.
“It is critical for radiologists to understand what could be potentially missed by the software so that the radiologist can use the information to decrease the chance of missed cancers,” Dr. Mullen wrote.
“An AI algorithm can overcall benign findings, potentially leading to an increase in recall rate (false-positive interpretation),” she continued. “The algorithm can also under call or miss findings, leading to a missed or delayed cancer diagnosis (false-negative interpretation).”
According to Dr. Mullen, it is important to gain insight into the types of findings that can be missed, how often this occurs, and whether there is a way for radiologists to partner with AI to limit missed findings.
Dr. Song noted that about one-third of the cancers missed by AI in this study were also missed by radiologists. “This suggests that embracing AI more actively, while continuously improving our mammography interpretation skills, will ultimately enhance patient care,” she said.
“This suggests that embracing AI more actively, while continuously improving our mammography interpretation skills, will ultimately enhance patient care.”
— SUNG EUN SONG, MD
Making the Most of AI in Practice
Dr. Song emphasized the importance of incorporating AI tools into daily radiology practice. “Radiologists are already familiar with machine learning–based CAD, which has been used since 1990s, but deep learning–based AI offers a new dimension of value,” she said. “It markedly reduces false positives and accelerates reading by highlighting only the key findings.”
While Dr. Song encourages radiologists to capitalize on AI’s ability to help them read faster, she emphasized the need to remain vigilant about areas AI is prone to miss.
For More Information
Access the Radiology study, “Invasive Breast Cancers Missed by AI Screening of Mammograms,” and the related editorial, “Improved Understanding of False-Negative AI Mammographic Interpretation.”
For quality breast imaging educational opportunities, check out the RSNA EdCentral Breast Cancer Awareness Playlist.
Read previous RSNA News stories on breast imaging: