Cognitive Bias in Radiology: Understanding the "Hard-Easy" Effect
Study explores how radiologists assess uncertainty as imaging cases become more challenging
When an imaging case becomes increasingly challenging, radiologists adjust their confidence—but perhaps not enough to reflect the full extent of the diagnostic uncertainty, according to a recent Radiology study examining a cognitive phenomenon known as the “hard-easy effect.”
The findings highlight an important aspect of diagnostic decision-making: recognizing not only when a case is difficult, but also when uncertainty warrants a change in approach, such as reviewing prior studies, obtaining additional imaging or consulting a colleague.
“The hard-easy effect occurs when confidence does not decline as sharply as accuracy when tasks become difficult,” said Eyal Bercovich, MD, one of the study’s authors.
Dr. Bercovich is a faculty member at the Herta and Paul Amir School of Medicine and the Department of Medical Imaging Sciences at the University of Haifa in Israel, and an attending radiologist in the Department of Medical Imaging, Rambam Health Care Campus in Haifa.
“In easy cases, confidence generally reflects performance. In difficult cases, people recognize some uncertainty but often underestimate how much the observed error rate has increased,” he said.
Cognitive biases stem from shortcuts that people, including medical professionals, develop to process large quantities of information quickly. They can affect how clinicians monitor their own confidence and use that information to determine what to do next.
Study Reveals Confidence Gap
In the study, 28 radiologists in their second year of residency or later and 50 nonradiology medical professionals reviewed bone radiographs. They identified whether bone lesions were absent or present. Their findings were then compared with a reference standard based on CT or MRI findings that were validated by a panel consisting of two board-certified musculoskeletal radiologists.
For each image, study participants rated their confidence and then recommended the next step—submitting/ordering additional imaging or consulting a senior colleague.
The investigators found that, among the radiologists and nonradiologists, the overconfidence rate was 14.73%. While radiologists had a higher success rate and confidence rate than nonradiologists, a consistent hard-easy effect emerged. The investigators noted that confidence was well-calibrated for easy cases, but they found substantial overconfidence (>40%) for difficult cases.
“In the easiest cases, accuracy was approximately 98% and confidence approximately 93%, indicating relatively good calibration,” noted study coauthor, Rakefet Ackerman, PhD, professor of cognitive psychology and vice dean for undergraduate studies in the Faculty of Data and Decision Sciences at Technion–Israel Institute of Technology in Haifa. “In the most difficult cases, accuracy fell to approximately 37%, while confidence remained around 80%.”
The finding illustrates the hard-easy effect: As diagnostic tasks become more difficult, people may recognize that they are less certain, but the decline in confidence does not necessarily keep pace with the decline in accuracy.
For radiologists, the study suggests that expertise provides an important advantage but does not eliminate this general feature of human decision-making.
Accuracy Improves, Bias Persists
Perhaps the most surprising study finding is that overconfidence was not predicted or mitigated by professional experience, training stage or clinical setting, said Elizabeth Krupinski, PhD, who wrote a related Radiology editorial.
Dr. Krupinski is professor and vice chair for research in the Department of Radiology & Imaging Sciences at Emory University in Atlanta.
“One would naturally expect that years of specialized residency, fellowship training and clinical practice would fine-tune a radiologist's metacognitive monitoring, making them highly aware of what they do and do not know,” she said. “While radiologists were undeniably more accurate overall, they remained just as vulnerable to the hard-easy effect as non-radiologists.”
“Confidence did decrease, indicating partial recognition of the increased difficulty, but not nearly enough to reflect the observed increase in error,” Dr. Ackerman said. “The decline in confidence was far smaller than the corresponding increase in error.”
“The response-time findings were particularly striking,” Dr. Bercovich noted. “It is intuitive to assume that spending more time will eventually improve the answer, yet the longest decisions were associated with poorer performance and greater overconfidence.”
This does not mean that faster interpretation is inherently better, he emphasized. “Rather, prolonged unresolved uncertainty may indicate that it is time to change strategy or seek assistance,” Dr. Bercovich said.
According to Dr. Bercovich, cognitive and metacognitive biases concern different aspects of the diagnostic process. Cognitive biases systematically influence how information is perceived or interpreted, while metacognitive biases affect how clinicians evaluate and regulate their own reasoning: how confident they are, how long they continue and whether they seek help.
“The most concerning combination, therefore, is an error accompanied by high confidence, because it is less likely to prompt reconsideration, consultation or additional imaging,” Dr. Bercovich noted. “The study targeted these metacognitive processes rather than the full spectrum of cognitive biases.”
“The study’s goal is not to make radiologists less confident or more hesitant. It is to make confidence more accurately reflect the evidence and ensure that unresolved uncertainty leads to an appropriate change in strategy.”
— RAKEFET ACKERMAN, PHD
Turning Uncertainty Into Better Decisions
Dr. Bercovich recommends radiologists treat prolonged unresolved uncertainty as meaningful information.
“When a case remains difficult, it may be more effective to change strategy: review the clinical context and prior studies, perform a structured second search, consult a colleague or recommend additional imaging, rather than simply continuing to look,” he said. “It is also useful to distinguish effort from evidence. Spending more time does not necessarily make conclusions more reliable.”
“As AI systems become more integrated into radiology training should also address trust calibration—helping clinicians understand when the AI system is reliable, when their own judgment is reliable and when disagreement should trigger further review rather than automatic deference to either,” Dr. Ackerman said.
To prevent automation bias, Dr. Krupinski noted, AI decision-support algorithms should be configured to withhold their findings, bounding boxes or text outputs until after the radiologist has completed their primary interpretation and logged their initial findings, though this may not be possible or logical with all algorithms.
“This preserves the radiologist’s native search pattern and critical thinking,” Dr. Krupinski said.
“The study’s goal is not to make radiologists less confident or more hesitant,” Dr. Ackerman emphasized. “It is to make confidence more accurately reflect the evidence and ensure that unresolved uncertainty leads to an appropriate change in strategy.”
“We hope our research will inspire the development of training programs for confidence calibration and for using calibrated confidence to guide next-step decisions,” she said.
For More Information
Access the Radiology study, “Metacognitive Processes in Radiologic Decision-Making: The Hard-Easy Effect and Temporal Thinking Dynamic,” and the related editorial, “Navigating the Blind Spots: The Critical Role of Metacognition in Diagnostic Accuracy.”
Read previous RSNA News stories on cognitive bias in radiology: