LLMs Improve Clinical Histories for Radiologists
Models perform well at extracting, summarizing critical details from electronic health records
In the evolving landscape of radiology and AI, large language models (LLMs) can transform clinical workflows. Recent studies have demonstrated that LLMs can generate automated clinical histories for oncologic imaging, offering more comprehensive information than traditional requisition histories and enhancing radiologists’ interpretative accuracy and patient safety.
Rajesh Bhayana, MD, assistant professor of radiology at the University of Toronto, co-authored a study published in Radiology that assessed the efficacy of GPT-4, a state-of-the-art LLM, in extracting essential clinical information from electronic health records to create structured histories for oncologic imaging.
Imaging plays a central role in oncology care. Clinical information provided by ordering physicians on requisitions helps inform the imaging protocol and interpretation. If clinical history is lacking, it could lead to missed findings for patients, Dr. Bhayana noted. “If we’re reading cancer imaging and the requisition doesn’t mention the primary cancer, how we interpret the study and the likelihood we pick up cancer may be impacted,” he said.
According to Dr. Bhayana, the primary reason for lack of information on imaging requisitions is the time constraint faced by referring clinicians. “Everyone across the board in health care is pressed for time,” he said. “Since a lot of information already exists in the chart in long-form clinical notes, LLMs could automatically extract certain information and summarize key details. Unlike humans, LLMs never get tired.”
AI-Generated Clinical Histories: Reliable and Scalable
The retrospective study looked at 207 randomly selected patients who had CT performed at a large Canadian cancer center from January to November 2023.
A multidisciplinary team informed the selection of 10 important parameters for oncologic imaging history, including primary oncologic diagnosis, treatment history and acute symptoms.
Clinical notes were independently reviewed to establish the reference standard regarding presence of each parameter. After prompt engineering with seven patients, GPT-4 automatically generated structured clinical histories for the 200 remaining patients.
For the 200 LLM-generated histories in the study, GPT-4 achieved near-perfect results in extracting key parameters from clinical notes. Notably, LLM-generated histories more frequently included critical details such as the primary oncologic diagnosis, acute or worsening symptoms, and relevant surgical history when compared to original requisition histories.
In an editorial published alongside the study, Neda Tavakoli, PhD, postdoctoral fellow at Northwestern University in Chicago, specializing in AI-driven algorithms in imaging, underscored the potential of LLMs to address a persistent challenge in radiology. “The lack of complete clinical histories has been a well-recognized issue for decades,” she wrote. “Despite various quality improvement initiatives aimed at improving documentation, the problem persists.”
Dr. Tavakoli highlighted that LLMs offer a sustainable and scalable solution. “These models do not suffer from fatigue, distraction, or the competing demands that impact human documentation,” she noted. “They consistently produce structured, complete histories that enhance radiologists’ ability to provide accurate and meaningful interpretations.”
Despite their promise, integrating LLMs into clinical practice may face technological challenges as well as concerns about reliability and oversight. Dr. Tavakoli emphasized that AI-generated histories should remain verifiable.
“For radiologists to trust these tools, there should be mechanisms that allow easy verification of key clinical details,” she wrote. “This could be achieved by linking extracted information back to the original notes.”
“These models do not suffer from fatigue, distraction, or the competing demands that impact human documentation. They consistently produce structured, complete histories that enhance radiologists’ ability to provide accurate and meaningful interpretations.”
— NEDA TAVAKOLI, PHD
Customizing AI Summaries for Different Radiology Workflows
Radiologists in the study strongly preferred LLM-generated histories over original histories for imaging interpretation and safety. However, they showed no significant preference between AI-generated and standard histories when developing exam protocols.
“Given that protocoling is a repetitive, efficiency-driven task, radiologists may favor more concise histories in this context,” Dr. Bhayana said. “This highlights the need to tailor LLM-generated extractions for specific tasks.
Structured histories may be optimal for reporting, but more concise one-liners or AI-based auto-protocoling could enhance workflow.”
For now, Dr. Bhayana noted, LLM-generated histories are more likely to augment brief clinician-entered histories, allowing radiologists to access additional details as needed.
What’s Next for AI Integration in Radiology?
Looking ahead, experts see LLMs as a transformative force in radiology.
“LLMs are a natural fit for many manual tasks that people don’t have time to do,” Dr. Bhayana said. “They can relieve administrative burden and allow clinicians to focus on what they’re trained to do.”
Dr. Tavakoli echoed this sentiment, noting that the integration of AI into radiology workflows represents a major step toward more precise and efficient patient care. “The applications of LLMs extend far beyond clinical histories,” she wrote. “From protocoling imaging studies to augmenting radiologic interpretation and streamlining reporting, these models have the potential to significantly enhance radiology practice.”
As research and implementation efforts continue, the role of AI in radiology is poised to expand, offering new opportunities to improve diagnostic accuracy, workflow efficiency and patient outcomes.
For More Information
Access the Radiology article, “Leveraging Large Language Models to Generate Clinical Histories for Oncologic Imaging Requisitions,” and the accompanying editorial, “AI-generated Clinical Histories for Radiology Reports: Closing the Information Gap.”
Read previous RSNA News stories about large language models: