Warning! OUTDATED BROWSER DETECTED!   Please update your browser immediately for a better experience on this website. Learn More
  • Augmented Reality Technology Poised to be a Game-Changer in Radiology

    Experts discuss the role of imaging in creating the virtual reality applications poised to change medicine on numerous levels. By Michael Hart

    July 1, 2017

    The global phenomenon Pokemon Go is one example of the fast-growing trend of augmented reality technology, which analysts predict will become a $100 million market within three years.

    While the market is primarily driven by the gaming and entertainment industry, healthcare — including radiology — is beginning to piggyback off the trend to develop virtual reality applications that have the potential to dramatically change medicine in a number of ways.

    “Our lab at the university has consistently been pulling technologies that have been developed outside of medicine to determine if we can apply them to medical imaging,” said Eliot Siegel, MD, a professor at the University of Maryland, College Park, and a member of the RSNA RadLex® Steering Committee.

    Unlike virtual reality, which creates a completely immersive environment when someone dons a headset, augmented reality allows users to experience their surroundings at the same time they are viewing computer-generated information.

    Back in February 2013, Dr. Siegel and his research team were eager to get their hands on one of the first augmented-reality devices that could be placed on a person’s head: Google Glass. More recently, the team has been focusing on Microsoft’s HoloLens, introduced in 2016, to gauge the potential for using these devices in radiology.

    Although wearable technology is still in its first iteration, Dr. Siegel expects significant changes for radiology in the not-too-distant future.

    “I believe wearable technology will be a major game changer in the way we’ll be practicing radiology in the future, and in the way health professionals of all kinds will perform their procedures going forward,” Dr. Siegel said.

    The step-by-step process for 3-D hologram projection with Microsoft HoloLens. From left: CT scan acquired with fiducial marker in place; de-identified DICOM stent converted to STL file; STL files converted to FBX rendering; FBX projected on patient and co-registered to fiducial marker.
    Images courtesy of the University of Maryland.

    Possible Imaging Applications

    Dr. Siegel said there are at least three areas in which the devices could be implemented in radiology: education, diagnostic radiology and imaged-guided intervention.

    Education is perhaps easiest to understand. While radiologists are becoming more familiar with 3-D printing, wearable optical devices take that one step further by creating a hologram — a 3-D image that the user can place in one location and move around in real time. By projecting 3-D images, students can “walk” around and view anatomy from different perspectives, aiding the learning process.

    There are numerous potentially promising applications for diagnostic radiology, including using the devices to view as many virtual images as they want in their own field of vision.

    “It gives you a tremendous amount of flexibility and the ability to personalize care,” Dr. Siegel said.

    Augmented-reality devices also allow radiologists to view high-quality images on a screen when they are away from their usual workplace.

    Vikash Gupta, MD, a radiology resident on Dr. Siegel’s team, said, “Often you’re trying to read something on an iPad or a laptop. With a headset, you can show images across a field where the view is really limitless.”

    It also opens up the potential to incorporate artificial intelligence and machine learning into treatment planning.

    “We can say, ‘Headset, please look at another area and tell me if it’s the same,’” Dr. Gupta said.

    In terms of image-guided intervention, an augmented-reality device allows the radiologist to take all the available data points — from CTs and MRIs, for instance — and create a hologram that can then be placed on the patient’s body during a procedure.

    “You can actually put a needle or a catheter into a patient and perform a procedure by using image guidance from a previous scan in a holographic projection,” Dr. Siegel said.

    Because access to information is often limited during a procedure, a surgeon might have to walk away from the patient to view an image. But by wearing a headset, the surgeon could have all the information necessary in his field of vision.

    “You can reference prior anatomy, prior scans, prior labs, prior medical notes,” Dr. Gupta said. “All the information is at your fingertips, without ever having to leave the patient’s side.”

    Technology Faces Limitations

    There are limitations, but many of them are related to the development of the augmented-reality technology itself, not necessarily its application in radiology.

    Marrying the devices’ optical display features to software that will segment, register and stream data in real time will be vital. Also, the image resolution is still in its early stages and most would say it is not yet on par with high-definition television.

    “The images are not quite as sharp as we’re used to with the current generation of monitors,” Dr. Siegel said.

    Refresh rates are still slow and the devices weigh about a pound, which can lead to fatigue for longer procedures. And it will take time for radiologists to adjust to the changes to their current workflows.

    “There is a learning curve,” Dr. Gupta said. “Our hope is that once radiologists see the benefit, it will outweigh the learning curve.”

    Dr. Siegel predicted that over the next three to five years other academic institutions will begin experimenting with the technology. And as younger radiologists who have grown up with this type of technology enter the specialty, that will enhance the odds of its mainstream acceptance.

    “It’s been wonderful to work with really savvy residents like Dr. Gupta and others who already have programming, game design and development in their backgrounds,” Dr. Siegel said. “We in radiology have the potential to be leaders in augmented-reality applications.”



    Augmented technology is poised to benefit radiology education. By projecting 3-D images, students can “walk” around and view anatomy from different perspectives. Image courtesy of the University of Maryland.

    Figure 4
    Augmented-reality devices allow radiologists to view high-quality images on a screen when they are away from their usual workplace. Image courtesy of the University of Maryland.

    Web Extras