QIBA Newsletter

QIBA Newsletter February 2022 • Volume 14, Number 1: DWI Profile: Experience and Lessons Learned in Validation of Site Conformance and the Quest for Simple yet Informative Technical Evaluation Procedures

qiba-logo

In This Issue:
IN MY OPINION ANALYSIS: TOOLS AND TECHNIQUES

QIBA EDUCATION
QIBA and QI/Imaging Biomarkers in the Literature

QIBA MISSION

Improve the value and practicality of quantitative imaging biomarkers by reducing variability across devices, sites, patients and time.

QIBA CONNECTIONS
QIBA Wiki

QIBA CONTACT
Contact Us

Timothy J. Hall, PhD
QIBA Chair


 

In My Opinion

DWI Profile: Experience and Lessons Learned in Validation of Site Conformance and the Quest for Simple yet Informative Technical Evaluation Procedures

By DARIYA I. MALYARENKO, PhD and THOMAS L. CHENEVERT, PhD  

QIBA Profiles are designed to provide a structured roadmap for imaging centers, scanner manufacturers, radiologists, and other stakeholders that seek use of quantitative image readouts for disease characterization and/or to monitor response to treatment. Despite clear goals and specifications, one faces many challenging aspects to validate conformance to QIBA roadmaps.

As detailed in Profiles, multiple “actors” (people, scanners, and software) are required to function in concert to achieve stated numerical claims, yet procedures to validate performance of each actor can be difficult to define, let alone implement in practice. Currently, an array of “checklists” to be completed by site personnel serves as an attestation of full or partial conformance, but validation of “self-attestation” checklist responses can be difficult. As a more tangible first step, the DWI Biomarker Committee focused on the one actor that can be objectively validated – the MRI scanner.

Toward this end, the DWI BC provided four essential elements:

  1. description of suitable DWI phantom(s)
  2. detailed DWI scan protocol
  3. target performance specifications, and
  4. analysis software to assess performance derived from DWI DICOM

Since water mobility (or ADC) is sensitive to temperature, the DWI Profile describes use of ice water-based phantoms that serve to both determine and control temperature, thus providing a “true value” so that absolute ADC bias of an MRI system is measurable. Simple ice water-based phantoms have been used to certify MRI scanner performance for site qualification and periodic QA in multiple clinical trials utilizing DWI in breast, prostate, and brain.

DWI phantom designs and protocols were adapted to specific trial objectives, but phantom data and experience gleaned from these trials certainly contributed to development of the QIBA DWI Profile. The next generation DWI phantom contains polyvinylpyrrolidone (PVP) materials for an array of ADC values to assess MRI system linearity, though still uses ice water for temperature control. PVP phantom preparation, scan protocol, analysis procedures and performance benchmarks were tailored for QIBA and are detailed in the current “QIBA DWI MRI Profile Conformance Testing” documentation available on the QIBA Wiki.

These procedures were evaluated by a handful of academic centers around the globe to test performance of their MRI systems within the context of QIBA DWI specifications. The exercise was certainly informative to all parties and provided QIBA DWI BC valuable feedback to further improve its Profile and streamline technical conformance validation steps. Moreover, DICOM datasets from these tests were shared with QIBA DWI BC for independent analysis to validate that specified technical benchmarks were met and consistent with the site’s results.

Motivated by the common feedback theme for “process simplification,” DWI BC members modified the PVP phantom analysis software for greater automation and DWI QC Report generation, with an example shown below. Key features built into the analysis workflow include robust handling of MRI manufacturer-specific DWI DICOM series conventions and image sort orders, and summary of performance measures relative to QIBA-specific benchmarks.

We used this workflow for subsequent validation of commercial DWI QC software (“qCal”) described in the accompanying article by Kevin Miller in this QIBA Newsletter issue. The numerical consistency of the qCal output versus QIBA tools was checked for representative DWI phantom DICOM examples from the dominant MRI vendors. It is hoped that this user-friendly commercial product will further streamline site conformance validation and provide real-time feedback for system performance based on DWI phantom evaluation.

Example of DWI Profile

Figure caption: Example of DWI Profile conformance test report for a PVP phantom scan. The report lists key scan protocol parameters (extracted from DICOM) and assessed scanner performance metrics for ice-water (central tube). The plots further summarize measured SNR, linearity, bias, and random error metrics for all PVP components.

Dariya Malyarenko, PhD

Dariya Malyarenko, PhD

Dariya Malyarenko, PhD, is an associate research scientist and lead image analyst of the MRI Research Core in the Radiology Department of the University of Michigan Medical School. Her current research focuses on physical and digital phantoms for advanced tissue diffusivity models, along with quality control and systematic bias correction to improve accuracy of DWI-derived cancer biomarkers in multi-center clinical oncology trials. She is a co-chair of the QIBA DWI Biomarker Committee.

Thomas Chenevert, PhD

Thomas Chenevert, PhD

Thomas Chenevert, PhD, is a professor of radiology at the University of Michigan Medical School. His current research focuses on correction of known systematic bias to improve accuracy of DWI-derived cancer biomarkers in NCI-sponsored multi-center clinical and co-clinical oncology trials.



 

Analysis Tools and Techniques

New QIBA Conformance Certification Service Supports Critical DWI Profile and Biomarker Claims for Breast, Liver, Brain, and Prostate

By KEVIN MILLER, MSE

DWI and derived quantities, such as ADC, are being incorporated into clinical trials as potential diagnostic or therapy response biomarkers. As presented in Dr. Nancy Obuchowski’s article in the October 2021 QIBA Newsletter, “The ability to characterize and quantify the bias of a measurement relative to a True or Reference Value is important to quantitative imaging [and] this is possible with biomarkers.” In support of these needs, QIBA’s DWI Biomarker Committee, co-chaired by Drs. Malyarenko and Boss, has developed a Profile —currently at “Consensus Stage” in the Profile development process — to quantify ADC biomarker performance thresholds, address deficiencies, and capture best practices in DWI acquisitions1.

However, from a practical perspective, MR users have been set back by the lack of a validated analysis tool, especially for multi-site and longitudinal studies. Drs. Malyarenko and Chenevert of the University of Michigan developed analysis tools based on the Profile to quantify site acquisition performance. These tools and analyses have resulted in improvements in the quality, repeatability, and consistency of scans with respect to specified DWI phantoms; however, the tools were not designed as commercial software products to be broadly disseminated.

In parallel with that work, CaliberMRI, Inc., a company focused on the development of quantitative MRI technologies and products, developed a quality control platform for MRI scans, known as qCal-MRTM, with the objective of providing a highly automated tool for evaluating the critical scan performance parameters with respect to reference value phantoms. The cloud-based software is designed to read DW-MRI DICOM data of the QIBA DWI phantom. The DWI DICOM is generated using multiple b-value acquisitions, where “b-value” is a factor representing the degree of diffusion weighting, based on the amplitude, timing, and duration of applied gradient pulses.

The resultant qCal-MR analysis report provides:

  1. visualization of DWI and derived ADC maps
  2. highly automated, efficient and consistent identification of ROIs
  3. statistical analyses of ROIs and VOIs applied to DWI and ADC maps
  4. tabulation of summary statistics of defined performance metrics including SNR, bias and random error and ADC linearity, and
  5. analyses of calculated performance as compared to NMR-verified reference values for the phantom.

CaliberMRI’s “QIBA” DWI phantom and a screenshot of a qCal-MRTM visualization are provided in Figures 1 and 2, respectively.

Figure 1 CaliberMRI’s “QIBA” Model 128 Diffusion Phantom.

Figure 1 CaliberMRI’s “QIBA” Model 128 Diffusion Phantom.

Figure 2 qCal-MRTM allows real time slice-by-slice visualization of automated analysis products.

Figure 2 qCal-MRTM allows real time slice-by-slice visualization of automated analysis products.

qCal-MRTM has been in use at many sites around the world for more than a year now. The cloud-based toolset has been maintained and advanced during that time, to the benefit of users. As an example, an MR-readable thermometer is now incorporated into the QIBA DWI phantom, and qCal-MRTM was expanded to provide automated readout and temperature correction for ADC calculations. In addition, qCal-MRTM was updated to incorporate NMR data generated by NIST for room temperature calibration of DWI scans, simplifying the phantom setup and preparation for quality control scans.

QIBA leadership recognized the relevance of the new commercial platform to the MRI DW committee’s efforts and established that validation of qCal-MRTM with respect to the Profile and the UM analysis tools and establishment of CaliberMRI as a scanner QC analysis service provider could yield a sustainable process for accelerating the adaptation of the DWI Profile in the medical imaging community. To that end, during 2021, qCal-MRTM was rigorously evaluated against UM benchmark cases. These baseline DICOMs, generated using scanners from Philips, GE, and Siemens at different sites with various staff members, provide a repeatable basis for validation and verification.

In December 2021, QIBA entered into an agreement with CaliberMRI to provide services to client sites for the certification of site conformance to the processes defined in the Profile for DWI MR. For more information on DWI Profile Conformance Certification, please email the author (kmiller@qmri.com) or visit CaliberMRI’s website (calibermri.com, or qmri.com).

Reference:
Boss, Malyarenko, et al; QIBA DWI Profile Consensus Version; 2019-Dec-20.

Kevin Miller, MSE

Kevin Miller

Kevin Miller, MSE, is the director of business development at CaliberMRI, Inc. He has developed advanced technology systems including quantitative MRI technology for medical and space industries for over 35 years. In addition to working in the design and analyses of instruments and systems, he has worked in business development for large and small businesses. Mr. Miller is a member of the QIBA DWI MR Biomarker Committee.




QIBA Leadership Announcements

Caroline Chung

New QIBA Vice Chair

We are pleased to announce that Caroline Chung, MD, MSc, FRCPC, CIP, has been selected to serve as the second QIBA vice chair (VC). Dr. Chung will transition into this role over the first quarter of the year, assuming full VC responsibilities on April 1, joining Drs. Hall and Zahlmann as a member of the QIBA leadership team. One of Dr. Chung’s areas of focus will be increasing engagement of clinicians in the QIBA process.

Dr. Chung is associate professor and director of imaging technology and innovation for the Department of Radiation Oncology at The University of Texas MD Anderson Cancer Center, in Houston, TX. She also serves as MD Anderson Cancer Center’s vice president and chief data officer. Welcome Dr. Chung!

QIBA Activities

QIBA Biomarker Committees are open to all interested persons.  Meeting summaries, the QIBA Newsletter and other documents are available on the QIBA website RSNA.ORG/QIBA and wiki http://qibawiki.rsna.org/.   


QIBA Resources:

Please contact QIBA@rsna.org for more information. We welcome your participation.

*Please note that the list of references has been migrated to EndNote.
*To obtain access to the RSNA EndNote citations, please send an email request to: qiba@rsna.org.

The list of references showcases articles that mention QIBA, quantitative imaging, or quantitative imaging biomarkers. In most cases, these are articles published by QIBA members or relate to a research project undertaken by QIBA members that may have received special recognition.

New submissions are welcome and may be directed to QIBA@rsna.org.