QIBA Newsletter

QIBA Newsletter October 2020 • Volume 12, Number 3: Developments from the QIBA Multiparametric Working Group

In This Issue:



Improve the value and practicality of quantitative imaging biomarkers by reducing variability across devices, sites, patients and time.


Contact Us

Daniel C. Sullivan, MD
QIBA Chair


In My Opinion

Developments from the QIBA Multiparametric Working Group


In the fall of 2018, QIBA reinstituted the metrology working group to discuss how to expand QIBA’s current single quantitative imaging biomarker (QIB) metrology framework to multiparametric applications. The group is composed of physicists, radiologists, and statisticians from a variety of settings, meeting twice monthly.  We have divided our discussions into four applications, or use cases:

  • Multi-dimensional descriptor: Panel of individual, but related, QIBs, each of importance.
  • Phenotype classification: Multiple QIBs used together in a decision tool to classify cases into phenotypes.
  • Risk prediction: Multiple QIBs used together in a decision tool to predict patient outcome or risk.
  • Radiomics: Computer extraction of potentially large numbers of derived metrics, or “radiomic features,” for prediction.

While work remains, below is a summary of key discussions to date:

  1. Concept of ground truth: In the first metrology workshops six years ago, we spent a good deal of brain power developing a working understanding of “ground truth. ”Once again, we are faced with the challenges of defining ground truth in a practical yet rigorous fashion. Phenotypes and patient risk are not phenomena that we can build into a phantom, so often we are reliant on subjective clinical definitions.

  2. Measuring change: When there are multiple QIBs measured at baseline and follow-up, we cannot treat each QIB separately when determining if change has occurred. Rather, change is assessed in a multivariate fashion by considering the change in all QIBs simultaneously, accounting for the correlation among them. Think of a multi-dimensional ellipsoid where the area inside the ellipsoid defines the region of change due to measurement error and regions beyond the ellipsoid define real change.

  3. Technical performance of QIB inputs: In nearly every discussion about how to validate a decision tool, we refer to how we must first validate the QIB inputs. Whereas I once believed that validation of the output was sufficient, there is strong consensus that there are two validation steps: validation of the inputs and validation of the output. Furthermore, precision trumps bias. Decision tools accommodate bias in the QIB inputs by assigning weights proportionally to the bias, but poor precision of the QIB inputs shows up in the performance of the tool’s output. For many years QIBA has rightly focused on precision of QIBs.

  4. Clinical usefulness: To initiate a new biomarker committee, QIBA has required some justification of its clinical usefulness, but in our Profiles, we carefully avoid claims about clinical usefulness. The development of decision tools, however, necessitates additional motivation and justification of clinical usefulness in order to define the role of the decision tool for a specific patient population and therapeutic setting. The multiparametric group continues to discuss the appropriate space where we make technical claims about the decision tool’s output without overstepping into clinical efficacy.

  5. Profile claims: The claims in a multiparametric profile will look different than our current claims. The technical performance of a decision tool output starts with discrimination, i.e. the ability to separate different phenotypes or outcomes. Calibration, the closeness of agreement between a prediction and ground truth, is also important. The third metric is the familiar precision, or closeness of agreement between replicates. The three metrics are related but distinct, and it is important to recognize that they are impacted by the clinical population and therapeutic setting.

The multiparametric working group is writing five papers for publication. We welcome new members to our deliberations.

Nancy Obuchowski, PhD
Nancy Obuchowski, PhD

Nancy Obuchowski, PhD, is Vice-Chairman of Quantitative Health Sciences at the Cleveland Clinic and Professor of Medicine at the Cleveland Clinic Lerner College of Medicine of Case Western Reserve University. She is a Fellow of the American Statistical Association. Her research interests include study design and statistical analysis methods for imaging screening and diagnostic tests and imaging biomarkers. She is a member of the QIBA Steering Committee.


QIBA Leadership Announcement

RSNA/QIBA is pleased to announce that Dr. Gudrun Zahlmann will be appointed as our newest QIBA Vice Chair, beginning January 1, 2021. Dr. Timothy Hall will continue to serve as the second Vice Chair, and Dr. Alexander Guimaraes will assume the QIBA Chair position.

We are deeply grateful to Dr. Daniel Sullivan for returning at short notice to the QIBA leadership role in November 2019. Our appreciation extends to Drs. Guimaraes, Hall and Zahlmann for agreeing to take on the QIBA leadership roles going forward.

QIBA Activities

QIBA Biomarker Committees are open to all interested persons.  Meeting summaries, the QIBA Newsletter and other documents are available on the QIBA website RSNA.ORG/QIBA and wiki http://qibawiki.rsna.org/.   

QIBA Resources:

Please contact QIBA@rsna.org for more information. We welcome your participation.

*Please note that the list of references has been migrated to EndNote.
*To obtain access to the RSNA EndNote citations, please send an email request to: qiba@rsna.org.

The list of references showcases articles that mention QIBA, quantitative imaging, or quantitative imaging biomarkers. In most cases, these are articles published by QIBA members or relate to a research project undertaken by QIBA members that may have received special recognition.

New submissions are welcome and may be directed to QIBA@rsna.org.


RSNA 2019

For more information: https://www.rsna.org/annual-meeting