QIBA Newsletter

In My Opinion

Developments from the QIBA Multiparametric Working Group

By NANCY OBUCHOWSKI, PhD

In the fall of 2018, QIBA reinstituted the metrology working group to discuss how to expand QIBA’s current single quantitative imaging biomarker (QIB) metrology framework to multiparametric applications. The group is composed of physicists, radiologists, and statisticians from a variety of settings, meeting twice monthly.  We have divided our discussions into four applications, or use cases:

• Multi-dimensional descriptor: Panel of individual, but related, QIBs, each of importance.
• Phenotype classification: Multiple QIBs used together in a decision tool to classify cases into phenotypes.
• Risk prediction: Multiple QIBs used together in a decision tool to predict patient outcome or risk.
• Radiomics: Computer extraction of potentially large numbers of derived metrics, or “radiomic features,” for prediction.

While work remains, below is a summary of key discussions to date:

1. Concept of ground truth: In the first metrology workshops six years ago, we spent a good deal of brain power developing a working understanding of “ground truth. ”Once again, we are faced with the challenges of defining ground truth in a practical yet rigorous fashion. Phenotypes and patient risk are not phenomena that we can build into a phantom, so often we are reliant on subjective clinical definitions.
   
   
2. Measuring change: When there are multiple QIBs measured at baseline and follow-up, we cannot treat each QIB separately when determining if change has occurred. Rather, change is assessed in a multivariate fashion by considering the change in all QIBs simultaneously, accounting for the correlation among them. Think of a multi-dimensional ellipsoid where the area inside the ellipsoid defines the region of change due to measurement error and regions beyond the ellipsoid define real change.
   
   
3. Technical performance of QIB inputs: In nearly every discussion about how to validate a decision tool, we refer to how we must first validate the QIB inputs. Whereas I once believed that validation of the output was sufficient, there is strong consensus that there are two validation steps: validation of the inputs and validation of the output. Furthermore, precision trumps bias. Decision tools accommodate bias in the QIB inputs by assigning weights proportionally to the bias, but poor precision of the QIB inputs shows up in the performance of the tool’s output. For many years QIBA has rightly focused on precision of QIBs.
   
   
4. Clinical usefulness: To initiate a new biomarker committee, QIBA has required some justification of its clinical usefulness, but in our Profiles, we carefully avoid claims about clinical usefulness. The development of decision tools, however, necessitates additional motivation and justification of clinical usefulness in order to define the role of the decision tool for a specific patient population and therapeutic setting. The multiparametric group continues to discuss the appropriate space where we make technical claims about the decision tool’s output without overstepping into clinical efficacy.
   
   
5. Profile claims: The claims in a multiparametric profile will look different than our current claims. The technical performance of a decision tool output starts with discrimination, i.e. the ability to separate different phenotypes or outcomes. Calibration, the closeness of agreement between a prediction and ground truth, is also important. The third metric is the familiar precision, or closeness of agreement between replicates. The three metrics are related but distinct, and it is important to recognize that they are impacted by the clinical population and therapeutic setting.
   

The multiparametric working group is writing five papers for publication. We welcome new members to our deliberations.