QIBA Newsletter October 2018 • Volume 10, Number 3: The Value of QI in Early and Late-phase Clinical TrialsIn This Issue:
IN MY OPINION
The Value of QI in Early and Late-phase Clinical Trials
By GUDRUN ZAHLMANN, PhD
The QIBA CT Volumetry Profile Conformance Assessment
By NICHOLAS PETRICK, PhD and KEVIN O’DONNELL, MASc
Improve the value and practicality of quantitative imaging biomarkers by reducing variability across devices, sites, patients and time.
Edward F. Jackson, PhD
In My Opinion
The Value of QI in Early and Late-phase Clinical Trials
By Gudrun Zahlmann, PhD
Biomarkers are used in clinical trials to guide the rationale of therapeutic intervention and to enable decisions on continuation/discontinuation of development processes for medical devices and especially for the development of novel therapeutics and drugs. Imaging techniques provide specific in vivo biomarkers that are additionally used for patient stratification, to inform trial protocol and trial design, to provide information for modeling of pharmacokinetics/pharmacodynamics, and to predict response as well as to monitor short-, mid- and long-term therapeutic outcomes. Which imaging biomarker to use depends on which question needs to be answered.
In early drug development phases, questions are, e.g., whether the new drug candidate reaches the target and, if so, in sufficient quantity or whether the expected mechanism of action can be observed. These activities often start pre-clinically. Here, a full tool box of potential biomarkers, including quantitative imaging biomarkers, is available. The biomarker(s) that are most informative are used. Often we see a variety of imaging techniques for pre-clinical in vivo imaging that provide valuable insights into biological processes and have the potential to guide decision in early drug development (1).
It becomes challenging when pre-clinical biomarkers need translation into clinical procedures. From an imaging perspective, it is easier to translate, e.g., MR sequences from animal into human use. One successful example was the use of DCE-MRI for the development of anti-angiogenic drugs (2). This example also illustrates how good communication and collaboration between drug companies and regulators can speed up drug development by introducing, validating and accepting a new biomarker (Ktrans based on DCE-MRI) for efficacy of, at that time, a novel drug class.
Such a seamless translation process is not always possible. Novel tracers for PET/SPECT need almost the same approval process as a new drug and are not always available for large scale, late-phase clinical trials. Therefore, we typically have a larger tool box for early development compared to late development phases (1).
Nevertheless, imaging biomarkers play an important role in late-phase clinical trials. The importance of imaging endpoints in clinical trials is reflected by specific FDA guidance to industry published April 2018 after thorough discussion (3).
QIBA´s mission is to support quantitative imaging biomarker applications in clinical trials and clinical care by standardizing all imaging procedure-related activities to enable reproducible and reliable measures from imaging techniques. This is in full agreement with the need of the researchers and regulators to build decisions on solid data. QIBA´s profiles are dedicated to this task. As a volunteer organization, QIBA has its own timelines in developing profiles, and needs synchronization with the needs of Profile users in clinical research, supported by industry or public funding. The next phase of QIBA activities should reflect this necessity.
- Mayer AT, Gambhir SS; The Immunoimaging Toolbox; the Journal of Nuclear Medicine, 2018; 59;8;1174-1182.O´Connor JPB, Parker GJM, Jackson A, Jayson GC; Dynamic contrast-enhanced MRI in clinical trials of antivascular therapies; Nature Reviews Clinical Oncology 9(2012)3, 167-177.
- Clinical Trial Imaging Endpoint Process Standards, Guidance for Industry, FDA Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER); April 2018.
QI / IMAGING BIOMARKERS IN THE LITERATURE
PubMed Search on: The Value of QI in Early and Late-phase Clinical Trials.
Each issue of QIBA Newsletter features a link to a dynamic search in PubMed, the National Library of Medicine's interface to its MEDLINE database. Link to articles on: The Value of QI in Early and Late-phase Clinical Trials.
Analysis Tools and Techniques
The QIBA CT Volumetry Profile Conformance Assessment
By Nicholas Petrick, PhD, and Kevin O’Donnell, MASc
The goal of a QIBA Profile is to achieve the measurement performance defined in the Profile Claims. The measurement process is modeled as a chain of "Activities" and, for each Activity, requirements are placed on the participating people and systems ("Actors") as needed to achieve the Claims. Below we discuss conformance testing for the CT Tumor Volume Change for Advanced Disease (CTV-AD) Profile  and our initial experiences with testing. To simplify conformance, requirements were grouped into checklists, including both simple observations and detailed assessment procedures.
Scanner and Reconstruction Software conformance requirements focus on achieving image quality sufficient for tumor segmentation by image analysis software. Based on expert opinion and our groundwork studies, most CT acquisition parameters were determined to manifest as an impact on resolution or noise, or as not having a strong impact on segmentation. Thus, conformance came down to meeting specific targets for spatial resolution (f50-value for the modulation transfer function [MTF]) and noise (voxel noise standard deviation) with only a few additional scanning requirements. Similar simplification may be effective in other QIBA Profiles. Scanner assessment of MTF and voxel noise was based on AAPM recommendations and the American College of Radiology (ACR) CT Accreditation Phantom such that the procedures are familiar and available at most sites.
Image Analysis Software conformance testing includes demonstrating tumor segmentation performance. A challenge is that performance on clinical datasets is most relevant, but "ground truth" is only known for phantom datasets. The current compromise assesses repeatability using clinical images and segmentation bias and linearity using images of artificial lesions in an anthropomorphic phantom (see Fig. 1). Test data is provided via the QIDW (Quantitative Imaging Data Warehouse) website, which means sites do not need to collect clinical cases or have access to an anthropomorphic phantom.
These assessments are non-trivial exercises due to the need to segment many lesions, but once a software package is qualified, testing does not need to be repeated. The phantom dataset has multiple scans of seven different nodules for a total of 39 volume estimates. The clinical dataset has 20 patients with two repeated scans each. These numbers are small and impact the robustness of our assessment procedure, but a balance between conformance testing effort and statistical power is required. Ideally larger datasets, especially for repeatability analysis, would be available to improve the statistical power of the analyses. In the future, digitally inserting realistic lesions into clinical images, such that truth is known, may provide a better solution for assessing both repeatability and bias/linearity within a single dataset .
While Technologist and Physicist conformance mostly involves training to highlight specific details of familiar procedures, the radiologist plays a critical QA role in the Profile by flagging problematic studies. For semi-automated algorithms, the radiologist affects segmentation performance and thus must perform the Image Analysis Software conformance testing for each software package used. This is a substantial burden, which may hinder the utilization of the Profile, and is an incentive to fully automate segmentation.Complexity is always a challenge to broad adoption of a specification. A key part of the QIBA Process involves a Technical Confirmation stage, where the Profile document is provided to several sites that then attempt to conform to the Profile requirements and determine if the requirements are understandable and practical. We found this review effort particularly useful and made numerous changes to the Profile to simplify the requirements and procedures before it was published as Technically Confirmed. For example, assessment datasets now include spreadsheets that can record test results and automatically compute conformance metrics, which should improve conformance data quality and reduce testing time. We would like greater automation of additional conformance testing steps to further reduce the burden on sites and radiologists to help improve utilization and acceptance by a wider community of QIBA Profile users.
QIBA CT Volumetry Technical Committee. QIBA Profile: CT Tumor Volume Change for Advanced Disease (CTV-AD). Technically Confirmed Profile June 22, 2018; Available from: http://qibawiki.rsna.org/index.php/Profiles.
QIBA® Profile Conformance Testing Now Available
We are excited to announce that QIBA® is now offering a new conformance testing service to help clinical sites demonstrate that they can achieve high quality quantitative imaging results. Clinical sites that achieve QIBA Profile specifications will receive a QIBA® Conformance Certification Mark to distinguish themselves as performing quantitative imaging studies with a high level of precision.
Scanner and analysis software vendors can also obtain the QIBA® Conformance Certification Mark to demonstrate that their specific medical device has been thoroughly tested and has achieved conformance with a QIBA Profile.
QIBA Conformance Certification Services, including an image quality assessment phantom and online phantom analysis services, are currently only available for the QIBA CT Small Lung Nodule Profile at:
QIBA Kiosk at RSNA 2018
QIBA Working Meeting at RSNA 2018
Wednesday, November 28, 2-6 pm – Lakeside Center, Room E253AB
QIBA Meet-the-Expert Poster SessionsLocation: McCormick Place, Chicago - Learning Center (Hall D)
Please visit the QIBA Kiosk poster area during the Meet-the-Expert (MTE) sessions at the 2018 RSNA Annual Meeting. Posters are on display all week and QIBA experts will be available during the lunch hour (12:30 – 1:30 pm, on Sunday, and 12:15 – 1:15 pm, Mon – Thurs).
These sessions provide an opportunity to interact with other meeting attendees and colleagues, learn about QIBA activities, discuss QIBA projects and share ideas.
Please contact QIBA@rsna.org for more information. We welcome your participation.
This list of references showcases articles that mention QIBA, quantitative imaging, or quantitative imaging biomarkers. In most cases, these are articles published by QIBA members or relate to a research project undertaken by QIBA members that may have received special recognition. New submissions are welcome and may be directed to QIBA@rsna.org.