Warning! OUTDATED BROWSER DETECTED!   Please update your browser immediately for a better experience on this website. Learn More
    • Science
    • QIBA Newsletter

      May 2016 • Volume 8, Number 2: A Regulatory Perspective on Quantitative Imaging and QIBA

      Improve the value and practicality of quantitative imaging biomarkers by reducing variability across devices, patients and time.
      QIBA Wiki
      Contact Us
      Edward F. Jackson, PhD
      QIBA Chair



      A Regulatory Perspective on Quantitative Imaging and QIBA

      By Nicholas Petrick, PhD, MS

      The mission of the U.S. Food and Drug Administration’s (FDA) Center for Devices and Radiological Health (CDRH) is to protect and promote public health. CDRH accomplishes this by promoting science-based decision making and assuring that patients and providers have timely and continued access to safe, effective, and high-quality medical devices and radiation-emitting products. With the growth of big data analytics and image processing, quantitative imaging (QI) tools are now widely available in clinical radiological image evaluation systems (PACS). One example is the Philips MDixson-Quant tool [1] for non-invasive triglyceride fat fraction calculation based on MR imaging data. QI is also important for other medical devices outside of traditional radiology, such as the recently granted de novo approval for the HeartFlow FFRCT device [2], which outputs a fractional flow reserve value, a derived quantity computed from simulated pressure, velocity and blood flow estimates based on information obtained from a 3D computer model generated from QI information extracted from coronary CT images. The next frontier in radiology could be the widespread introduction of radiomic tools. QI tools and the characterization of their performance are of paramount interest to CDRH because of their growing public health impact across a wide range of medical devices.

      While industry and academia are well positioned to develop novel QI tools, CDRH strives to provide U.S. patients with first access to safe and effective tools through the recognition of regulatory-grade assessment methods. One approach CDRH is taking to advance method development is to explore a qualification pathway for medical device development tools (MDDTs). Draft guidance on MDDTs was published in 2013 by CDRH, and a MDDT pilot is currently underway [3]. An MDDT is a scientifically validated, regulatory-grade tool that aids device development and regulatory evaluation within a specified context of use. QIBA’s QI-related efforts are well aligned with CDRH’s MDDT framework, especially in the context of phantom and assessment method development for QI tools.

      Radiological QI tools are generally cleared as components of larger PACS systems without specific QI claims or clinical validation. CDRH is very interested in developing assessment methods that reduce the regulatory burden for clearing/approving QI tools with specific claims. One approach CDRH is exploring is the broader use of phantoms, simulation, and theoretical bounds within an overall assessment paradigm. Some initial work by CDRH scientists has shown good agreement in volume estimation performance, in terms of the standard deviation of percent error (SPE) among physical phantom, simulation, and theoretical studies for low-contrast lesions. The results given in Li et al. [4] suggest that in silico methods or theoretical bounds might be used instead of phantom studies and potentially even some clinical studies, thereby reducing the complexity of an overall QI assessment. Continued work by QIBA, academia, industry, and CDRH is essential for developing a truly robust and least burdensome assessment framework and achieving the full potential of QI to improve clinical decision making.

      Nicholas Petrick

      Nicholas Petrick, PhD, MS, is Acting Director for the Division of Imaging, Diagnostics and Software Reliability within the U.S. Food and Drug Administration’s Center for Devices and Radiological Health, and is an FDA Senior Biomedical Research Scientist. He earned his BS degree from Rochester Institute of Technology in Electrical Engineering and his MS and PhD degrees from the University of Michigan in Electrical Engineering Systems. Dr. Petrick’s areas of interest include quantitative imaging and computer-aided diagnosis and x-ray imaging, with an emphasis on validation and assessment methods for these technologies. He currently serves as the FDA representative to the QIBA Steering Committee and is a member of the CT Volumetry Biomarker Committee.


      1. Philips Medical Systems mDIXON-Ouant 2014 [cited 2016 March 18]; U.S. Food and Drug Adminstration 510(k) Summary (K133526)]. http://www.accessdata.fda.gov/cdrh_docs/pdf13/K133526.pdf.
      2. HeartFlow FFTCT V1.4 2013 [cited 2016 March 16]; U.S. Food and Drug Adminstration De Novo Summary (DEN130045)]. http://www.accessdata.fda.gov/cdrh_docs/reviews/DEN130045.pdf.
      3. Medical Device Development Tools - Draft Guidance for Industry, Tool Developers, and Food and Drug Administration Staff. 2013 [cited 2015 March 17]; http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm374427.htm.
      4. Li, Q., et al., Volume Estimation of Low-contrast Lesions with CT: a Comparison of Performances from a Phantom Study, Simulations and Theoretical Analysis. Physics in Medicine and Biology, 2015. 60(2):671-688; http://iopscience.iop.org/article/10.1088/0031-9155/60/2/671/met


      Each issue of QIBA Newsletter features a link to a dynamic search in PubMed, the National Library of Medicine's interface to its MEDLINE database. Link to articles on: “A Regulatory Perspective on Quantitative Imaging and QIBA.”

      Back to Top




      Key Statistical Issues When Testing QIBA Claims

      By Nancy Obuchowski, PhD

      Whether conducting groundwork projects to evaluate a QIBA Profile Claim, writing the conformance section of a Profile, or preparing to confirm your claim in a multi-site trial, there are five key statistical issues to keep in mind:

      1. For both cross-sectional and longitudinal claims, a good estimate of precision is needed, where “precision” is the closeness in agreement between measured quantity values obtained by replicate measurements [1, 2], and “good estimate” refers to an estimate based on a representative sample with sufficient sample size so that the 95% confidence interval gives a narrow range of values. It cannot be assumed that precision is constant for all subjects (healthy and diseased), disease characteristics (spherical and spiculated lesions), and over the range of measurand values. Thus, a precision Profile is needed, which gives estimates of precision over various characteristics. These precision estimates should not exceed the precision value used in the claim. [3].
      2. For cross-sectional claims, a good estimate of bias is needed, i.e., difference between the average of measurements made on an object and its true value [1]. As with precision, bias cannot be assumed constant across subject and disease characteristics, thus a bias Profile is required. Some QIBA Profiles may not allow any bias, so actors must show their bias is negligible, <5%. Other Profiles may allow some level of bias. A measure of the total error is then used (i.e., bias and precision combined into one measure). Actors with good precision are allowed a little bias such that their total error satisfies the claim [3].
      3. There are two types of longitudinal claims: those requiring the same imaging procedures (e.g., same scanner, image analysis method, and reader) at the two time points, and those allowing different imaging procedures. In the former case, good precision and the property of linearity are the only requirements for testing the claim [3]. In the latter case, a bias Profile is also required; performance is measured by the total error so that imaging procedures with different magnitudes of bias and precision can be used at the two time points.
      4. Linearity is the ability to provide measurements that are directly proportional to the value of the measurand [1]. A regression line of measured values is fit against the true values, and the slope, its 95% confidence interval (CI), and the linear fit are evaluated [3]. Ideally, the slope and CI are close to one so the measured change is an estimate of the true change. Actors with slopes differing from one may need a larger study to obtain a precise estimate of the slope.
      5. Studies designed to test QIBA claims should specify the sample, hypotheses being tested and required sample sizes. Clinical samples are often needed to estimate precision, whereas simulated samples (e.g., phantoms) are used to test bias and linearity. Samples should span the relevant ranges of subject and disease characteristics to provide the precision and bias Profiles. The statistical null hypothesis of “compliance not met” is evaluated, with the alternative hypothesis being “compliance met” [3]. Sample size is a function of the performance value in the claim and the expected performance of the imaging procedure. Sample sizes ≤30 are common [3].

      Addressing each of these five issues will assure the testing of a Profile claim is statistically robust.

      Nancy Obuchowski

      Nancy Obuchowski, PhD, is Vice-Chairman of Quantitative Health Sciences at the Cleveland Clinic and Professor of Medicine at the Cleveland Clinic Lerner College of Medicine of Case Western Reserve University. She is a Fellow of the American Statistical Association. Her research interests include study design and statistical analysis methods for imaging screening and diagnostic tests and imaging biomarkers. She is a member of the QIBA Steering Committee.


      1. Kessler LG, et al. The emerging science of quantitative imaging biomarkers: terminology and definitions for scientific studies and for regulatory submissions. Stat Meth Med Res 2015; 24:9-26.
      2. Raunig D, et al. Quantitative imaging biomarkers: a review of statistical methods for technical performance assessment. Stat Meth Med Res 2015; 24:27-67.
      3. Obuchowski NA, et al. Statistical Issues in Testing Conformance with the Quantitative Imaging Biomarker Alliance (QIBA) Profile Claims. Acad Radiol 2016; 23:496-506.

      Back to Top

      FOCUS ON


      2016 QIBA Annual Meeting

      Approximately 90 people representing radiologists, physicists, industry and government gathered April 13th and 14th in Alexandria, VA, for the 9th QIBA Annual Meeting.

      Plenary sessions included guest speakers from two NIH institutions: NIBIB and NIMH. Three panel discussions focused on topics of cross-modality interest: Claim Guidance, Profile Feasibility Testing, and Profile Conformance.

      Each modality was given the opportunity to report through its respective Coordinating Committee on the activities, accomplishments, and challenges of the past year. A significant part of each day was dedicated to breakout meetings of the various Biomarker Committees to continue work on their Profiles and discuss projects and strategies for deployment and adoption.

      QIBA Annual Meeting

      The QIBA Annual Meeting fosters stakeholder collaboration and information sharing among members from academia, the medical device industry, the pharmaceutical and other business sectors, and government.

      QIBA Biomarker Committees (BC) Continue to Grow

      Within the last six months, the number of QIBA BCs has grown from eight to12, with the most recent addition of a Contrast Enhanced Ultrasound (CEUS) BC. In addition, a number of Task Forces have been created to support Profile development.

      • Click here for a complete list of active Biomarker Committees.
      • Click here for the most recent organization chart.

      QIBA meeting summaries, the QIBA Newsletter and other documents are available in two locations:

      Please contact QIBA@rsna.org for more information.

      Back to Top

      QIBA and QI/Imaging Biomarkers in the Literature


      This list of references showcases articles that mention QIBA, quantitative imaging, or quantitative imaging biomarkers.

      In most cases, these are articles published by QIBA members or relate to a research project undertaken by QIBA members that may have received special recognition. New submissions are welcome and may be directed to QIBA@rsna.org.

      Back to Top