Warning! OUTDATED BROWSER DETECTED!   Please update your browser immediately for a better experience on this website. Learn More
    • QIBA Newsletter Header

      January 2015 • Volume 7, Number 1

      In This Issue:
      The Time is Now for a PET Amyloid QIBA Profile
      By: ANNE M. SMITH, PHD
      PubMed Search on Quantitative PET Amyloid Imaging
      Comparison Challenges Put Algorithms to the Test
      Improve the value and practicality of quantitative imaging biomarkers by reducing variability across devices, patients and time.
      QIBA Wiki
      Contact Us
      Daniel C. Sullivan, MD
      RSNA Science Advisor
      The Time is Now for a PET Amyloid QIBA Profile
      Anne M. Smith, PhD

      To address the needs of the aging baby boomers at risk for Alzheimer Disease (AD), the FDA has approved three PET 18-F amyloid-β tracers in the last two years, with others in the pipeline, including promising tau protein tracers. Concurrently, economic policies have changed the practice, distribution, and reimbursement of healthcare. Rather than working in opposition, these forces can be harnessed to accelerate PET amyloid-β and tau imaging in the clinic by all stakeholders working toward the goal of harmonizing PET acquisition, analysis and results reporting—sooner rather than later.

      The current clinical use of PET amyloid-β imaging is a binary visual interpretation of the images—positive if amyloid burden is present (i.e., PET tracer accumulation), or negative if not (no tracer accumulation). In the case of a negative result, neurologists can rule out AD as the likely cause of the patient’s symptoms. A quantitative measure of amyloid burden could potentially have two benefits that would add to the binary visual result:

      1. An additional quantitative prognosis for disease stratification.
      2. A quantitative amyloid burden response measured before and after treatment.

      With the first potential benefit, cases at the ends of the spectrum are often easily interpreted and are supported by other information from the patient’s workup. The most difficult cases are those that fall in the “gray area” of the spectrum (see Figure 1), where an additional positive quantitative assessment could give more confidence to the radiologist and neurologist. This additional information could also be useful for optimizing patient selection for a given therapy or clinical trial.

      Gray Area

      Figure 1


      The second potential benefit is analogous to using PET-FDG to monitor an individual patient’s treatment response to a cancer therapy. The change in amyloid burden could be quantified between a pre- and post-treatment scan. A decrease in amyloid burden may indicate that a treatment is having a desired effect and should be continued.

      Uniting Around PET Amyloid-β

      Unlike PET-FDG which has decades of clinical trials, scientific research and clinical experience to build on for probability thresholds and significant changes after therapy, PET-amyloid-β and tau imaging are still in their infancy. Having multiple tracers adds to the complexity since each has its own recommended protocol in respect to acquisition, processing and evaluation. Although we are still years away from building the same level of confidence in quantitative results as the PET-FDG standardized uptake value, we, as a community, can accelerate the process by using the PET-FDG experience and accumulating large amounts of PET amyloid data for retrospective analysis.

      While the CMS decision in September 2013 not to offer reimbursement coverage for PET amyloid scans with the exception of patients enrolled in CMS-approved clinical trials was a disappointment to some, it is an opportunity for us in the PET community to start building our experience with PET amyloid-β and tau and store the pertinent information in image databases accessible for further clinical research. However, a database is only as good as the data it contains. A necessary first step is to define a workflow protocol that will ensure a harmonized approach to patient preparation, data acquisition, data analysis, data reporting and data storage—the exact task that the QIBA PET Amyloid Biomarker Committee is undertaking at the moment. Join us!

      Anne SmithAnne Smith, PhD, is a Staff Systems Engineer for Siemens Molecular Imaging in Knoxville, Tennessee. She has worked in the medical imaging field for 25 years and has a passion for commercializing mature scientific discoveries for broad clinical use. Her scientific interests focus on PET image quality, image analysis and translational imaging of new biomarkers. An active member of QIBA, Dr. Smith serves as a co-chair of the PET-Amyloid Biomarker Committee, is a member of the FDG-PET Biomarker Committee and is an industry representative on the QIBA Steering Committee.


      Each issue of QIBA Newsletter features a link to a dynamic search in PubMed, the National Library of Medicine's interface to its MEDLINE database. Link to articles on:Quantitative PET Amyloid Imaging.”
      Back to Top
      Comparison Challenges Put Algorithms to the Test
      Andrew J. Buckler, MS

      A number of the biomarkers selected by QIBA utilize sophisticated image processing algorithms that process the acquired data to produce the resulting readings. Evaluating the relative performance of these algorithms and the software systems within which they are implemented is difficult due to the large number of suppliers as well as the needs for rich data resources on which to base tests.  The performance results are needed both to inform QIBA Profile claims and specifications as well as to ascertain whether candidate systems comply with them. 

      As one solution, the CT Volumetry Biomarker Committee has explored the logistics and infrastructure for conducting public challenges, while collecting meaningful data to aid in its completion of the Profile. The first challenge focused on intra- and inter-algorithm bias and variability using phantom data sets.  Ten different algorithms—both semi-automated and fully-automated—were applied to CT scans of synthetic lung tumors in anthropomorphic phantoms to characterize their performance individually, and to estimate inter-algorithm variability collectively. The goal was to determine how the wide variety of available algorithms performed on tumors meeting specifications outlined in the QIBA Profile as well as others which did not. Eight of the algorithms met the QIBA claim when applied to tumors described by the QIBA Profile, while two did not.  For tumors outside the scope of the Profile, all 10 struggled, though to varying degrees.

      More recently, the Committee organized a challenge using CT scans of non-small cell lung cancer patients in a test-retest protocol. Some organizations from the first challenge also participated in the second, along with new organizations which had not previously participated. Eight of 12 participating algorithms performed sufficiently to meet QIBA compliance as judged on this data set. Based on these results, change in tumor volume can be measured with confidence to within +9% using any of the eight compliant algorithms. This figure was needed to separate variability in analysis from other parts of the workflow process. A rich set of other performance data was collected including detailed assessment of not only the computed result but also of the actual segmented region characteristics themselves.

      Perhaps the greatest utility of this work and public algorithm challenges in general—from the point of view of a group or a company seeking to commercialize analysis software for tumor volumetry—is the performance of their algorithm compared with other similar algorithms.  Individualized reports inclusive of raw data and intermediate analysis results have been provided to participants in both challenges. 

      The value of the results is highest to those who contributed actual segmentation boundaries, given the ability to distinguish true positives and negatives from false positives and negatives at a level of granularity allowing algorithm optimization.

      This data is instrumental to inform the definition of a performance standard for CT lesion volumetry algorithms. Participating groups also benefit in that algorithm weaknesses are identified. This is greatly aided by inclusion of the segmentation results, which provides insight into when algorithms produce outlier segmentations in need of correction.

      Finally, the manner in which these tests are run and the data collected has implications regarding the interpretation and use of metrics computed and reported.  For example, execution of these tests by a trusted third-party on sequestered data sets may increase their utility.

      Andrew BucklerAndrew J. Buckler, MS, is President and CEO of Elucid Bioimaging Inc. Mr. Buckler serves as Program Director for QIBA and as Scientific Liaison for the CT Biomarker Committees. At Elucid, he is developing novel techniques for the detection and characterization of quantitative imaging in cardiovascular and cancer indications.

      Back to Top
      QIBA Biomarker Committees

      The ongoing work of the Biomarker Committees is posted on the QIBA wiki page: http://qibawiki.rsna.org. New participants in QIBA Biomarker Committees are always welcome; please contact QIBA@rsna.org for more information.

      Back to Top
      Articles are divided into two categories:

      1. Articles that are generated by Quantitative Imaging Biomarkers Alliance (QIBA) research teams
      2. Articles that reference QIBA

      These are articles published by QIBA members, or ones that relate to a research project undertaken by QIBA members that may have received special recognition. New submissions are welcome and may be directed to QIBA@rsna.org

      Back to Top