Print This Page

Exhibit Learning Objectives:

• Learn about the LIDC's goals and methods for creating: a publicly available database, for the development, training, and evaluation of Computer-Aided Diagnosis (CAD) methods, for lung cancer detection and diagnosis using helical CT.

Learning Objectives:

• Learn about challenges in interpreting CT image data sets for the detection and diagnosis of lung cancer

Learning Objectives:

• Learn about the intricacies of establishing spatial "truth" for lesion location and boundary.

The Challenge: Lung Cancer

• Cancer of the lung and bronchus is the leading fatal malignancy in the United States.
• Five-year survival is low, but treatment of early-stage disease improves chances of survival considerably.

The Challenge: Lung Cancer

• Given:
• Promising results from recent studies involving the use of helical computed tomography (CT) for the early detection of lung cancer
• As well as rapid developments in Multi-detector CT (MDCT) technology which provide for the possibility of the detection of smaller lung nodules and offers a potentially effective tool for earlier detection.
• There has been an increased interest in computer-aided diagnosis (CAD) techniques applied to CT imaging for lung cancer to assist radiologists' with their decision-making.

The NCI Response: Forming the LIDC
To stimulate research in the area of CAD, the National Cancer Institute (NCI) formed a consortium of institutions to develop the standards and consensus necessary for constructing an image database resource of thoracic helical CT images.

Motivation

• The development of CAD methods by the imaging research community would be facilitated and enhanced through access to a repository of CT image data

Motivation

• The development of CAD methods by the imaging research community would be facilitated and enhanced through access to a repository of CT image data
  1. It would provide data to researchers without access to clinical images

Motivation

• The development of CAD methods by the imaging research community would be facilitated and enhanced through access to a repository of CT image data
  1. It would provide data to researchers without access to clinical images
  2. It would also allow for meaningful comparisons of different CAD methods

The LIDC
This consortium - called the Lung Image Database Consortium (LIDC) - seeks to establish standard formats and processes by which to manage lung images and the related technical and clinical data that will be used by researchers to develop, train and evaluate CAD algorithms for lung cancer detection and diagnosis.

Member Institutions

• Five institutions were selected to form the Lung Image Database Consortium (LIDC)

Cornell University
UCLA
University of Chicago
University of Iowa
University of Michigan

Contributing Participants
Claudia Henschke, Cornell
David Gur, U. of Pittsburgh
Robert Wagner, FDA
Nicholas Petrick, FDA
Lori Dodd, NCI
Ed Staab, NCI
Daniel Sullivan, NCI
Houston Baker, NCI
Carey Floyd, Duke
Aliya Husain, U. of Chicago
Matthew Brown, UCLA
Christopher Piker, U. of Iowa
Peyton Bland, U. of Michigan
Andinet Asmamaw, Cornell
Richie Pais, UCLA
Antoni Chan, Cornell
Gary Laderach, U. of Michigan
Junfeng Guo, U. of Iowa
Charles Metz, U. of Chicago
Roger Engelmann, U. of Chicago
Adam Starkey, U. of Chicago
Jim Sayre, UCLA
Mike Fishbein, UCLA
Andy Flint, U. of Michigan
Barry DeYoung, U. of Iowa
Brian Mullan, U. of Iowa
Madeline Vazquez, Cornell

Mission
The mission of the LIDC is the sharing of lung images, especially low-dose helical CT scans of adults screened for lung cancer, and related technical and clinical data for the development and testing of computer-aided detection and diagnosis technology

Principal Goals
To establish standard formats and processes for managing thoracic CT scans and related technical and clinical data for use in the development and testing of computer-aided diagnostic algorithms.

Principal Goals
To establish standard formats and processes for managing thoracic CT scans and related technical and clinical data for use in the development and testing of computer-aided diagnostic algorithms.

To develop an image database as a web-accessible international research resource for the development, training, and evaluation of computer-aided diagnostic (CAD) methods for lung cancer detection and diagnosis using helical CT.

The Database

• The database will contain:
  1. a collection of CT scan images
  2. a searchable relational database

Fundamental Issues for the LIDC

LIDC Challenge #1 - Define a Nodule

• Though at first this seems trivial, the LIDC had significant discussion about what to include and what not to include as a nodule
• A Nodule is part of a spectrum of focal abnormalities.
• This spectrum includes scars, cancers, benign lesions, calcified lesions, etc.

What is a Nodule?
"nodule: any pulmonary or pleural lesion represented in a radiograph by a sharply defined, discrete, nearly circular opacity 2-30 mm in diameter" from the Fleischner Society's Glossary of Terms for Thoracic Radiology (AJR 1984)

Chest x-ray with nodules.

What is a Nodule?
"nodule: round opacity, at least moderately well marginated and no greater than 3 cm in maximum diameter" from the Fleishner Society's Glossary of Terms for CT of the Lungs (Radiology 1996)

Lung CT Scan slice with a nodule.

Lung CT Scan slice - Is this a nodule?; the next images are of the nodule in a series of slices

What is a Nodule?
A spectrum of abnormalities
3 images showing a scar, a spiculated nodule, and a calcified nodule

LIDC Challenge #1 - Define a Nodule

• LIDC Response is to develop a Nodule Visual Library using:
• Cases that ARE in Nodule portion of spectrum
• Cases that are OUTSIDE Nodule portion of spectrum
• Classification by Thoracic Radiologists
• In Development Now
• Expected Completion Feb 2004.

Truth Assessment

• Investigators will require "truth" information

Truth Assessment

• Investigators will require "truth" information
• location of nodules

Truth Assessment

• Investigators will require "truth" information
• location of nodules
• spatial extent of nodules

Truth Assessment

• Investigators will require "truth" information
• location of nodules
• spatial extent of nodules
• Spatial "Truth" will be estimated by "Radiologic Truth"

LIDC Challenge #2
Define the Boundary of a Nodule

• Though it seems that the boundary of a nodule should be easy to define, we (and others) have found that there is considerable inter-reader variability in defining the boundary of a nodule.
• This is difficult enough with a solid nodule, but even more difficult with spiculated nodules, ground glass nodules or non-solid nodules.

Spiculated Nodule - an image of a nodule in a CT slice
Instructions to the thoracic radiologists were "Draw the boundary of the nodule"

A series of images of a spiculated nodule, showing the contours drawn by the Expert Readers

Reader and Method Variability in Drawing Boundaries

• Five radiologists using 3 drawing methods:
• One manual 3-panel (3D) drawing method
• Two different semiautomatic 3D methods

Case 5, Slice 19 - a series of images of CT images of a lung slice showing the readers' outlines around the nodule, created as they used different outlining methods

Create a Probabilistic Description of Nodule Boundary

• For each voxel, sum the number of occurrences (across reader and method combinations) that it was included as part of the nodule
• Create a probabilistic map of nodule voxels
• Higher probability voxels are shown as brighter; lower probability are darker
• Can use apply a threshold and show only voxels > some prob. Value if desired.

Probabilistic Description of Boundary
Showing an image of area enclosed by the combined outlines of all the readers

Apply Threshold if Desired - the same nodule with an outline around the nodule

LIDC Challenge #2
Define the Boundary of a Nodule

• Do we need to reconcile these Boundaries?
• LIDC's answer is no.
• LIDC Approach will be to:
• Come to a consistent definition of the desired boundary (include just solid portion? non-solid portion?)
• Assess reader variability of contours
• Construct a probabilistic description of boundaries to capture reader variability

LIDC Challenge #3-
Data Collection Process

• Recent research has demonstrated that Single reads are not sufficient - At least two and perhaps four readers may be required.
• Not practical to do joint readings across five institutions
• LIDC Will NOT do a forced consensus read.

LIDC Challenge #3-
Data Collection Process

• Will do a Two-Staged Process:
• Perform independent (Blinded) readings of cases by multiple radiologists
• Compile readings and redistribute composite readings
• Perform a Second, Unblinded read by same set of radiologists
• Each reader can see readings of every other reader.
• No forced consensus
• Capture probabilistic detection (e.g. a nodule can be identified by 3 of 4 readers) and probabilistic contours.

LIDC Process Model 2.4
October 2003
Overview

• Prerequisites
• major data collection steps, and
• data collected at each step.

A schematic drawing of the various parts of the process:

Prerequisites

LIDC Activities
Patient/Nodule criteria
CT scan criteria
Labeling vocabulary
Image quality criteria

Actions

Apply Inclusion Criteria:

• IF Meets Scan Parameter Criteria [NOTE: All NLST & ELCAP eligible]
• AND IF Meets Patient Inclusion Criteria
• THEN Include in Db, Label Nodule Characteristics and Score Image Quality

Data Collected
Image data illustrated with a lung CT slice
Non-Image Data
Demographic Data
Labeling
Image Quality score
Scan Classification
Patient Classification

Taking the process apart :

Prerequisites

• Definition of Nodules to be included in Db
• Agreement on Marking /Contouring process

Actions

• Radiologist Review Process (described next)

Data Collected

4 lung images as a sample, with outlines drawn around the nodule
Identified lesions for each condition: Each reader, blinded and unblinded read

• Location
• Outline
• Label

Blinded Reads - Each Reader Reads Independently

Blinded Read for Reader 1 - Marks only one nodule

Blinded Read for Reader 2 - Marks 2 nodules (1 is same as reader 1)

Blinded Read for Reader 3 - Marks 2 nodules (1 is same as reader 1)

Blinded Read for Reader 1 - Did not mark any nodules

UnBlinded Reads - Readings in which readers are shown results of other readers

Unblinded Read for Reader 1 - Now Marks Two Nodules

Unblinded Read for Reader 2 - Still Marks Two Nodules

Unblinded Read for Reader 3 - Now Marks Three Nodules

Unblinded Read for Reader 4 - Now Marks Three Nodules

Composite of Unblinded Reads for All Four Readers

Database Implementation
TASKS COMPLETED (see reports on website):

• Specification of Inclusion Criteria:
• CT scanning technical parameters
• Patient inclusion criteria
• Process Model for Data collection
• Determination of Spatial "truth" Using Blinded and Unblinded reads
• Development of Boundary Drawing/Contouring Tools

Database Implementation
TASKS ONGOING (expected completion date)

• Definition of Nodule - Nodule Visual Library (Feb 04)
• Evaluation of Boundary Variability (Feb 04):
• Inter-Reader Variability
• Boundary Drawing Tool Variability

Implementation Timeline

Publications/Presentations

• LIDC Overview manuscript
• In Preparation, submission in 1st Quarter 04
• Assessment Methodologies manuscript
• In Preparation, submission in 1st Quarter 04
• Special Session SPIE Medical Imaging
• Sunday evening Feb 15, 2004

To learn more about the LIDC:
Return for CME Category 1 Credit:

• Monday - Thursday
• 12:15 pm to 1:15 pm

At these times, LIDC members will be here to describe the efforts of the consortium, this exhibit and any other questions you might have