The Lung Image Database Consortium LIDC:

1
The Lung Image Database Consortium (LIDC)

• Fundamental Issues for the Creation of a Resource
  for the Image Processing Research Community

2
Exhibit Learning Objectives

• Learn about the LIDCs goals and methods for
  creating
• a publicly available database,
• for the development, training, and evaluation of
  Computer-Aided Diagnosis (CAD) methods,
• for lung cancer detection and diagnosis using
  helical CT.

3
Learning Objectives

• Learn about challenges in interpreting CT image
  data sets for the detection and diagnosis of lung
  cancer

4
Learning Objectives

• Learn about the intricacies of establishing
  spatial truth for lesion location and boundary.

5
The Challenge Lung Cancer

• Cancer of the lung and bronchus is the leading
  fatal malignancy in the United States.
• Five-year survival is low, but treatment of
  early-stage disease improves chances of survival
  considerably.

6
The Challenge Lung Cancer

• Given
• Promising results from recent studies involving
  the use of helical computed tomography (CT) for
  the early detection of lung cancer
• As well as rapid developments in Multi-detector
  CT (MDCT) technology which provide for the
  possibility of the detection of smaller lung
  nodules and offers a potentially effective tool
  for earlier detection.
• There has been an increased interest in
  computer-aided diagnosis (CAD) techniques applied
  to CT imaging for lung cancer to assist
  radiologists with their decision-making.

7
The NCI Response Forming the LIDC
To stimulate research in the area of CAD, the
National Cancer Institute (NCI) formed a
consortium of institutions to develop the
standards and consensus necessary for
constructing an image database resource of
thoracic helical CT images.
8
Motivation

• The development of CAD methods by the imaging
  research community would be facilitated and
  enhanced through access to a repository of CT
  image data

9
Motivation

• The development of CAD methods by the imaging
  research community would be facilitated and
  enhanced through access to a repository of CT
  image data
• (1) It would provide data to researchers without
  access to clinical images

10
Motivation

• The development of CAD methods by the imaging
  research community would be facilitated and
  enhanced through access to a repository of CT
  image data
• (1) It would provide data to researchers without
  access to clinical images
• (2) It would also allow for meaningful
  comparisons of different CAD methods

11
The LIDC
This consortium - called the Lung Image Database
Consortium (LIDC) - seeks to establish standard
formats and processes by which to manage lung
images and the related technical and clinical
data that will be used by researchers to develop,
train and evaluate CAD algorithms for lung cancer
detection and diagnosis.
12
Member Institutions

• Five institutions were selected to form the
  Lung Image Database Consortium (LIDC)

13
Member Institutions
Cornell University UCLA University of
Chicago University of Iowa University of
Michigan
14
Steering Committee
Cornell University David Yankelevitz Anthony P.
Reeves UCLA Michael F. McNitt-Gray Denise R.
Aberle University of Chicago Samuel G. Armato
III Heber MacMahon University of Iowa Geoffrey
McLennan Eric A. Hoffman University of
Michigan Charles R. Meyer Ella
Kazerooni NCI Laurence P. Clarke Barbara Y.
Croft
15
Contributing Participants
Claudia Henschke, Cornell David Gur, U. of
Pittsburgh Robert Wagner, FDA Nicholas Petrick,
FDA Lori Dodd, NCI Ed Staab, NCI Daniel Sullivan,
NCI Houston Baker, NCI Carey Floyd, Duke Aliya
Husain, U. of Chicago Matthew Brown,
UCLA Christopher Piker, U. of Iowa Peyton Bland,
U. of Michigan Andinet Asmamaw, Cornell
Richie Pais, UCLA Antoni Chan, Cornell Gary
Laderach, U. of Michigan Junfeng Guo, U. of
Iowa Charles Metz, U. of Chicago Roger Engelmann,
U. of Chicago Adam Starkey, U. of Chicago Jim
Sayre, UCLA Mike Fishbein, UCLA Andy Flint, U. of
Michigan Barry DeYoung, U. of Iowa Brian Mullan,
U. of Iowa Madeline Vazquez, Cornell
16
Mission
The mission of the LIDC is the sharing of lung
images, especially low-dose helical CT scans of
adults screened for lung cancer, and related
technical and clinical data for the development
and testing of computer-aided detection and
diagnosis technology
17
Principal Goals
To establish standard formats and processes for
managing thoracic CT scans and related technical
and clinical data for use in the development and
testing of computer-aided diagnostic algorithms.
18
Principal Goals
To establish standard formats and processes for
managing thoracic CT scans and related technical
and clinical data for use in the development and
testing of computer-aided diagnostic
algorithms. To develop an image database as a
web-accessible international research resource
for the development, training, and evaluation of
computer-aided diagnostic (CAD) methods for lung
cancer detection and diagnosis using helical CT.
19
The Database

• The database will contain
• a collection of CT scan images
• a searchable relational database

20
Fundamental Issues for the LIDC
21
LIDC Challenge 1 - Define a Nodule

• Though at first this seems trivial, the LIDC had
  significant discussion about what to include and
  what not to include as a nodule
• A Nodule is part of a spectrum of focal
  abnormalities.
• This spectrum includes scars, cancers, benign
  lesions, calcified lesions, etc.

22
What is a Nodule?
nodule any pulmonary or pleural lesion
represented in a radiograph by a sharply defined,
discrete, nearly circular opacity 2-30 mm in
diameter from the Fleischner Society's
Glossary of Terms for Thoracic Radiology (AJR
1984)
23
(No Transcript)
24
What is a Nodule?
nodule round opacity, at least moderately well
marginated and no greater than 3 cm in maximum
diameter from the Fleishner Society's
Glossary of Terms for CT of the Lungs (Radiology
1996)
25
(No Transcript)
26
Is this a Nodule?
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
NOTE This is the slice which was shown earlier
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
What is a Nodule?

• a spectrum of abnormalities

Calcified Nodule
Spiculated Nodule
Scar
49
LIDC Challenge 1 - Define a Nodule

• LIDC Response is to develop a Nodule Visual
  Library using
• Cases that ARE in Nodule portion of spectrum
• Cases that are OUTSIDE Nodule portion of spectrum
• Classification by Thoracic Radiologists
• In Development Now
• Expected Completion Feb 2004.

50
Truth Assessment

• Investigators will require truth information

51
Truth Assessment

• Investigators will require truth information
• location of nodules

52
Truth Assessment

• Investigators will require truth information
• location of nodules
• spatial extent of nodules

53
Truth Assessment

• Investigators will require truth information
• location of nodules
• spatial extent of nodules
• Spatial Truth will be estimated by Radiologic
  Truth

54
LIDC Challenge 2Define the Boundary of a Nodule

• Though it seems that the boundary of a nodule
  should be easy to define, we (and others) have
  found that there is considerable inter-reader
  variability in defining the boundary of a nodule.
  
• This is difficult enough with a solid nodule, but
  even more difficult with spiculated nodules,
  ground glass nodules or non-solid nodules.

55
SPICULATED NODULE Instructions to Thoracic
Radiologists were Draw the Boundary of the
Nodule
56
SPICULATED NODULE Expert Number 1 Contour
57
SPICULATED NODULE Expert Number 2 Contour
58
SPICULATED NODULE Comparison of Contours
59

Reader and Method Variability in Drawing
Boundaries

• Five radiologists using 3 drawing methods
• One manual 3-panel (3D) drawing method
• Two different semiautomatic 3D methods

60
Case 5, Slice 19
61
Radiologist 1 - Method 1
62
Radiologist 1 - Method 2
63
Radiologist 1 - Method 3
64
Radiologist 2 - Method 1
65
Radiologist 2 - Method 3
66
Radiologist 3 - Method 1
67
Radiologist 3 - Method 2
68
Radiologist 3 - Method 3
69
Radiologist 4 - Method 1
70
Radiologist 4 - Method 2
71
Radiologist 4 - Method 3
72
Radiologist 5 - Method 1
73
Radiologist 5 - Method 3
74
Create a Probabilistic Description of Nodule
Boundary

• For each voxel, sum the number of occurrences
  (across reader and method combinations) that it
  was included as part of the nodule
• Create a probabilistic map of nodule voxels
• Higher probability voxels are shown as brighter
  lower probability are darker
• Can use apply a threshold and show only voxels gt
  some prob. Value if desired.

75
Probabilistic Description of Boundary
76
Apply Threshold if Desired
77
LIDC Challenge 2Define the Boundary of a Nodule

• Do we need to reconcile these Boundaries?
• LIDCs answer is no.
• LIDC Approach will be to
• Come to a consistent definition of the desired
  boundary (include just solid portion? non-solid
  portion?)
• Assess reader variability of contours
• Construct a probabilistic description of
  boundaries to capture reader variability

78
LIDC Challenge 3- Data Collection Process

• Recent research has demonstrated that Single
  reads are not sufficient At least two and
  perhaps four readers may be required.
• Not practical to do joint readings across five
  institutions
• LIDC Will NOT do a forced consensus read.

79
LIDC Challenge 3- Data Collection Process

• Will do a Two-Staged Process
• Perform independent (Blinded) readings of cases
  by multiple radiologists
• Compile readings and redistribute composite
  readings
• Perform a Second, Unblinded read by same set of
  radiologists
• Each reader can see readings of every other
  reader.
• No forced consensus
• Capture probabilistic detection (e.g. a nodule
  can be identified by 3 of 4 readers) and
  probabilistic contours.

80
LIDC Process Model 2.4 October 2003

• Overview
• Prerequisites
• major data collection steps, and
• data collected at each step.

81
Prerequisites
LIDC Activities
IRB approvals
Actions
Participants Scanned as part of Study/Clinical
Program
Data Collected
82
Prerequisites
LIDC Activities

• Definition of Nodules to be included in Db
• Agreement on Marking /Contouring process

Actions
Radiologist Review Process (described next)
Data Collected
Reader 3
Reader 1
Reader 2
Reader 4

• Identified lesions for each condition Each
  reader, blinded and unblinded read
• Location
• Outline
• Label

83
Blinded Reads Each Reader Reads Independently
(Blinded to Results of Other Readers)
84
Blinded Read for Reader 1 Marks Only One Nodule
Reader 1
85
Blinded Read for Reader 2 Marks Two
Nodules (Note One nodule is same as Reader 1)
Reader 2
86
Blinded Read for Reader 3 Marks Two
Nodules (Note Again, One nodule is same as for
Reader 1)
Reader 3
87
Blinded Read for Reader 4 Did Not Mark Any
Nodules
Reader 4
88
UnBlinded Reads Readings in Which Readers
Are Shown Results of Other Readers
Each Reader Marks Nodules After Being Shown
Results From Other Readers Blinded Reads (Each
Reader Decides to Include or Ignore).
89
Unblinded Read for Reader 1 Now Marks Two
Nodules (Originally only marked one)
Reader 1
90
Unblinded Read for Reader 2 Still Marks Two
Nodules (No Change)
Reader 2
91
Unblinded Read for Reader 3 Now Marks Three
Nodules (Originally only marked two)
Reader 3
92
Unblinded Read for Reader 4 Now Marks Three
Nodules (Originally did not mark any)
Reader 4
93
Composite on Unblinded Reads for All Four Readers
4 Markings
2 Markings
2 Markings
94
Database Implementation

• TASKS COMPLETED (see reports on website)
• Specification of Inclusion Criteria
• CT scanning technical parameters
• Patient inclusion criteria
• Process Model for Data collection
• Determination of Spatial "truth" Using Blinded
  and Unblinded reads
• Development of Boundary Drawing/Contouring Tools

95
Database Implementation

• TASKS ONGOING (expected completion date)
• Definition of Nodule - Nodule Visual Library
  (Feb 04)
• Evaluation of Boundary Variability (Feb 04)
• Inter-Reader Variability
• Boundary Drawing Tool Variability

96
Implementation Timeline

• Task Date Expected
• Specify Complete Data Model Jan 04
• Specify LIDC internal workflow Jan 04
• Data passing, Performing reviews
• Initial implementation, testing workflow
  Jan/Feb 04
• Database Implementation- Start Jan 04
• Database Implementation- Completion Mar/Apr
  04
• Implement Public Interface to Database Apr/May
  04
• PUBLIC ACCESS TO CASES EXPECTED MAY/JUN 04

97
Publications/Presentations

• LIDC Overview manuscript
• In Preparation, submission in 1st Quarter 04
• Assessment Methodologies manuscript
• In Preparation, submission in 1st Quarter 04
• Special Session SPIE Medical Imaging
• Sunday evening Feb 15, 2004

98
To learn more about the LIDC

• Return for CME Category 1 Credit
• Monday Thursday
• 1215 pm to 115 pm
• At these times, LIDC members will be here to
  describe the efforts of the consortium, this
  exhibit and any other questions you might have

99
Return to Exhibit Home Page

• Click Here