Detection and Assessment of Abnormality in Medical Images

1
Detection and Assessment of Abnormality in
Medical Images

• MS Thesis Presentation
• Candidate K Sai Deepak
• Adviser Prof. Jayanthi Sivaswamy

Center for Visual Information Technology IIIT
Hyderabad India
31-March-2012
2
Agenda

• Computer Aided Diagnosis
• Modes of Healthcare
• CAD in Primary Care (examples)
• Disease Screening
• CAD in Disease Screening
• Challenges for existing CAD
• Proposed Methodology
• Detecting Abnormality Instead of Disease
• Detection of Lesions using Motion Patterns
• Detection and Assessment of Retinopathy
• Diabetic Macular Edema
• Method
• Experiments and Results
• Detection of Multiple Lesions
• Classification of Lesions in Mammograms
• Mammographic Lesions
• Experiments and Results

Source of all the figures are explicitly
mentioned in the MS Thesis
3
PART I Computer Aided Diagnosis
4
Computer Aided Diagnosis (CAD)
Computer Aided Diagnosis

• Aid of computers in the process of diagnosis
• Computer aided diagnosis (CAD) has become one of
  the major support systems assisting medical
  experts in diagnosis through images
• CAD tools are used for measurement, display and
  analysis of both the structural and functional
  aspects of the body from images

5
CAD with Images
Computer Aided Diagnosis

• Visualization enhancement for visual analysis
  (Ex. Windowing, MIP, MAP, AIP, Zoom, Contrast
  Inversion etc.)
• Detection detect the presence of disease
  manifestation
• Localization and Segmentation Localize or
  segment the spatial regions containing disease
  manifestation
• Other utilities can be used for measurement of
  various structures from images (length, volume
  etc. )

6
Healthcare Primary Care and Disease Screening
Computer Aided Diagnosis
Point of Consultation in basic healthcare Patients
with Symptoms arrive Undergo specialized tests
if required for Diagnosis Treatment is planned
based on Diagnosis
Performed on Public health initiative Most
patients have no disease symptoms Detection is
performed by a trained professional Referred to
expert on positive detection
Secondary and Tertiary Care Centers are where
patients usually visit on referral for advanced
care
7
CAD in Primary Care
Computer Aided Diagnosis

• Traditionally CAD has been used in Primary Care

8
CAD in Primary Care
Computer Aided Diagnosis

• Patient visits the doctor with a complaint
• If required, the patient is then referred by the
  doctor for specific imaging in order to diagnose
  the problem
• Acquired images are analyzed by the experts
  (Ophthalmologist, Radiologist) to arrive at a
  diagnosis
• The diagnosis report is used by doctor for
  planning treatment

9
PART II Disease Screening
10
Disease Screening
Disease Screening

• Disease screening is performed at specific
  community healthcare centers to prevent ensuing
  mortality and suffering from chronic ailments
• Challenges Geographical reach, Disease awareness
  and Social barriers and Availability of experts
  are common in screening
• Tele-radiology provides significant help but the
  work load of a medical expert increases
  significantly due to large number of patients
  participating in population screening
• Diabetic Retinopathy and Breast Cancer screening
  are already conducted or being adopted in several
  countries and is the focus of this work

11
CAD in Disease Screening
Disease Screening

• Existing CAD tools use a disease centric approach
  for disease detection
• It requires application of several methods/tools
  for detecting all the possible lesions in a
  disease
• Multiple CAD tools are used for identifying
  different Diabetic Retinopathy (DR)
  manifestations
• Existing CAD systems are not able to meet the
  needs of disease screening in Diabetic
  Retinopathy 1
• Poor sensitivity of disease detection
• Large number of normal patients are detected as
  abnormal

1 M. D. Abramoff, M. Niemeijer, M. S.
Suttorp-Schulten, M. A. Viergever, S. R. Russell,
and B. van Ginneken. Evaluation of a system for
automatic detection of diabetic retinopathy from
color fundus photographs in a large population of
patients with diabetes. Journal of Diabetes Care,
31193198, 2007.
12
Summary of Challenges
Disease Screening

• Existing CAD tools use a disease centric approach
  for detection and segmentation of disease
• In Screening most of the patients are normal
  (80-90 for DR BC)
• Multiple tools result in cascading effect of
  detected FPs
• Doctors spend a lot of time in rejecting normal
  patients
• Other challenges in disease centric approach
• Illumination and Contrast
• Tissue Pigmentation
• A disease centric CAD system has to robustly
  learn all possible manifestations of a disease
  which is challenging
• Patients with diseases outside the purview of
  screening are ignored
• referral could be useful for a patient suffering
  non DR disease detected in DR screening

13
Other Challenges Disease Vs Normal Background
Disease Screening
14
PART III Proposed Methodology
15
Detecting Abnormality instead of Disease
Proposed Methodology

• Non conformance to expected behaviour (normal) in
  the data is considered as abnormality
• Features of normal medical images can be used to
  model expected normal behaviour
• Abnormality detection is relevant in disease
  screening where detecting the presence of
  abnormality is of initial interest
• Retinal image screening for detecting Diabetic
  Retinopathy
• Mammographic screening for detecting malignancy
  of lesions

16
Two Stage Methodology for CAD
Proposed Methodology

• Stage 1- Detection of abnormality
• Derive motion pattern for detection of lesions
• Extract relevant features to represent normal
  sub-space
• Detect outliers as abnormal
• Stage2-Assessment of abnormality
• Derive relevant features based on domain
  knowledge from abnormal cases
• Determine the severity of disease

17
Two Stage Methodology for CAD
Proposed Methodology

• Stage 1- Detection of abnormality
• Only normal cases are required for disease
  detection
• Variations observed in the normal cases are
  captured by the normal feature sub-space
• Single point of control on the permitted figure
  of false alarms
• Stage2-Assessment of abnormality
• Fewer normal cases to be examined by experts

18
Motion Pattern Detecting Localized Lesions
Proposed Methodology

• Motivation - Effect of motion on human visual
  system and detectors in camera
• Spatial/temporal averaging of intensities in
  retina
• Smearing of intensities corresponding to moving
  object is observed in images acquired with camera
• Inducing motion in images
• Lesions can be observed as a set of localized
  pixels with contrast against background
• A smear of pixel along the direction of motion
  can be observed in motion pattern
• Spread and extent of lesions in motion pattern
  depends on the sampling rate at each location and
  duration of motion
• Contrast of the spatially enhanced lesions in
  motion pattern relies on the coalescing function
• Motion pattern on Background
• Uniformity in motion pattern for textured
  background can be observed

Original Image (Uniform Background)
Rotational Motion Pattern
19
PART IV Detection and Assessment of Macular
Edema
20
Macular Edema Detection and Assessment
-Showcase 1- Retinopathy

• Diabetic Macular Edema (DME) is a sight
  threatening condition that occurs due to diabetic
  retinopathy
• DME requires immediate referral to
  Ophthalmologists
• Presence of Hard Exudates is used as an indicator
  of DME during retinal disease screening

21
Existing Approaches in DME Detection
-Showcase 1- Retinopathy

• Several local and global schemes have been
  proposed for DME detection
• Local Schemes
• local schemes try to successfully segment or
  localize the exudate clusters
• Techniques including adaptive intensity
  thresholding, background suppression (median
  filtering, morphology), color and edge detection
  have been proposed
• several normal pixels are also detected as
  candidates in normal images increasing the number
  of false alarms in the system
• Global Schemes
• global schemes try to ensure that at least the
  brightest pixels corresponding to HE in the image
  are detected
• Techniques based on intensity thresholding, edge
  strength, and visual words using features on SIFT
  keypoints have been used to classify images

22
Proposed Workflow
-Showcase 1- Retinopathy

• Steps
• Landmark Detection and Region of Interest
  Extraction
• Generation of Motion Patterns
• Feature Selection
• Abnormality Detection
• Abnormality Assessment

23
Detection of Landmarks in CFI
-Showcase 1- Retinopathy
Singh, J. and Joshi, G. D. and Sivaswamy, J.
Appearance-based object detection in colour
retinal images. In ICIP, pages 14321435,
2008. G. D. Joshi and J. Sivaswamy and K Karan
and S. R. Krishnadas. Optic disk and cup boundary
detection using regional information. ISBI, pp.
948951, 2010.
24
Selection of ROI
-Showcase 1- Retinopathy
25
Motion Pattern Rotational Motion
-Showcase 1- Retinopathy
Effect of sampling rate on motion pattern
(decreasing rotation steps)-

• Coalescing Function
• Mean - Arithmetic mean of all samples were
  taken
• Extrema Maximum or Minimum of all samples are
  taken at each pixel location

26
Selection of Motion Pattern
-Showcase 1- Retinopathy
normal
abnormal

• A normal retinal image was created by averaging
  the green channel of 400 retinal images
• The abnormal retina is modeled by adding a
  bright lesion to emulate HE

effect of abnormality (lesion) on retinal
background can be observed as change in local
information with respect to the motion pattern of
normal retina
- motion pattern
- Gradient magnitude of motion pattern
- Shannons entropy
27
Selection of Parameters Class Discriminability
-Showcase 1- Retinopathy
Size of normal retina 150150 Neighborhood size
77
28
Motion Pattern for Edema Detection
-Showcase 1- Retinopathy

• A circular ROI is determined around macula and
  the Optic disc is masked to avoid false positives
• Rotational motion is induced in the green channel
  image
• Maxima is used as the coalescing function
• Features derived on motion pattern are used for
  learning the normal sub-space and detecting
  abnormality

29
More Motion Patterns
-Showcase 1- Retinopathy
Sample ROIs and Motion Pattern (S- Subtle Hard
Exudates)
Normal ROI
Abnormal ROI
30
Feature Extraction
-Showcase 1- Retinopathy

• To effectively describe motion pattern, we use a
  descriptor derived from the Radon space

• The desired feature vector is obtained by
  concatenating 6 projections (0-180 degrees)
• Each projection has 6 bins resulting in a
  feature vector of length 36

31
Abnormality Detection
-Showcase 1- Retinopathy

• PCA Data Description
• The eigenvectors corresponding to the covariance
  matrix of the training set is used to describe
  the normal subspace
• Feature vector for a new case is projected to
  this subspace (first 6 eigen vectors)
• Residual e is defined as,

• Classification between normal and abnormal cases
  is then performed using an empirically determined
  threshold on e

32
Detection Performance (ROC Curves)
-Showcase 1- Retinopathy
Receiver Operating Characteristic curve

• DMED - 122 images
• Normal - 68
• Abnormal 54
• Normal images used for training - 18
• MESSIDOR 400 images
• Normal - 274
• Abnormal 126
• Immediate referral - 85
• Normal images used for training 74
• Diaretdb0 db1 122 images
• Normal 25
• Abnormal - 97
• Combined Dataset 644 images
• Normal 367
• Abnormal - 277

33
Comparison against Disease Centric Methods
-Showcase 1- Retinopathy

• DMED
• Normal - 68
• Abnormal 54
• Normal images used for training - 18

MESSIDOR Normal - 274 Abnormal 126 Normal
images used for training 74
23 L. Giancardo, F. Meriaudeau, T. P.
Karnowski, Y. Li, K. W. Tobin Jr, and E. Chaum.
Automatic retina exudates segmentation without a
manually labelled training set. IEEE ISBI, pages
1396 1400, April 2011.
2 C. Agurto, V. Murray, E. Barriga, S. Murillo,
M. Pattichis, H. Davis, S. Russell, M. Abramoff,
and P. Soliz. Multiscale am-fm methods for
diabetic retinopathy lesion detection. IEEE TMI,
29(2)502 512, feb. 2010.
34
Detection of subtle hard exudates
-Showcase 1- Retinopathy
35
Assessment of Severity
-Showcase 1- Retinopathy

• Macula is devoid of significant vasculature
• It is characterized by rough rotationally
  symmetry

36
Assessment of Severity
-Showcase 1- Retinopathy
Dataset MESSIDOR
The threshold is expressed as a percentage (p) of
the symmetry measure S of normal ROIs used in the
abnormality detection task
37
Detection of Multiple Abnormalities
-Showcase 1- Retinopathy
Abnormalities Hemorrhage, Hard Exudates, Drusen
Dataset DMED,MESSIDOR and Diaretdb0
Normal Cases - 362 Abnormal Cases - 302
38
PART V Classification of Lesions in Mammograms
39
Assessment of Mammographic Lesions
-Showcase 2- Breast Cancer

• Breast cancer is responsible for about 30 percent
  of all new cancer cases with a high mortality
  rate in women
• Screening for its early detection with mammograms
  has been explored for more than 3 decades now
  with moderate success
• Correct classification of anomalous areas in the
  mammograms through visual examination is
  challenging even for experts

40
Existing Approaches in Mammogram Analysis
-Showcase 2- Breast Cancer

• 1- Lesions are first detected from mammograms
• 2- Malignancy of detected lesions are identified
  using several texture and shape features
• Typical features used
• size
• shape
• density
• Smoothness of borders
• Brightness and contrast
• local intensity distribution
• The feature space is very large and complex due
  to the wide diversity of the normal tissues and
  the variety of the abnormalities

41
Classification of Mammographic Lesions
-Showcase 2- Breast Cancer

• Given a lesion, its malignancy is of question
• Features derived over motion pattern is used for
  learning the behavior of benign class
• Any deviation in lesion property is identified as
  a sign of malignancy

Benign lesions
Malignant lesions
42
Motion Pattern Class Discriminability
-Showcase 2- Breast Cancer

• Three sample benign and malignant lesions were
  selected
• Motion pattern was applied using rotation and
  translation to analyze class discriminability
  between benign and malignant class
• Maximum and Mean are the coalescing functions
  used

43
Classification Performance (ROC Curve)
-Showcase 2- Breast Cancer

• Mini-MIAS
• Benign - 68
• Malignant 51
• Benign lesions for training - 20

• An evaluation of the proposed scheme for
  learning normal subspace was conducted using KNN
  classifier
• The value of K was considered as 3 for
  computing the sensitivity and specificity values
  in the classification tasks
• An ROC curve is drawn by varying the normalized
  Euclidean distance from 0-1

44
Conclusion

• We identified and listed the challenges in image
  based disease screening for diabetic retinopathy
  and breast cancer
• We proposed and evaluated a method for
  abnormality detection and assessment
• a hierarchical approach to the problem of
  abnormality detection
• Evaluation of the proposed hierarchical approach
  has been performed
• on several publicly image datasets of CFI and
  mammograms
• improvement in the disease detection performance
  over methods in literature

45
Acknowledgement

• This work is dedicated to my Parents and Teachers
• Extremely grateful to Prof. Jayanthi Sivaswamy
  for giving me the opportunity to pursue MS by
  research
• Thankful to all lab mates in CVIT for their
  support
• Guidance of Gopal and Mayank was extremely
  valuable
• Debates and discussion with Sandeep, Kartheek and
  Saurabh were always insightful

46
Publications

• 1. Patents
• (a) Jayanthi Sivaswamy, N V Kartheek Medathati, K
  Sai Deepak, A System for generating Generalized
  Moment Patterns, Submitted to Indian Patent
  Office, 2010 (Application Number 3939-CHE-2010)
• 2. Papers
• Conference
• (a) K Sai Deepak, Gopal Datt Joshi, Jayanthi
  Sivaswamy, Content-Based Retrieval of Retinal
  Images for Maculopathy, ACM International Health
  Informatics Symposium, November, 2010
• Journal
• (a) K Sai Deepak, N V Kartheek Medathati and
  Jayanthi Sivaswamy, Detection and Discrimination
  of disease related abnormalities, Elsevier
  Pattern Recognition 2011 (In Press)
• (b) K Sai Deepak, Jayanthi Sivaswamy, Automatic
  Assessment of Macular Edema from Color Retinal
  Images, IEEE Transactions on Medical Imaging 2011

47
Supplementary Slides
48
Imaging Modalities
Computer Aided Diagnosis
Optical Imaging - Ophthalmology
X-ray Imaging - Mammography

• High resolution optical camera
• Pupil may be dilated before imaging
• Pixel resolutions typically range from 0.5K to
  2K2K
• Radiometric resolution is typically 8 bits per
  channel

• Low energy X-ray scanner
• Displays change of density among tissues
• Pixel resolutions can range from 1K2 to 3K2
• Radiometric resolution 8-12 bits

49
CAD in Disease Screening Diabetic Retinopathy
Disease Screening
Hemorrhage Detection
FP1
Exudate Detection
FP2
Neovascularization Detection
FP3
Microaneurysms Detection
FP4
Maximum False alarms in disease centric approach
FP1 FP2 FP3 FP4
50
CAD Retinopathy (Color Fundus Image)
Disease Screening
51
CAD Breast Lesions (Mammograms)
Disease Screening
Benign Lesion
Malignant Lesion
52
Illumination and Contrast
Disease Screening

• Presence of one or more of additive bias,
  multiplicative bias and difference in brightness
• These variations often increases the complexity
  of modeling the normal background especially when
  there can be several other structures present in
  the normal image

53
Tissue Variation (Pigmentation Density)
Disease Screening

• Tissue characteristics for the same structure can
  vary across race and often across patients,
  within a race.
• This variation manifests as differences in
  intensity, hue and/or pigmentation
• These variations can be significant enough for an
  automated disease detection technique to classify
  an image as abnormal

54
CAD with Images - Visualization
Computer Aided Diagnosis
52 year old Patient with Back Pain
MAP of Sagittal view Bones appear bright in X-ray
Windowing Tissues of varying densities can be
examined
55
CAD with Images - Detection
Computer Aided Diagnosis
Normal Retina
Abnormal Retina
56
CAD with Images Segmentation
Computer Aided Diagnosis
Vessels Segmented
Original Image
57
Feature Extraction
-Showcase 1- Retinopathy

• To effectively describe motion pattern, we use a
  descriptor derived from the Radon space

58
PCA DD
-Showcase 1- Retinopathy
Wdk is a matrix of first k eigen vectors
Vector X is projected on the new sub-space
Xproj W(WTW)-1 WX
Re-construction error e(X) is computed as,
e(X) X - Xproj2