RegionBased Feature Extraction of Prostate Ultrasound Images: A KnowledgeBased Approach Using Fuzzy

1
Region-Based Feature Extraction of Prostate
Ultrasound Images A Knowledge-Based
ApproachUsing Fuzzy Inferencing

• Eric K. T. Hui
• University of Waterloo, M.A.Sc. Seminar
• Wednesday, November 12, 2003
• 430 PM in DC 2584

2
Outline

• Introduction
• Medical Background
• Related Researches
• Problem Formulation
• Proposed Feature Extraction
• Analysis
• Conclusions
• Future Works
• Questions and Comments

3
Introduction- Prostate Cancer -

• Prostate cancer is the most frequently diagnosed
  cancer in Canadian men
• 18,800 will be newly diagnosed.
• 4,200 will die of it.
• Exact cause remains unknown.
• Early detection is the key in controlling and
  localizing cancerous cells.

4
Introduction- TRUS -

• Digital transrectal ultrasonography (TRUS)
• One of the early detection techniques.
• Low cost, high availability, high safety,
  immediate results.
• TRUS can be used to plan and guide prostate
  biopsy.
• This thesis tries to automate the cancerous
  region detection process.

5
Introduction- Features -

• Feature
• Measurement of some characteristics (e.g.
  darkness, texture).
• A good feature should be discriminative so that,
  ideally, the cancerous regions are mapped to a
  different range of feature values in the feature
  space than the non-cancerous regions.

feature value
benign
cancerous
6
Introduction- This Thesis -

• This thesis proposes a new feature extraction
  method
• Spatial location, symmetry, and other geometric
  measurements of the regions-of-interest, in
  addition to the greylevel and texture.
• Uses a semi-automatic fuzzy inferencing system
  (FIS) to relate all the features and mimic
  radiologists knowledge.

Outline
7
Medical Background- Male Reproductive System -
penis
8
Medical Background- Prostate Zonal Anatomy -
vas deferens
bladder
seminal vesicles
anterior fibromuscular stroma (AFMS)
central zone (CZ)
ejaculatory duct
transition zone (TZ)
verumontanum
peripheral zone (PZ)
rectum
urethra
9
Medical Background- BPH -
Back

• Young and healthy prostate

• Prostate with Benign prostatic hyperplasia (BPH)

10
Medical Background- Prostate Cancer -

• Prostate cancer involves the growth of malignant
  prostate tumours and can be life threatening.
• Uneven statistical distribution
• 70 originates in PZ.
• 10 originates in CZ.
• 20 originates in TZ.
• Cancer tends to be localized in the early stage,
  any asymmetry on the axial view might suggest
  cancer development.

11
Medical Background- TRUS Imaging -

• Echoicities

12
Medical Background- TRUS Imaging -

• TRUS imaging
• About 80 of prostate cancer tissues consist of
  hypoechoic tissues (mixed with other
  echoicities).
• Different probes (e.g. end-fire, side-fire) give
  different shapes of the captured image of the
  prostate.

Image
13
Medical Background- Summary -

• Uneven cancer statistical distribution.
• Asymmetry of regions-of-interest.
• TRUS echoicities.
• Different probes give different prostate shapes.

Outline
14
Related Researches

• Transform-Based
• Fourier Transform
• Gabor Transform
• Wavelet Transform

• Statistic-Based
• First-Order Statistics
• Second-Order Statistics

15
Related Researches- Fourier Transform -

• Fourier Transform
• Decompose into pure frequencies
• Not localized in spatial domain.
• A global operator.

Chapter Outline
16
Related Researches- Gabor Transform -

• Gabor Transform
• Windowed Fourier Transform.
• Trade off between spatial and frequency
  resolutions.

Frequency Domain
0
Spatial Domain
0
17
Related Researches- Gabor Transform -

• Gabor Filter
• A variation of the Gabor Transform.
• Translate the window in the frequency domain to
  capture different frequency components.

18
Related Researches- Gabor Transform -

• Gabor Filter
• Its anisotropic (i.e. orientation dependent).

texture orientation
path of ultrasound wave
Chapter Outline
19
Related Researches- Wavelet Transform -

• Wavelet Transform
• Multiresolution Analysis (MRA).
• Different dilations of basis functions to analyze
  different scales.

20
Related Researches- Transform-Based Limitations -

• Limitations of transform-based methods
• Similar frequency spectrum.

Frequency Domain
Spatial Domain
Chapter Outline
21
Related Researches- First-Order Statistics -

• First-Order Statistics
• Greylevel of each pixel.
• One of the most discriminative features.

Cancerous Region
TRUS Image
Chapter Outline
22
Related Researches- Second-Order Statistics -
Back

• Second-Order Statistics
• Statistics on two neighbouring pixels.
• Requires a window defining the neighbourhood.
• Greylevel Difference Matrix (GLDM)
• Contrast (CON)
• Mean (MEAN)
• Entropy (ENT)
• Inverse Difference Moment (IDM)
• Angular Second Moment (ASM)

23
Related Researches- Summary -

• All these methods were successfully applied to
  extract features from
• Modalities with good resolution and image
  quality, such as CT and MRI.
• High-level structures such as the overall
  prostate or large regions (at least 6464
  pixels).
• However,

24
Related Researches- Summary -

• However, they are not suitable for extracting
  features of low-level structures in ultrasound
  images.
• Any size of the window or wavelet basis
• Too large for region boundary integrity.
• Too small for reliable statistics.

Outline
25
Problem Formulation- Resources -

• Average image size 188.6346.3 pixels.
• Average cancerous region size 2920.3 pixels that
  is smaller than a circle with radius of 30.5
  pixels!

Original TRUS Image
Prostate Outline
Cancerous Region Outline
TZ Outline
26
Problem Formulation- Objectives -

• To come up with a new set of features that can
  help differentiate cancerous regions in a TRUS
  image from the rest of the prostate.
• Desirable criteria
• The features can be applied to analyze low-level
  structures, such as the cancerous regions
  (30-radius circle).
• The boundary integrity of each region-of-interest
  should be well preserved.
• The features should be isotropic.
• The features should be discriminative enough to
  differentiate cancerous regions from the benign
  regions.

Outline
27
Proposed Feature Extraction Method- Overview -
input
Region Segmentation
Image Registration
Raw-Based Feature Extraction
Model-Based Feature Extraction
Greylevel
Texture
Region Geometry
Symmetry
Spatial Location
design only
Feature Evaluation
FIS
PDF Estimation
Membership Functions
output
Feature Design Parameters
MI Evaluation
Fuzzy Rules
Feature Selection
Outline
28
Proposed Feature Extraction Method- Region
Segmentation -

• Some region segmentation methods that I have
  tried
• Graph-theory-based method by constructing Minimum
  Spanning Tree (MST).
• Thresholding on histogram.

Graph-theory-based method
Thresholding-based method
29
Proposed Feature Extraction Method- Region
Segmentation -

• Thresholding-based method

Original
Gaussian Blurred
Histogram
Greylevel Segmentation
Zonal Segmentation
Morphological Operators open and holes
Resulting Segmentation
Overview
30
Proposed Feature Extraction Method- Image
Registration -

• Prostates have different shapes on TRUS images
  due to
• Different physical shapes.
• Different probes (e.g. side-fire, end-fire).
• Prostates may not be located at the centre of the
  image.

31
Proposed Feature Extraction Method- Image
Registration -

• The idea is to deform all the prostates into a
  common model shape
• The model shape should allow the ease of
  specifying the relative spatial location of a
  given point with respect to the whole prostate.
• The model shape should be similar to an average
  prostate outline.
• The model shape should be reflectionally
  symmetric about the vertical axis located at the
  centre of the image.

32
Proposed Feature Extraction Method- Image
Registration -

• A compromise

33
Proposed Feature Extraction Method- Image
Registration -
Affine Transformation
Outline-Based
Texture-Based
Fluid-Landmark-Based Transformation
Define Landmarks
Model-Based
Estimate Optimal Trajectories
Calculate Velocity Vectors
Interpolate Missing Pixels
34
Proposed Feature Extraction Method- Image
Registration -

• Define landmarks
• 16 equally spaced landmarks on the prostate
  outline.
• 2 equally spaced landmarks on the vertical axis.
• No medical knowledge of the anatomical structure
  is required.

35
Proposed Feature Extraction Method- Image
Registration -

• Lagrangian trajectory
• The initial, intermediate, and final positions.
• Velocity vectors
• Displacement of the position of a landmark (in a
  unit of time).

36
Proposed Feature Extraction Method- Image
Registration -

• Estimate optimal trajectories
• Minimize
• Iterative gradient decent

37
Proposed Feature Extraction Method- Image
Registration -

• Interpolate the optimal velocity vectors for the
  whole image space
• Optimal velocity vectors of the landmarks
• Optimal velocity vectors of the whole image space

38
Proposed Feature Extraction Method- Image
Registration -

• Optimal velocity vectors
• Interpolate the optimal Lagrangian trajectories
  for the whole image

39
Proposed Feature Extraction Method- Image
Registration -

• Interpolating missing pixels in the resulting
  image using linear interpolation.

After deformation
Before deformation
40
Proposed Feature Extraction Method- Image
Registration -

• Now, we can easily measure spatial location and
  symmetry!
• Original images
• Registered Images

Overview
41
Proposed Feature Extraction Method- Greylevel -

• Blur with Gaussian filter.
• Design parameter
• Take average over each region-of-interest.

TRUS
Pixel-Based Greylevel (GL)
Region-Based Greylevel (GL)
Overview
42
Proposed Feature Extraction Method- Texture -

• GLDM with different window size.
• Design parameter

Equations
Pixel-Based
Region-Based
CON
MEA
ENT
IDM
ASM
Overview
43
Proposed Feature Extraction Method- Symmetry -

• Difference from flipped feature images.
• Design parameter none.

Greylevel- Symmetry (GS)
Texture- Symmetry (GS)
Pixel-Based before inverse-deformation
Pixel-Based
Region-Based
Overview
44
Proposed Feature Extraction Method- Spatial
Location -

• Define coordinate system using a cone.
• Design parameter

45
Proposed Feature Extraction Method- Spatial
Location -

• Spatial Radius (SR) 0 at origin, 1 at the
  perimeter.
• Spatial Angle (SA) 0 at top, 1 at bottom.

Spatial- Radius (SR)
Spatial- Angle (SA)
Pixel-Based before inverse-deformation
Pixel-Based
Region-Based
Overview
46
Proposed Feature Extraction Method- Region
Geometry -

• Region Area (RA) number of pixels.
• Region Roundness (RR)
• perimeter of a circle with the same area
  divided by
• perimeter of the region.

Region Area (RA)
Region Roundness (RR)
Overview
47
Proposed Feature Extraction Method- Feature
Evaluation -

• How to fine-tune design parameters?
• How to evaluate each feature?
• How to compare the features?

Original TRUS
Expected Cancerous Region
SR
ASM
48
Proposed Feature Extraction Method- PDF
Estimation -

• We can analyze its probability density function
  (PDF).
• Parzen Estimation is used.

P(xCancerous)
P(xBenign)
P(x)
49
Proposed Feature Extraction Method- MI
Evaluation -

• Entropy
• Measures the degree of uncertainty.
• Mutual information between feature and class
• Measures the decrease in entropy with an
  introduction of a feature F.
• Measures the interdependence between class and
  feature.
• Bounds

50
Proposed Feature Extraction Method- Feature
Design Parameters -

• Using MI(FC), the optimal design parameter for
  each feature can be selected more objectively.

51
Proposed Feature Extraction Method- Feature
Selection -

• Select only a subset of the features.
• For efficiency, and sometimes accuracy.
• Need to eliminate
• uninformative features low MI(FC).
• redundant features high MI(F1F2).

52
Proposed Feature Extraction Method- Feature
Selection -
Back

• Use MI(FC) to eliminate uninformative features.

53
Proposed Feature Extraction Method- Feature
Selection -

• Use MI(F1F2) to eliminate redundant features.

54
Proposed Feature Extraction Method- Feature
Selection -

• Checking the feature selection visually

TRUS
Expected
CON
MEA
ENT
IDM
ASM
GL
GS
TS
SR
SA
RA
RR
Overview
55
Proposed Feature Extraction Method- Fuzzy
Inferencing System -

• Each feature by itself is not discriminative
  enough.
• Need to find out the relationship between the
  selected features by analyzing them jointly
  (collectively).
• This thesis proposes to use aFuzzy Inferencing
  System (FIS).
• The idea is to come up a set of fuzzy rules that
  relate all the selected features.

56
Proposed Feature Extraction Method- Fuzzy
Inferencing System -
57
Proposed Feature Extraction Method- Membership
Functions -

• Design the breakpoints of the membership
  functions using PDFs.
• Inspect local minima.
• Inspect intersection.
• Semi-automatic.

P(xCancerous)
P(xBenign)
P(x)
58
Proposed Feature Extraction Method- Membership
Functions -

• Chosen breakpoints and fuzziness.

59
Proposed Feature Extraction Method- Fuzzy Rules -

• Generate fuzzy rules for each image

60
40
MF1
MF2
MF3
MF4
MF5
MF1
MF2
MF3
Ratio3 0.6
Rule 1 if (FEATURE1 is MF2) and (FEATURE2 is
MF3) then (CANCEROUS)
Rule 2 if (FEATURE1 is MF3) and (FEATURE2 is
MF3) then (CANCEROUS)
Rule 3 if (FEATURE1 is MF3) and (FEATURE2 is
MF3) then (LIKELY-CANCEROUS)
Rule 4 if (FEATURE1 is MF1) and (FEATURE2 is
MF2) then (BENIGN)
Overview
60
Analysis

• Some successful sample results

Original TRUS
Expected Cancerous Region
Proposed Feature Image
61
Analysis

• Some less successful sample results

Original TRUS
Expected Cancerous Region
Proposed Feature Image
62
Analysis

• Comparison between proposed feature extraction
  method with other methods

Individual region-based features
Combined feature
Pixel- vs. Region-Based
13 improvement due to FIS!
57 improvement due to new features!!!
Outline
63
Conclusions

• Large-Fluid-Landmark Deformation was used to
  deform prostates into a common model shape.
• PDFs were used to
• Evaluate each feature individually using MI(FC).
• Eliminate redundant features using MI(F1F2).
• Design membership functions semi-automatically.
• Generate fuzzy rules automatically.
• Fuzzy rules mimics radiologists medical
  knowledge.
• 13 improvement due to FIS!
• 57 improvement due to new features, especially
  Spatial Location features.

Outline
64
Future Works

• Investigate on region segmentation that can best
  serve the proposed feature extraction method.
• Fully automate the membership function design
  using PDFs.
• Define optimal thresholds for classifying the new
  feature.

65
Questions and Comments?

66
References

• Medical Basics
• M. D. Rifkin, Ultrasound of the Prostate
  Imaging in the Diagnosis and Therapy of Prostatic
  Disease, 2nd Edition, Lippincott Williams and
  Wilkins, 1996.
• Texture Analysis
• A. H. Mir, M. Hanmandlu, S. N. Tandon, Texture
  Analysis of CT Images, IEEE Engineering in
  Medicine and Biology, November / December 1995.
• K. N. B. Prakash, A. G. Ramakrishnan, S. Suresh,
  T. W. P. Chow, Fetal Lung Maturity Analysis
  Using Ultrasound Image Features, IEEE
  Transactions on Information Technology in
  Biomedicine, Vol. 6, No. 1, March 2002.
• O. Basset, Z. Sun, J. L. Mestas,G. Gimenez,
  Texture Analysis of Ultrasound Images of the
  Prostate by Means of Co-occurrence Matrices,
  Ultrasound Imaging 15, 218-237 (1993).
• Image Registration
• Sarang C. Joshi and Michael I. Miller, Landmark
  Matching via Large Deformation Diffeomorphisms,
  IEEE Transactions on Image Processing, Vol. 9,
  No. 8, August 2000.
• Symmetry
• Q. Li, S. Katsuragawa, K. Doi, Improved
  contralateral subtraction images by use of
  elastic matching technique, Medical Physics, 27
  (8), August 2000.
• Feature Selection
• R. Battiti, Using Mutual Information for
  Selecting Features in Supervised Neural Net
  Learning, IEEE Transactions on Neural Networks,
  Vol. 5, No. 4, July 1994.
• Please see my thesis for all other references.