Data Mining via Geometrical Features of Segmented Images

1
Data Mining via Geometrical Features of
Segmented Images

• Luca Galli, Antonella Petrelli, Andrea
  Colapicchioni
• Advanced Computer Systems S.p.A.

• ESA-EUSC 2005 Frascati, Italy
• Image Information Mining Theory and
  Application to Earth Observation

2
Summary

• Data Mining for high-spatial resolution images
• Segmentations problems
• Image Segmentation by SWA Algorithm
• Regions Information Extraction
• Applications - object-based image analysis
• KIM tool
• Conclusions

3
Data Mining for high-spatial resolution images
with high-spatial resolution satellite sensors
(IKONOS, QUICKBIRD) very detailed imaging of
urban environment
more difficult to apply traditional digital image
investigation methods
Object-based image analysis approach
4
Chicken and Egg Problem

• Segments that differ by coarse scale difference
  introduces specific difficulties
• coarse measurement must rely on an arbitrary
  chosen set of pixels blurring and
  over-smoothing
• segment detection requires coarse scale
  measurement.
• Segmentation via global optimisation methods

5
SWA Algorithm (1/5)

• Segmentation by Weighted Aggregation
• (Sharon-Brandt-Basri)
• Building a 4-connected graph G(V,E,W) from the
  image
• V nodes (pixel images)
• E edges (neighbouring pixels)
• W coupling between nodes (dissimilarity between
  nodes).
• Segment detection by finding the cuts that
  minimize a normalize-cut measure.
• J. Shi, and J. Malick, Normalized Cut and Image
  Segmentation
• Multiscale Weighted Aggregation, which induces a
  irregular pyramid for fast computation.

6
SWA Algorithm (2/5)
For every segment S ? V we associate a state
vector u(u1, u2, , un), where
normalize cut measure associated with S is
defined by
Find the optimal partition of the graph
cut
7
SWA Algorithm (3/5)

• Coarsening graph transformation

• For each scale s the coarser graph Gs1 is
  defined by set of nodes C ? Vs such that every
  node in Vs \C is strongly connected to C
  (coarse scale representative pixels)
• Successive nodes reduction (about half number)

Strongly connected criteria Sparse
inter-scale interpolation matrix (coarse-fine)
us-1 ? Ps-1,s us
8
SWA Algorithm (4/5)
ws1 PT ws P
Recursive aggregation optimal graph
partition (each segment is represented by a
single node) segments low saliency
9
SWA Algorithm (5/5)

• Regional properties modify the weights using
• aggregate pixel statistics
• Top-down procedure determine the location of the
  segments in the image. Inter-scale interpolation
  rule at each segment representative pixels from
  the highest level downward to image pixel level,
  assigning to each image pixel the label
  corresponding to the highest likelihood

us-1 ? Ps-1,s us
10
Optical Remotely Sensed Data

• Gaussian image model Bhattacharya distance to
  modify the coarse-scale coupling
• statistical moments computed recursively linear
  complexity preserved
• Multispectral/Hyperspectral images extension
• Multidimensional Gaussian stationary model, and
  PCA transform
• Total unsupervised by introducing a stop
  criteria for level coarsening.

11
San Peter (IKONOS)
12
San Peter (IKONOS)
13
Glinska (SMART - 11 bands)
14
Glinska (SMART - 11 bands)
15
Toulose (SPOT 5 - 4 bands)
16
Toulose (SPOT 5 - 4 bands)
17
Toulose (SPOT 5) Zoom
18
Toulose (SPOT 5) Zoom
19
Experiments

• Parameters
• Image level coupling constant a 0.2
• coarse-level coupling constant ? 1.5
• Increasing (decreasing) both the coupling
  constants the segmentation procedure tends to
  over-segment (under-segment) the image.
• Computational cost
• (Laptop Pentium IV 2.66 GHz)

20
Regions Information Extraction
From Pixel-based to Object-based image analysis
2 steps

• encode of the segmentation

• extraction of informations with respect to shape
  and multispectral data values of segmentations
  regions.

21
first step the encode of the segmentation
CONTOUR OF A REGION
sequence of nodes and arcs joining the nodes

• each arc belongs to the contour of just to
  regions

• each node belongs to the contour of three or
• more regions.

sequence of the nodes and the arcs ? topology
form of the arcs ? geometry
22
the geometry of the segmentation
inter-pixel

23
the topology of the segmentation
for holes (regions strictly contained in other
regions) ? artificial nodes
24
second step the extraction of descriptors
through regions reconstruction
for each region of images segmentation
forms descriptors area, compactness, moment
descriptors (Hu, Zernike)
Contents information mean and variance of data
values of multispectral image
25
moment descriptors
TO DESCRIBE THE SHAPE FORM OF REGIONS
invariance with respect to
scale
translations
rotations
HU MOMENTS ? NOT-ORTHOGONAL
ZERNIKE MOMENTS ? ORTHOGONAL
26
APPLICATIONS IN KIM/KES
Integration in KIM/KES
attach to all pixels of regions of the
segmentation
27
APPLICATIONS IN KIM/KESS
ZERNIKE MOMENTS
order 2 A20, A22
order 3 A31, A33
28
WORK IN PROGRESS
Original image GLINSKA (SMART 11 bands)
29
WORK IN PROGRESS
class forest a posteriori map
30
WORK IN PROGRESS
merged regions
31
WORK IN PROGRESS
Region spectral mean value before merging
32
WORK IN PROGRESS
Region spectral mean value after merging
33
WORK IN PROGRESS
Associating map REGION ? CLASSES

• Range of the classes
• forest
• river
• agriculture field

34
WORK IN PROGRESS
merged regions
35
WORK IN PROGRESS
Region spectral mean value before merging
36
WORK IN PROGRESS
Region spectral mean value after merging
37
Conclusions

• Introduced unsupervised multiscale image
  segmentation
• algorithm that extends to multichannel remote
  sensing
• optical data the segmentation by SWA method.
• Effectiveness of the method for the
  segmentation of
• remotely sensed images, which possess complex
  and
• heterogeneous multiscale and multispectral
  characteristics.
• Because it combines together region-based and
• edge-based aspects.
• Object characterization spectral statistics,
  shape and topology
• Integration in the KIM/KES tool