Image Segmentation Part II

1
Image Segmentation Part II
Dr. Ramprasad Bala Computer and Information
Science UMASS Dartmouth CIS 585 Image
Processing and Machine Vision
2
Aggregating Edges

• The border tracking algorithm discussed earlier
  assumes each segment is closed and distinct.
• There was never any point at which the border
  could be split into two or more segments.
• When the input is instead a labeled edge image
  with value 1 for edge and 0 for non-edge pixels,
  the problem if tracking edge segments is more
  complex.

3

• The image above has three segments. Pixel 3,3
  is a junction pixel where three different edge
  segments meet. Pixel 5,3 is corner pixel and
  may be considered a segment end point as well, if
  the application requires ending segments at
  corners.

4
Aggregating Edges

• An algorithm that tracks segments like these has
  to be concerned with the following tasks
• Starting a new segment
• Adding an interior pixel to a segment
• Ending a segment
• Finding a junction
• Finding a corner

5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
Hough transforms

• If an image consists of objects with known shape
  and size, segmentation can be viewed as a problem
  of finding this object within an image.
• The original Hough transform was designed to
  detect straight lines and curves
• A big advantage of this approach is robustness of
  segmentation results that is, segmentation is
  not too sensitive to imperfect data or noise.

9
Hough Transform

• The original Hough transform was designed to
  detect straight lines and curves

10
Hough Transform

• Any straight line in the image is represented by
  a single point in the k,q parameter space .
• The main idea of line detection is to determine
  all the possible line pixels in the image, to
  transform all lines that can go through these
  pixels into corresponding points in the parameter
  space, and to detect the points (a,b) in the
  parameter space that frequently resulted from the
  Hough transform of lines yaxb in the image.

11

• Detection of all possible line pixels in the
  image may be achieved by applying an edge
  detector to the image
• Then, all pixels with edge magnitude exceeding
  some threshold can be considered possible line
  pixels.
• In the most general case, nothing is known about
  lines in the image, and therefore lines of any
  direction may go through any of the edge pixels.
  In reality, the number of these lines is
  infinite, however, for practical purposes, only a
  limited number of line directions may be
  considered.
• The possible directions of lines define a
  discretization of the parameter k.
• Similarly, the parameter q is sampled into a
  limited number of values.

12
Example
13

• The parameter space is not continuous any more,
  but rather is represented by a rectangular
  structure of cells. This array of cells is called
  the accumulator array A, whose elements are
  accumulator cells A(k,q).
• For each edge pixel, parameters k,q are
  determined which represent lines of allowed
  directions going through this pixel. For each
  such line, the values of line parameters k,q are
  used to increase the value of the accumulator
  cell A(k,q).
• Clearly, if a line represented by an equation
  yaxb is present in the image, the value of the
  accumulator cell A(a,b) will be increased many
  times -- as many times as the line yaxb is
  detected as a line possibly going through any of
  the edge pixels.

14

• Lines existing in the image may be detected as
  high-valued accumulator cells in the accumulator
  array, and the parameters of the detected line
  are specified by the accumulator array
  co-ordinates.
• As a result, line detection in the image is
  transformed to detection of local maxima in the
  accumulator space.
• The parametric equation of the line ykxq is
  appropriate only for explanation of the Hough
  transform principles -- it causes difficulties in
  vertical line detection (k -gt infinity) and in
  nonlinear discretization of the parameter k.

15

• The Hough transform does not suffer from
  representing the line as
• The lines still correspond to a point

16

• Discretization of the parameter space is an
  important part of this approach.
• Also, detecting the local maxima in the
  accumulator array is a non-trivial problem.
• In reality, the resulting discrete parameter
  space usually has more than one local maximum per
  line existing in the image, and smoothing the
  discrete parameter space may be a solution.
• Generalization to more complex curves that can be
  described by an analytic equation is
  straightforward.

17

• Consider an arbitrary curve represented by an
  equation f(x,a)0,
• where a is the vector of curve parameters.

18
Line Detection Using Hough Transforms
19
Hough Space
20
Peak Detection
21
Lines detected
22
Extending to other curves

• If we are looking for circles, the analytic
  expression f(x,a) of the desired curve is
• where the circle has center (a,b) and radius r.
• Therefore, the accumulator data structure must be
  three-dimensional.

23
(No Transcript)
24
Hough Circles - experiment
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
Fitting Models to Segments

• Mathematical models that fit data not only reveal
  important structure in the data, but also can
  provide efficient representations.
• A straight line model might be used for edge
  pixels of a building or a planar model might
  apply to surface data from the face of a
  building.
• Convenient models exist for circles, cylinders
  and many other shapes.

29
Method of Least-Squares

• Fitting Straight Line
• The data for fitting might be obtained from one
  of the methods discussed earlier.
• One straight line model is a function with two
  parameters y f(x) c1x c0.
• Suppose we want to test whether or not a set of
  observed points (xj,yj), j 1,n form a line.
  To do this we need to determine the best
  parameters c1 and c0 of the linear function and
  see how close the points are to the function.

30
(No Transcript)
31
Least-Square Error Criteria

• Definition The measure of how well a model y
  f(x) fits a set of n observations (xj,yj), j
  1,n is
• The best model y f(x) is the model with the
  parameters minimizing this criteria.

32
Root-Mean-Square Error

• The RMSE is the average difference of
  observations from the model
• Note that for the straight line fit, this
  difference is not the Euclidean distance of the
  observed point from the line, but the distance
  measured parallel to the y-axis.

33
Max-Error Criteria

• The measure of how well a model yf(x) fits a set
  of n observations (xj,yj), j 1,n is
• MAXE max((f(xj) yj)j1,n)
• This measure depends only on the worst fit point,
  whereas the RMS error depends on the fit of all
  of the points.

34
Closed Form Solutions

• The closed form solution for the parameters if
  the best-fitting straight line can be explicitly
  written as follows
• In this equation (xj,yj) are constants and will
  have its global minimum at points (c1,c0) where

35
Writing out the derivatives and combining them
will yield a matrix eqn
36
Empirical Interpretation of Error

• The interpretation of error is straight forward
  in the case of 2D imagery.
• For example, we might accept the fit if all
  observed points are within a pixel or two of the
  model.
• If individual points are far from the fitted line
  (outliers), they could indicate feature detection
  error, an actual defect in the object,or that a
  different object or model exists.
• In these cases, it is ok to delete the outliers
  from the set of observations and repeat the fit.

37
Problems in Fitting

• Outliers Since every observation contributes to
  the RMS error, a large number of outliers may
  render the fir worthless.
• The initial fit may be so far off the ideal that
  it is impossible to identify and delete the
  outliers.
• Methods of robust statistics can be applied in
  such cases.

38

• Error definition The mathematical definition of
  error as a difference along the y-axis is not a
  true geometric distance thus the least squares
  fit does not necessarily yield a curve or surface
  that best approximates the data in geometric
  space.
• The right most point is close to the circle but
  will produce a large error along the y-axis.

39

• Nonlinear Optimization Sometimes a closed form
  solution to the model parameters is not
  available. The error criteria can still be
  optimized by using a technique that searches
  parameter space for the best parameters.
  Hill-climbing, gradient-based search or even
  exhaustive search can be used for optimization.
• High Dimensionality Methods of obtaining local
  minima in these cases can be very hard.
• Fit Constraint The model being fit must satisfy
  additional constraints. For example the line
  were fitting must be perpendicular to another
  line. Constrained optimization may be a solution
  in these problems.

40
Identifying Higher-level Structures

• Combining these segments (edge or regions) we can
  extract higher-level structures. We will look at
  two here Ribbons and Corners.
• Ribbons are produced by imaging elongated objects
  in 2D or 3D. In examples like roads or pens the
  sides of the ribbons are approximately parallel
  but not necessarily straight. We confine our
  attention to ribbons that are straight.

41
Ribbons

• Definition A ribbon is an elongated region that
  is approximately symmetrical about its major
  axis. Often, but not always, the edge of a ribbon
  contrast symmetrically with its background.

42
Detecting Straight Ribbons

• By using the Hough parameters along with points
  lists more complex image structures can be
  detected.
• Two edges whose directions differ by 180o provide
  evidence of a possible ribbon.
• If in addition, the point lists are located near
  each other, then there is evidence of a large
  linear feature.

43
(No Transcript)
44
Detecting Corners

• Significant region corners can be detected by
  finding pairs of detected edge segments E1 and E2
  in the following relationship.
• Lines fit to edge point sets E1and E2 intersect
  at point u,v in the real image coordinate.
• Point u,v is close to extreme point on both
  sets of E1 and E2.
• The gradient directions of E1 and E2 are
  symmetric about their axis of symmetry.

45

• This definition models only corners of type L
  constraint (2) rules out those of type T, X
  and Y.
• The computed intersection will have sub-pixel
  accuracy.
• Edge segments can initially be identified by
  Hough transform or by boundary following or line
  fitting.
• For each pair (d1, ?1, d2,?2) satisfying the
  above criteria, add the quad (d1, ?1, d2,?2,
  u,v, a) to a set of candidate corners.
• The angle a is formed at the corner.
• This set of corner features can be used for
  building higher level descriptions, or can be
  used directly in image matching or warping (will
  be seen later).

46
(No Transcript)
47
Motion Segmentation

• Boundaries in Space-Time
• The contours of moving objects can be identified
  using both spatial and temporal contrast.
• Spatial and temporal gradients can be computed
  and combined if we have tow images Ix,y,t and
  Ix,y,t ?t of the scene.
• The spatio-temporal gradient magnitude can be
  defined as the product of the spatial gradient
  magnitude and the temporal gradient magnitude.

48
Aggregation of Motion Trajectories

• Region segmentation can be performed on the
  motion vectors by clustering according o image
  position, speed and direction.
• Clustering should be very tight for a translation
  object, because points of the object should have
  the same velocity.
• Through more complex analysis, objects that
  rotate and translate can also be detected.

49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)