Human Posture Recognition with Convex Programming

1
Human Posture Recognition with Convex Programming

• Hao Jiang, Ze-Nian Li and Mark S. Drew
• School of Computing Science
• Simon Fraser University
• Burnaby, BC, V5A 1S6

2
Human Posture Recognition

• Recognizing human postures is very important in
  vision and multimedia.
• It has many applications in surveillance, human
  computer interaction, image and video database
  analysis and retrieval.
• At the same time, recognizing human postures is a
  hard problem.

3
The Challenges of Human Posture Recognition

• It is hard to recognize human postures because
• Articulated nature of a human body
• No segmentation schemes are available for general
  images or videos.
• Strong background clutters.
• Large appearance changes because of clothing
• Different schemes have been studied.

4
Methods for Posture Recognition

• Methods having been studied
• Silhouette based method with background
  subtraction
• Multi-camera based methods
• Tracking body movement
• Chamfer matching based schemes
• Shape context based schemes
• These methods are not sufficient to address the
  problem robustly.

5
The Proposed Method

• We will present a matching based scheme that has
  the following properties
• Based on a robust convex (linear) programming
  matching scheme
• Work for cases where no background subtraction is
  available
• Able to deal with strong background clutters
• Able to deal with large appearance changes

6
Matching Distance Transform
Canny Edge Detection
Distance Transform
Template Generation
Template Image
Feature Point Selection
Delaunay Triangulation
Matching With LP
Target Image
Canny Edge Detection
Distance Transform
Object Recognition
result
7
Matching as a Labeling Problem
Target p
p
fp
Target
Clutter
q
fq
Target q
Template Mesh
Target Image
8
The Labeling Problem

• The matching problem can be formulated as the
  following optimization problem

Matching cost Smoothing term
9
Convex Relaxation

• The original problem is a hard non-convex
  problem. We convert it to LP

c(s,j)
fp-fq
10
Properties of the Relaxation

• For convex problems, LP exactly solves the
  continuous extension of the original problem.
• For general non-convex problems, LP solves the
  problem where each matching surface is replaced
  by the lower convex hull.
• The cheapest basis set for each site
  corresponds to the lower convex hulls vertices

11
The Effect of Covexification
For non-convex problems, the relaxation replaces
each c(m,j) by its lower convex hull surface
c(0,j)
For site 0
Label
Label
c(i,j)

Convexification
c(M-1,j)
For site M-1
Label
Label
Lower Convex Hull Vertices
Basic Labels
12
Searching Scheme of Simplex Method

• Using simplex method, there are at most three
  adjacent non-zero weight basis labels

Searching for one site

Non-zero-weight basis label

Zero-weight basis label
non basis label
Continuous label
13
Successive Relaxation Scheme

• Single relaxation may miss the global optimum
  because of convexification effect
• An intuitive scheme is to shrink the trust region
  and reconvexify the data in the smaller region
• This scheme is found to be able to greatly
  improve the matching results

14
The Trust Region Shrinking
15
Successive Relaxation Scheme (An Example)

• min C(1,r1) C(2,r2)0.5r1-r2

16
Shape Recognition

• We have to define the goodness of matching
• Matching cost (M) Average difference of the
  template and target image in the ROI.
• Deformation (D) Affine transformation
  compensated pairwise distance changes
• Shape Context in the ROI (C).
• Finally, we use M aDbC to quantify the
  matching

17
Random Dots Experiment
Noise 100 Random Disturbance 5
Noise 50 Random Disturbance 5
Noise 50 Random Disturbance 10
Noise 100 Random Disturbance 10
18
Matching Synthetic Images Results
LP
ICM
BP
GC
(a) Template model showing distance transform
(b) Matching result of proposed scheme (c)
Matching result by GC (d) Matching result by
ICM. (e) Matching result by BP.
19
Matching Leaves
20
Experiment Results
An example where traditional methods fail. (a)
Template image (b) Target image (c) Edge map
of template image (d) Edge map of target image
(e) Template mesh (f) Matching result of the
proposed scheme (g) ICM matching result (h)
Sliding template search result.
21
Gesture Recognition Results
Template Top match Second match
22
Gesture Recognition Results
23
Video Browsing Result
24
Video Browsing Result
25
Multiple Target Matching Results
26
Conclusion and Future Directions

• We present a robust matching framework for human
  posture recognition
• The method can be applied to multimedia data
  retrieval in image or video database, or human
  computer interaction applications
• In future work
• We will add tempera information for behavior
  recognition

27
Future work

• The successive reconvexification is in fact very
  general. It can be used to increase the
  robustness of many other matching schemes, such
  as BP and GC
• The proposed matching can be used for many other
  applications, such as tracking, object
  recongnition, motion estimation etc.

28
The End