Hierarchical Part-Based Human Body Pose Estimation

1
Hierarchical Part-Based Human Body Pose Estimation

• Ramanan Navaratnam
• Arasanathan Thayananthan
• Prof. Phil Torr
• Prof. Roberto Cipolla

University Of Cambridge Oxford Brookes
University
2
Introduction
Input
3
Introduction
Input
Output
4
Overview

1. Motivation
2. Hierarchical parts
3. Template search
4. Pose estimation in a single frame
5. Temporal smoothing
6. Summary Future work

5
Overview

1. Problem motivation ???
2. Hierarchical parts
3. Template search
4. Pose estimation in a single frame
5. Temporal smoothing
6. Summary Future work

6
Overview

1. Problem motivation ???
2. Hierarchical parts
3. Template search
4. Pose estimation in a single frame
5. Temporal smoothing
6. Summary Future work

7
Overview

1. Problem motivation ???
2. Hierarchical parts
3. Template search
4. Pose estimation in a single frame
5. Temporal smoothing
6. Summary Future work

8
Motivation

• Real-time Object Detection for Smart Vehicles
• D. M. Gavrila V. Philomin (ICCV 1999)

• Filtering using a tree-based estimator
• Stenger et.al. (ICCV 2003)

9
Motivation

• Real-time Object Detection for Smart Vehicles
• D. M. Gavrila V. Philomin (ICCV 1999)

• Filtering using a tree-based estimator
• Stenger et.al. (ICCV 2003)

• Exponential increase of templates with dimensions

10
Motivation

• Pictorial Structures for Object Recognition
• P. Felzenszwalb D. Huttenlocher (IJCV 2005)

• Human upper body pose estimation in static
  images
• M.W. Lee I. Cohen (ECCV 2004)

11
Motivation

• Pictorial Structures for Object Recognition
• P. Felzenszwalb D. Huttenlocher (IJCV 2005)

• Human upper body pose estimation in static
  images
• M.W. Lee I. Cohen (ECCV 2004)

• Part based approach
• Assembling parts together is complex

12
Motivation

• Automatic Annotation of Everyday Movements
• D. Ramanan D. A. Forsyth (NIPS 2003)

• 3-D model-based tracking of humans in actiona
  multi-view approach
• D. M. Gavrila L. S. Davis (CVPR 1996)

13
Motivation

• Automatic Annotation of Everyday Movements
• D. Ramanan D. A. Forsyth (NIPS 2003)

• 3-D model-based tracking of humans in actiona
  multi-view approach
• D. M. Gavrila L. S. Davis (CVPR 1996)

• State space decomposition

14
Hierarchical Parts
15
Hierarchical Parts
16
Hierarchical Parts
17
Hierarchical Parts
18
Hierarchical Parts
Conditional prior
p(xi/xparent(i))
Spatial dimensions (translation)
Joint Angles
19
Hierarchical Parts
20
Hierarchical Parts
Detection Threshold 0.81
21
Hierarchical Parts
Detection Threshold 0.81
22
Template Search
23
Template Search
24
Template Search
25
Template Search

• Features
• Chamfer distance
• Appearance

26
Template Search

• Features
• Chamfer distance
• Appearance

27
Template Search

• Features
• Chamfer distance
• Appearance

28
Template Search

• Features
• Chamfer distance
• Appearance

29
Template Search

• Features
• Chamfer distance
• Appearance

30
Template Search

• Features
• Chamfer distance
• Appearance

31
Template Search

• Features
• Chamfer distance
• Appearance

32
Template Search

• Features
• Chamfer distance
• Appearance

33
Template Search

• Features
• Chamfer distance
• Appearance

34
Template Search

• Learning Appearance
• Match T pose based on edge likelihood only in
  initial frames
• Update 3D histograms in RGB space that
  approximates P(RGB/part) and P(RGB)

35
Pose Estimation in a Single Frame
36
Pose Estimation in a Single Frame
37
Pose Estimation in a Single Frame
38
Temporal Smoothing
HMM
39
Temporal Smoothing
T t
HMM
40
Temporal Smoothing
HMM
41
Temporal Smoothing
42
Temporal Smoothing
43
Summary Future work

• Summary
• Realtime process (unoptimized code at 1Hz, 2.4
  Ghz IG RAM)
• 3D pose
• Automatic initialisation and recovery from
  failure

44
Summary Future work

• Summary
• Realtime process (unoptimized code at 1Hz, 2.4
  Ghz IG RAM)
• 3D pose
• Automatic initialisation and recovery from
  failure
• Future work
• Extend robustness to illumination changes
• Non-fronto-parallel poses
• Poses when arms are inside the body silhouette
• Simple gesture recognition by assigning semantics
  to regions of articulation space