Estimating Human Shape and Pose from a Single Image

1
Estimating Human Shape and Pose from a Single
Image

• Peng Guan
• Alex Weiss Alexandru Balan Michael J.
  Black
• Brown University
• Department of Computer Science

ICCV 2009
2
Body shape and pose from 1 image?
3
Introduction

• What others do
• Estimating 3D human pose in uncalibrated
  monocular imagery
• Use silhouette in multi-camera setting to recover
  3D body shape
• Most work assumes the existence of a known
  background to extract foreground silhouette
• In previous body models, height is correlated
  with other shape variations

• What we do
• Estimating both 3D shape and pose in uncalibrated
  monocular imagery
• Use additional monocular cues including smooth
  shading
• Use GrabCut to produce foreground region
• Make height variation concentrated along one
  shape basis vector, which allows height
  constrained fitting

4
Previous Work

• 3D pose and shape estimation from multiple,
  calibrated, cameras

Balan, A., Sigal, L., Black, M. J., Davis, J.,
Haussecker, H, Detailed human shape and pose
from images, Proc. IEEE Conf. on Computer Vision
and Pattern Recognition, CVPR, Minneapolis, June
2007
5
SCAPE Body Model
D. Anguelov, P. Srinivasan, D. Koller, S. Thrun,
J. Rodgers, and J. Davis. SCAPE Shape completion
and animation of people. SIGGRAPH, 24(3)408416,
2005.
Pose Training Set
6
Body shape/pose from 1 image Problems

1. High dimensional body model (shape and pose)
   initialization problem.
2. Background unknown
3. Single, monocular image
4. poorly constrained
5. Shape/Pose ambiguities
6. Silhouette insufficient

7
Previous Work (CJ Taylor 2000, Lee Chen 1985)

• 3D pose estimation using orthographic camera
  assumption

C. J. Taylor, Reconstruction of Articulated
Objects from Point Correspondences in a Single
Uncalibrated Image, Computer Vision and Image
Understanding, Vol 80, No 10, Pgs 349-363,
October 2000
8
C.J.Taylor Model
9
Solution 1 Pose Initialization
Better
Shape initialized to mean body shape.
10
Solution 2 Segmentation
C. Rother, V. Kolmogorov, and A. Blake.
GrabCut Interactive foreground extraction
using iterated graph cuts. SIGGRAPH,
23(3)309314, 2004.
11
Problem Pose/Shape ambiguities
Body shape and pose fitted to a single camera view
12
Solution 3 Height Preserving Shape Space
13
Shape space without height preserving
14
Problem Silhouette not sufficient
15
Solution 4 Edge Cues
16
Problem Shape not well constrained
17
Solution 5 Parametric Shape from Shading
M. de la Gorce, N. Paragios and David Fleet.
Model-Based Hand Tracking with Texture, Shading
and Self-occlusions. IEEE Conference in Computer
Vision and Pattern Recognition (CVPR), Anchorage
2008.
18
Shading/Overall Cost function
Shading cost function
Overall cost function
19
Experiment Lab Images
20
Experiment Lab Images
21
Quantitative Comparison
22
Experiment Internet Images
23
Experiment Paintings
24
(No Transcript)
25
Conclusions

• Contributions
• Solution to a new problem Human pose and shape
  estimation from a single image
• Parametric shape from shading for estimating
  human shape from complex images and paintings
• Attribute-constrained body model
• Limitations
• Single point light assumption and simplified
  model of surface reflection
• User assistance for pose initialization
• Minimal clothing for shading

26
Acknowledgement

• Financial support NSF IIS-0812364 and the RI
  Economic Development Corp.
• Peng Guan, Alexander Weiss, Alexandru Balan,
  Michael Black, Estimating Human Shape and Pose
  from a Single Image, Int. Conf. on Computer
  Vision, ICCV, Kyoto, Japan, Sept. 2009
• Alexander Weiss GrabCut 3D pose initialization
• Alexandru Balan Height preserving shape space
• David Hirshberg Projection of model edge

27

• Thank you!