Capture of Hair Geometry from Multiple Images

1
Capture of Hair Geometryfrom Multiple Images

• Sylvain Paris Hector M. Briceño François X.
  Sillion

ARTIS is a team of the GRAVIR - IMAG research
lab(UMR 5527 between CNRS, INPG, INRIA and UJF),
and a project of INRIA.
2
Motivation

• Digital humans more and more common
• Movies, games
• Hairstyle is important
• Characteristic feature
• ? Duplicating real hairstyle

3
Motivation

• User-based duplication of hair
• Creation from scratch
• Edition at fine level
• Image-based capture
• Automatic creation
• Copy of original features
• Edition still possible

4
Our approach
Dense set of 3D strandsfrommultiple images

• Digital copy of real hairstyle
• Only static geometry
• (animation and appearance as future work)

5
Outline

• Previous work
• Definitions and overview
• Details of the hair capture
• Results
• Conclusions

6
Previous workShape reconstruction

• Computer Vision techniques
• Shape from motion, shading, specularities
• 3D scanners
• Difficulties with hair complexity
• ? Only surfaces

7
Previous workLight-field approach

• Matusik et al. 2002
• New images from
• Different viewpoints lights
• Alpha mattes
• ? Duplication of hairstyle
• ? No 3D strands
• ? Not editable

Matusik02
8
Previous workEditing packages

• Hadap and Magnenat-Thalmann 2001
• Kim et al. 2002
• Dedicated tools to help the user

? 3D strands ? Total control ? Time-consuming ?
Duplication very hard
Hadap01 MIRALab, University of Geneva
9
Previous workProcedural Image-based

• Kong et al. 1997
• Hair volume from images
• Procedural filling
• ? 3D strands
• ? Duplication of hair volume
• ? No duplication of hairstyle
• ? New procedure for each hair type

Kong97
10
Previous workImage-based
Grabli et al. 2002 Fixed camera, moving light 3D
from shading ? 3D strands ? Duplication of
hairstyle ? Partial reconstruction (holes)
Sample input image
Captured geometry
We build upon their approach.
Grabli02
11
Our approach

• Dense and reliable 2D data
• Robust image analysis
• From 2D to 3D
• Reflection variation analysis
• Light moves, camera is fixed.
• Several light sweeps for all hair orientations
• Complete hairstyle
• Above process from several viewpoints

12
Outline

• Previous work
• Definitions and overview
• Details of the hair capture
• Results
• Conclusions

13
Definitions

• Fiber
• Strand
• Visible entity
• Segment
• Project on 1 pixel
• Orientation

1mm
14
Setup input
15
Input summary

• We use
• 4 viewpoints
• 2 sweeps per viewpoint
• 50 to 100 images per sweep
• Camera and light positions known
• Hair region known (binary mask)

16
Main steps

• Image analysis
• 2D orientation
• Highlight analysis
• 3D orientation
• Segment chaining
• 3D strands

17
Main steps

• Image analysis
• 2D orientation
• Highlight analysis
• 3D orientation
• Segment chaining
• 3D strands

All camerastogether
18
Measure of 2D orientationDifficult points

• Fiber smaller than pixel ? aliasing
• Complex light interaction
• Scattering, self-shadowing
• Varying image properties

Select measure methodper pixel
19
Measure of 2D orientationUseful information

• Many images available

Select light positionper pixel
20
Our approach
Try several options ? Use the best

• Based on oriented filters

response
? argmax K? ? I
?
90
0
180
Most reliable ? most discriminant Lowest variance
21
Filter selection
22
Implementation

• 1. Pre-processing Filter images
• 2. For each pixel, test
• Filter profiles
• Filter parameters
• Light positions
• Pick option with lowest variance
• 3. Post-processing Smooth orientations
  (bilateral filter)

2
4
8
8
23
Per pixel selection
24
2D results
8 filter profiles 3 filter parameters 9 light
positions
Our result
Sobel Grabli02
(More results in the paper)
25
Main steps

• Image analysis
• 2D orientation
• Highlight analysis
• 3D orientation
• Segment chaining
• 3D strands

All camerastogether
26
Mirror reflectionComputing segment normal
a
a
3 accuracy Marschner03
For each pixel Light position?
27
Practical measure
28
Orientation from 2 planes
(3D position determined later)
29
Main steps

• Image analysis
• 2D orientation
• Highlight analysis
• 3D orientation
• Segment chaining
• 3D strands

Camerasone by one
All camerastogether
30
Starting point of a strand

31
Chaining the segments
32
Blending weights
33
Ending criterion

• Strand grows until
• Limit length (user setting)
• Out of volume (visual hull)

34
Outline

• Previous work
• Definitions and overview
• Details of the hair capture
• Results
• Conclusions

35
Results
36
Result summary

• Similar reflection patterns
• Duplication of hairstyle
• Curly, wavy and tangled
• Blond, auburn and black
• Middle length, long and short

37
Conclusions

• General contributions
• Dense 2D orientation (filter selection)
• 3D from highlights on hair
• System
• Proof of concept
• Sufficient to validate the approach
• Capture of a whole head of hair
• Different hair types

38
Limitations

• Image-based approach only visible part
• Occlusions not handled (curls)
• Head poor approximation
• Setup makes the subject move
• During light sweep
• Between viewpoints

39
Future work

• Better setup and better head approximation
• Short term
• Data structures for editing and animation
• Reflectance
• Long term
• Hair motion capture
• Extended study of filter selection

40
Thanks Questions ?

• The authors thank Stéphane Grabli, Steven
  Marschner, Laure Heïgéas, Stéphane Guy, Marc
  Lapierre, John Hughes, and Gilles Debunne.

Rendering usingdeep shadow mapskindly provided
by Florence Bertails.
The images in the previous work appear by
courtesy by NVIDIA, Tae-Yong Kim, Wojciech
Matusik, Nadia Magnenat-Thalmann, Hiroki
Takahashi, and Stéphane Grabli.
41
Questions
Visual hull
Grazing angle
2D validation
Comparisons
Post-processing
Pre-processing
42
Reference image

• Four radial sines
• discontinuities
• Wavelength 2 pixels
• aliasing

43
Results on reference image
Our result(mean error 2.3)
Variance
44
Comparison with Sobel
Our result(mean error 2.3)
Sobel filter(mean error 17)
45
Visual hull

• 90 between the viewpoints
• Sharp edges

46
Reliable regions
? Front facing view
? Grazing view
47
Frequency selection
Band-filtering(difference of Gaussians)
Input image
48
Bilateral filtering

• Accounting for the adjacent pixels
• Spatial distance
• Filter reliability (variance)
• Appearance similarity (color)
• Weighted mean (Gaussian weights)

49
Comparison
Withoutvariance selection
Reference
50
Comparison
Withoutbilateral filtering
Reference