Detail Preserving Shape Deformation in Image Editing

1
(No Transcript)
2
Detail Preserving Shape Deformation in Image
Editing

• Hui FangGoogle, Inc.
• John C. HartUniversity of Illinois,
  Urbana-Champaign

3
Original
Morphed
Retextured
4
Relation to Previous Work

• Detail preserving image deformation relies on
  texture synthesis
• Pixel based Heeger and Bergen 1995 Wei and
  Levoy 2000
• Patch based Efros and Freeman 2001 Kwatra et
  al. 2003
• Using the original image as the texture source
• Image Analogies Hertzmann et al. 2001
• Texture by Numbers Brooks et al. 2003
• We pay additional attention to the behavior of
  the texture synthesized around a user specified
  control curve
• Feature map Wu and Yu 2004
• Guidance vector field Kwatra et al. 2005

5
Relation to Previous Work

• v. Image Completion Sun et al. 2005
• extended isotropic image textures via linear
  motion
• whereas
• we deform each texture patch non-linearly to fit
  a deformed silhouette
• v. Object-Based Image Editing Barrett and Cheney
  2002
• populated masked regions with cut-and-paste
  patches
• whereas
• our textures are feature aligned at pixel scale

6
Feature Aligned Retexturing

• Deformation of feature curves
• Curvilinear Coordinates generated from feature
  curves
• Textured Patch Generation
• Image Synthesis

7
Deformation of Feature Curves

• Each pair of pixels in control curves defines a
  deformation vector
• Deformation vectors interpolated smoothly across
  whole image by solving aLaplacian

D
??2D(x,y) 0
8
Curvilinear Coordinates

• We create matching sets of curvilinear
  coordinates around source target feature curves

9
Curvilinear Coordinates

• We create matching sets of curvilinear
  coordinates around source target feature curves
• We first define tangents along the source curve

T
10
Curvilinear Coordinates

• We create matching sets of curvilinear
  coordinates around source target feature curves
• We first define tangents along the source curve
• And then solve a Laplacian ?2T(x,y) 0to
  interpolate it across the source image

T
11
Curvilinear Coordinates

• We create matching sets of curvilinear
  coordinates around source target feature curves
• We first define tangents along the source curve
• And then solve a Laplacian ?2T(x,y) 0to
  interpolate it across the source image
• Do the same to the target

T
T
12
Curvilinear Coordinates

• We apply Euler integration tosource target
  tangent fields
• Extend source target spinecurve parallel to
  tangent field
• qj1,0 qj,0 eT(qj,0)
• Extend source target ribcurves perp. to
  tangent field
• qj,k1 qj,k e 100-1 T(qj,k)
• Smooth result by iterating qj,k qj,k 70
  ?2qj,k

13
Textured Patch Generation

• Synthesize texture on target coordinate patch
  using texture from source coordinate patch
• Source colors sampled at curvilinear coordinates
  are bilinearly filtered from surrounding integer
  pixel locations
• Target colors defined at curvilinear coordinates
  splat onto surrounding image pixels with a unit
  radius cone filter

14
Image Synthesis

• Individual feature-aligned texture patches grown
  and merged using GraphCut Kwatra et al. 2003
• Patch synthesis order prioritized by proximity to
  feature curve and previously synthesized patches
• Poisson Image Editing Perez et al. 2003 further
  hides boundary between neighboring patches.

15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
Scale Adaptive Retexturing

• The deformation field can compress large source
  areas into small target areas
• Causes texture continuity problems and blockiness
• Use smaller texture patches in areas where
  deformation compresses highly

19
Video demo
20
Original
Morphed
Retextured
21
Original
Morphed
Retextured
22
Hair Beards
Morphed
Retextured
23
Original
Morphed
Retextured
24
Morphed
Retextured
25
(No Transcript)
26
Failure case
27
Running time
28
Questions?