Digital Face Cloning

1
Digital Face Cloning

• David Bennett Christophe Hery Steve Sullivan
• George Borshukov J.P. Lewis Lance Williams
• Paul Debevec Fred Pighin Li Zhang

2
Boston
Seattle
San Francisco
Los Angeles

• David Bennett Christophe Hery Steve Sullivan
• George Borshukov J.P. Lewis Lance Williams
• Paul Debevec Fred Pighin Li Zhang

3
Definition

• Process of capturing an actors performance
  and/or likeness in a digital face model

4
E.g. photogrammetry (input)
5
E.g. photogrammetry (output)
6
Motivation

• Replicate a live actor (e.g. celebrity)
• Stunts
• Extreme makeup
• Younger/older/slimmer clone
• Creating realistic faces from first principles is
  difficult (digitizing is easier than modeling)

7
Purpose of the course

1. Review the field
2. Suggest directions

8
Digital Clones History
9
CG Humans in the movies

• First 3D actor representation in the movies
  Peter Fonda in FutureWorld (1976!)
• Not intended to be photorealistic

10
1990s
11
1990s

• Terminator 2 (brief, texture projection)
• Jurrasic Park (legs only, does not count)
• Titanic (distant)
• Startup attempt Virtual Celebrity/Global Icons,
  purchased virtual rights to Sammy Davis Jr.,
  others

12
1998 Hollywood is ready

• "I don't see them as competing with real actors,
  Richard Masur, head of Screen Actors Guild
• Projected uses de-aging, celebrity vacuum ads.
• A television series with a virtual cast was
  considered

13
Recent and current
14
Recent and current

• Space Cowboys (2000), Enemy at the Gates (2001)
  brief and distant, but successful
• Disney Facial Cloning test (afternoon session)
• Matrix sequels (2003) superpunch full-screen
  for 13 seconds (afternoon session)
• SpiderMan 2 (reflectance capture)
• Lemony Snicket CG baby in several shots
  (afternoon session)
• Discovery Channel Churchill/Hitler clones (MPC)

15
Observation

• There has not been a breakthrough introduction
  of digital clones
• Rather, clones were initially used in the
  distance, and briefly they are getting closer,
  and withstanding longer study.
• Current state of the art full screen for
  several seconds (see afternoon Perception session)

16
Syllabus morning (theory)

• 830 Introduction and Overview Fred Pighin
• 900 Face Scanning and Dense Motion Capture
  Technologies Li Zhang
• 1000 Break or Mathematical Background (schedule
  change)
• 1015 Reflectance Modeling and Capture Paul
  Debevec and J.P. Lewis
• 1115 Facial Parameterization and Cross-Mapping
  J.P. Lewis and Fred Pighin
• 1215 Lunch

17
Syllabus afternoon (practice)

• 130 Case study Face Cloning at ILM Steve
  Sullivan and Christophe Hery
• 215 Perception Experiment (schedule change)
• 230 Case Study Disney Facial Cloning Lance
  Williams
• 315 Case study Leaping the Uncanny Valley with
  Data   (Face Cloning in the Matrix sequels)
  George Borshukov
• 400 Break
• 415 Case study Polar Express David Bennet
• 500 Perception of Facial Realism J.P. Lewis
• 515 Panel on the Future of Digital Face Cloning
  All

18
Face Cloning at Siggraph

• Courses
• 10. Realistic Materials in Computer Graphics
• 28. From Mocap to Movie The Making of "The Polar
  Express
• 29. High-Dynamic-Range Imaging and Image-Based
  Lighting
• Papers (Skin Faces session on Monday)
• Automatic Determination of Facial Muscle
  Activations From Sparse Motion Capture Marker
  Data
• Face Transfer With Multilinear Models
• Postproduction Relighting and Reflectance
  Transformation with Time-Multiplexed Illumination
  
• Sketches
• Image-Based Rendering From a Sparse Set of Images
• Implementation of Modeling Hair From Multiple
  Views
• Model Flowing Capturing and Tracking of
  Deformable Geometry
• Performance Geometry Capture for Spatially
  Varying Relighting
• Joint Motion and Reflectance Capture for
  Relightable 3D Video