Environment Matting and Compositing

1
Environment Matting and Compositing
CS 395 Advanced Graphics Presented by Jay
Hetler Douglas E. Zongker Dawn M. Werner
Brian Curless David H. Salsin
2

• What is Matting?
• Extracting the foreground image from the
  background
• Describing the opacity of the foreground at every
  point

• What is Compositing?
• Holding back the part of the new background that
  theforeground blocks.
• Placing the foreground element in a new
  background

3
Original Uses
Weather Reports
Cinema Special Effects
4
Matting Equation

• C F (1- a)B
• C Color
• F Foreground color
• B Background color
• a Amount of light that passes through the
  foreground

5
How difficult is Matting?
6
Matte Image
7
Good Results?
8
Problems with Matting

• Images do not look realistic
• Lack of Refracted Light
• Lack of Reflected Light

Solution Modify the Matting Equation
9
Environment Matting Equation

• C F (1- a)B F
• C Color
• F Foreground color
• B Background color
• a Amount of light that passes through the
  foreground
• F Contribution of light from Environment that
  travels through the object

10
Explanation of F

• F R M(T , A )

m
S
i
i
i
i 1
R Reflectance coefficient M Texture mapping
operator for axis-aligned rectangle (A) of
texture (T)
11
Environment Mattes
12
Performance

• Calibration
• Matting 10-20 minutes extraction time for each
  texture map (Pentium II 400Mhz)
• Compositing 4-40 frames per second
• Real-Time?

13
How much better is Environment Matting?
Alpha Matte Environment Matte
Photograph
14
How much better is Environment Matting?
Alpha Matte Environment Matte
Photograph
15
Movies!
16
Importance of Sidedrops
Light comes from other parts of the environment
17
Sidedrop Example
18
Sidedrop x2
19
Failure Cases
Box-filter
Gaussian-weighted Photograph
20
Failure Cases
Failure of Engineering
Failure of Theory
Composite Photograph
21
Next Steps

• Depth
• Accuracy
• Real-Time Capture

22
Progress?
Depth
Accuracy
No Real-Time