Computational photography MIT 6.098, 6.882 Bill Freeman, Fredo Durand

1
Computational photography MIT 6.098, 6.882 Bill
Freeman, Fredo Durand

• Finish digital forensics
• Analyzing multiple images
• Shapetime photography
• Image stacks
• Analysing and synthesizing motion sequences
• Motion without movement
• Motion magnification

April 20, 2006
2
Analyzing multiple images

• Bill Freeman
• Fredo Durand
• MIT Computational Photography, 6.882
• April 20, 2006

3
Multiple-exposure images by Marey
4
Strobe photograph by Edgerton
5
Other photographs by Doc Edgerton
6
What hardware was needed to make these
photographs?

• Strobe light, capacitor, thyristor

7
Computational photography

• Surely we can update those photographic
  techniques, adding the generality and flexibility
  of digital methods. Analyze and re-render the
  images.

8
Computational photography

• Fredo and Bill describing computational
  photography
• Fredo using computation to make better quality
  photographsto enhance.
• Bill using computation to reveal things about
  the world that we otherwise couldnt seeto
  reveal.

9

• How display a single-frame summary of multiple
  frames?

10
(No Transcript)
11
Typical frame
12
Average over 50 frames
13
Median filter over time
14
Vector median filter (20x20 patchs)
15

• 2x2 vector median 2x2 vector least median

16
Shapetime photography

• Joint work with Hao Zhang, U.C. Berkeley
• 2002

17
Video frames
18
(No Transcript)
19
Resulting composite image
Frame index of each displayed pixel
20
Frame index of each displayed pixel
Resulting composite image
With edge-preserving regularization
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
Z-cam, made by 3DV
http//www.3dvsystems.com
26
3DV camera operation
http//www.3dvsystems.com
27
3DV camera operation
http//www.3dvsystems.comv
28
3DV camera operation
http//www.3dvsystems.com
29
RGB image
30
Z image
31
shapetime video image
32
(No Transcript)
33
Zitnick et al, Siggraph 2004
34

• Show Michael Cohen slides, a selection from
• http//research.microsoft.com/cohen/FindingMagic
  InAnImageStack.pdf

35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44

• Demonstrate MSR group shot program, downloadable
  from
• http//research.microsoft.com/cohen/
• or
• http//research.microsoft.com/projects/GroupShot/

45
Analyzing and synthesizing motion

• Bill Freeman
• Fredo Durand
• MIT Computational Photography, 6.882
• April 20, 2006

46
Aperture Problem and Normal Flow
47
Aperture Problem and Normal Flow
48
Aperture Problem and Normal Flow
49
Aperture Problem and Normal Flow
50
Aperture Problem and Normal Flow
51
Aperture Problem and Normal Flow
52
Optical flow constraint equation
53
Aperture Problem and Normal Flow
The gradient constraint
Defines a line in the (u,v) space
Normal Flow
54
Combining Local Constraints
v
etc.
u
55
Lucas-Kanade (a good, generic motion analysis
method) Integrate gradients over a patch
Assume a single velocity, u, v, for all pixels
within an image patch. Find the (u, v) that
minimizes the BCCE squared residual over the
patch
Setting derivative w.r.t. (u, v) equal to zero
gives
Note similarity of LHS matrix to Harris corner
detector. When full-rank (corner-like),
specifies a unique (u, v).
56
Motion without movement

• Joint work with Ted Adelson and David Heeger, MIT
• 1991

57
A linear combination of quadrature-phase filters
can advance the local phase
58
Convolved with an image, the image data now
modulates the local amplitude. People
mis-attribute the phase advance to translation.
(Steerable filters allow synthesizing motion in
arbitrary directions.)
59
Motion without movement video
60
http//www.cs.yorku.ca/kosta/Motion_Without_Movem
ent/Motion_Without_Movement.html
61
http//www.cs.yorku.ca/kosta/Motion_Without_Movem
ent/Motion_Without_Movement.html
Konstantinos G. Derpanis
62
Motion Magnification

• (go to other slides)