Image Inpainting

1
Image Inpainting

• Marcelo Bertalmío, Minnesota
• Guillermo Sapiro, Minnesota
• Vicent Caselles, Barcelona
• Coloma Ballester, Barcelona

SIGGRAPH 00
2
Overview

• What is image inpainting
• Related work
• Digital inpainting
• Examples
• Concluding remarks

3
What is inpainting?

• Modifying an image in a non-detectable form
• Detail of "Cornelia, Mother of the Gracchi" by J.
  Suvee (Louvre).
• Taken from Emile-Male The Restorers Handbook of
  easel painting.

4
Photo restoration

• Restoration courtesy of D. Henry, Precious Photos
  Inc.

5
Object removal

• From D. King, The Commissar vanishes.

6
Related work Films

• e.g. Kokaram et al.
• Doesnt work for stills or static objects

n-1
n1
n
7
Related work Texture synthesis

• Hirani, Efros, Heeger, DeBonet, Simoncelli, etc.
• Not practical for rich regions
• Not designed for structured regions
• Copy information instead of learn and
  interpolate

8
Related work Disocclusion

• Masnou-Morel, Nitzberg-Mumford, etc.
• Limitations Topology, angles

See also Chan-Shen 00
9
Our Contribution

• User only selects region to inpaint
• Rich background and topology not an issue
• Less than 5 minutes on a PC

10
How conservators inpaint

• Minneapolis Institute of Art

11
Automatic digital inpainting

• Propagate information
• Evolutionary form

12
Digital inpainting (contd)

• L smoothness estimator (Laplacian)
• N isophote direction (time variant)

13
The equation

• Plus numerical schemes (Osher)
• Boundary conditions
• Gray values (in a band)
• Directions (in a band)

14
Example
15
Example Text removal
16
Example Photo restoration
17
Example Special effects
18
Example Scratch removal
19
Example The evolution
20
Example Structure but not texture is reproduced.
21
Extension A variational formulation

• Solved via E-L Coupled 2nd order PDEs
• Full theory given
• Similar practical results than 3rd order equation

See also Chan-Shen 00
22
Concluding remarks

• Technique imitates professionals
• Key concepts
• Information propagation
• Both gray values and directions are needed
• Use a band surrounding the region
• Sharp results
• Low complexity
• Texture is not reproduced

23
Concluding remarks (cont.)

• Connected to thin film and fluid dynamics (A.
  Betozzi)
• Opens then door to high order PDEs
• Extended to a variational formulation

24
Acknowledgments

• Institute Henri Poincare in Paris, France.
• Tom Robbins, Elizabeth Buschor,
  Santiago Betelú, Stan Osher, Eero Simoncelli,
  Andrea Bertozzi.
• Supported by ONR-Math, ONR Young Investigator
  Award, Presidential Early Career Awards for
  Scientists and Engineers, NSF CAREER Award,
  NSF-LIS, and IIE-Uruguay.

25
The end

• Thank you