Gradient-Based Image Editing

1
Gradient-Based Image Editing
2
Poisson Image Editing

• Patrick Perez, Michel Gangnet and Andrew Blake
• Microsoft Research UK

3
Outline

• Introduction
• Poisson solution to guided interpolation
• Guided Interpolation
• Discrete Poisson solver
• Seamless cloning
• Importing gradients
• Mixing gradients
• Selection editing
• Texture flattening
• Local illumination change
• Local color change
• Seamless tiling
• Conclusion

4
Introduction

• A generic interpolation machinery based on
  solving Poisson equations
• A variety of novel tools for seamless editing of
  image regions
• Import the source image region to a target region
• Modify appearance seamless in selected image
  region illumination, color, texture, etc.

5
Introduction

• Image editing
• Global changes (color/intensity corrections,
  Filters, deformations)
• Local changes confined to a selection
• Selected region
• Seamless
• Changes slight distortion -gt complete
  replacement

Seams partly hidden by feathering
6
Introduction

• Seamless editing and cloning of a selection
  region
• Mathematical tool
• Poisson partial differential equation
• Laplacian of unknown function over the domain of
  interest
• Dirichlet boundary conditions
• Known function value along the boundary

7
Boundary Conditions

• Dirichlet boundary condition
• It specifies the values of a solution needs to
  take on the boundary of the domain
• Neumann boundary condition
• It specifies the values of the derivative of a
  solution is to take on the boundary of the domain

8
Poisson Equation

• Second-order variations extracted by Laplacian
  operator are the most significant perceptually
• Scalar function on a bounded domain is uniquely
  defined by its values on the boundary and its
  Laplacian in the interior
• Poisson equation therefore has a unique solution

9
Poisson Equation

• Solving Poisson equation interpretation as
  minimization problem
• Gradient of the function is the closest (in
  L2-norm) to some prescribed vector field
  (Guidance vector field) under given boundary
  conditions
• Possible choice of guidance vector filed
• Source image
• Mixing suitably gradient of source and target
  image
• Modify a region of the image

10
Related Work

• Poisson equation has been used extensively in
  computer vision
• Rescale gradient field of High Dynamic Range
  (HDR) image
• Solving Poisson equation with Neumann boundary
• Edit image via a sparse set of edge elements
• Solving a Laplace equation with Dirichlet
  boundary
• Spot removing
• Replace the brightness by harmonic interpolation
  (solving a Laplace equation)

11
Poisson Solution to Guided Interpolation

• Guided Interpolation

S Image domain
Closed subset of S
Guidance vector field
fDestination function f Unknown function
Gradient field of a source function
12
Guided Interpolation

• Membrane interpolant

The minimizer must satisfy the associated
Euler-Lagrange equation
13
Guided Interpolation

• Guidance filed a vector field v

The fundamental machinery of Poisson editing of
color images independently solve the three
color channels
14
Guided Interpolation

• With a conservative vector field v (it is the
  gradient of some gradient function)
• Correction function
• Poisson equation -gt Laplace equation

Membrane interpolant
15
Discrete Poisson solver

• Can be discretized and solved in a number of ways
• S, become finite point sets defined on a
  discrete grid
• For each pixel p in S, Np is its 4-connectd
  neighbors in S
• The boundary
• Pixel pair
• The task is to compute the pixel values over
  the region

16
Discrete Poisson solver

• Dirichlet boundary conditions defined on a
  boundary of arbitrary shape, so discrete the
  problem directly rather than the Poisson equation

17
Discrete Poisson solver

• Solution linear equations
• For all pixels p interior to , no boundary
  terms

A classical, sparse, symmetric, positive-definite
system
18
Discrete Poisson solver

• Arbitrary shape of boundary , use
  iterative solvers
• Gauss-Seidel iteration
• V-cycle multigrid
• Both are fast enough for interactive editing

19
Seamless Cloning

• Importing gradients
• Gradient field directly from source image g

20
Seamless Cloning

• Very simple user interaction by drawing outlines

21
Feature Exchange

• Draw precise boundary for specific object

22
Seamless Cloning

• Transfer only part of the source content

23
Inserting object with holes
Seamless cloning
Color-based cut-and-paste
Seamless cloning and destination averaged
Mixed seamless cloning
24
Mixing gradients

• Linear combination of source and destination
  gradient fields
• Non-conservative guidance fields

25
Mixing gradients

• Transparent object

26
Mixing Graidents
Source/destination Seamless cloning
mixed seamless cloning
27
Selection Editing

• Modify one image in its gradient domain
• Texture flattening
• Spatially selective illumination changes
• Background and foreground color modification
• Seamless titling

Non-linear modification of the original gradient
field
Modify the original image, provide a new source
image or new boundary
28
Texture Flattening

• M a binary mask tuned on at a few locations of
  interest
• Preserve main structure

29
Local Illumination Changes

• Modify the gradient of original image then use it
  as a new guided vector field
• Modify locally the apparent illumination
• Highlight under-exposed foreground object
• Reduce specular reflections

30
Local Color Changes

• Given a source color image g
• The destination image f is set to be the
  luminance channel from g,
• The user selects a region containing the
  object, and this may be some what bigger than the
  actual object
• Modify the colors of background or in the
  selected region as new destination/source image
• The Poisson equation is solved in each color
  channel.

31
Local Color Changes
new source image by multiplying the RGB channels
by 1.5, 0.5, 0.5
Original Image
Decolorization f is the luminance of g

32
Seamless tiling

• Seamless tiling by enforcing periodic boundary
  conditions with Poisson solver
• Use original image as source image, set identical
  boundary from opposite sides of the original
  boundary

33
Conclusion

• A Generic framework of guided interpolation
• A series of tools to edit in a seamless and
  effortless manner the contents of an image
  selection
• The modification can be
• Replacement by
• Mixing with other images
• Alternations of some aspects of the original
  image, such as texture, illumination or color
• No need for precise object delineation

34
Extensions to Poisson Editing

• 3D mesh editing
• Guided vector field
• Image matting
• Good boundary finding

35
Drag-and-Drop Pasting

• http//www.cse.cuhk.edu.hk/leojia/all_project_web
  pages/ddp/drag-and-drop_pasting.html

36
(No Transcript)
37
Detail Preserving Shape Deformation in Image
Editing

• http//graphics.cs.uiuc.edu/huifang/deformation.h
  tm

38
photomontage

• http//grail.cs.washington.edu/projects/photomonta
  ge/

39
Efficient Gradient Domain Editing

• http//agarwala.org/efficient_gdc/comparisons