SuperPatches

1
Super-Patches

• Image Tessellation with Arbitrarily Shaped Patches

Eddie K. H. Ng, University of Toronto
Joint work with Brendan Frey.
2
Motivations

• Patch-based models have been successfully applied
  to many different areas of computer vision.
• Patch size / shape selection remains a skillful
  labor and confined to simple geometric shapes.
• Causes problems in part-based object recognition.

3
Goals

• Learn a set of coherent patches of arbitrary
  shapes and sizes.
• Use the resulting patches to enhance performance
  of various tasks in computer vision.

4
Inspiration

• Super-pixels Ren Malik, 2003
• Create a local, coherent entity while
  maintaining overall structure
• Reduce computational complexity.

X. Ren and J. Malik. Learning a classification
model for segmentation. In Proc. 9th Int. Conf.
Computer Vision, volume 1, pages 10-17, 2003.
Image taken from http//www.cs.sfu.ca/mori/resear
ch/superpixels/.
5
Basic Concepts 1

• The shape and size of each patch is specified
  through a kernel function (e.g. Gaussian (m, s)).
• The set of kernel functions compete for the
  ownership each pixel.

Initial Conditions
Patches after Competition
Input Image
Input Image
6
Basic Concepts 2

• The tessellation is then a map showing the
  respective ownerships.
• Free parameter - number of patches

Patches after Competition
Tessellated Image
Input Image
7
Basic Concepts 4

• But how does the patches compete with each other?
• Each patch is mapped to a region in the Epitome.

Prior knowledge of Image Classes (epitome)
Tessellated Image
Input Image
8
Basic Concepts 5

• Cost Function
• Optimize cost function using conjugate gradient

9
Run Demo
10
Other Explored Paths

• Variations on the cost function
• Garbage Model
• Isotropic Patches
• Shape Regularization
• Grow-and-Merge
• Different formulation of the problem
• Other kernel functions
• Other learning algorithms

11
References

• Epitome
• N. Jojic, B. J. Frey, A. Kannan, Epitomic
  analysis of appearance and shape , ICCV 2003.
• Low-level vision
• W.T. Freeman, E.C. Pasztor, O. T. Carmichael,
  Learning Low-level vision, International Journal
  of Computer Vision 40(1), 25-47, 2000
• Super-pixels
• X. Ren and J. Malik. Learning a classification
  model for segmentation. In Proc. 9th Int. Conf.
  Computer Vision, volume 1, pages 10-17, 2003.
• Web resources
• www.psi.utoronto.ca
• http//research.microsoft.com/jojic/epitome.htm