COS 429: Computer Vision

1
COS 429 Computer Vision
Thanks to Chris Bregler
2
COS 429 Computer Vision

• Instructor Szymon Rusinkiewicz
  smr_at_cs.princeton.edu
• TA Nathaniel Dirksen ndirksen_at_cs.princeton.e
  du
• Course web page http//www.cs.princeton.edu/cours
  es/cs496/

3
What is Computer Vision?

• Input images or video
• Output description of the world
• But also measuring, classifying, interpreting
  visual information

4
Low-Level or Early Vision

• Considers local properties of an image

Theres an edge!
5
Mid-Level Vision

• Grouping and segmentation

Theres an object and a background!
6
High-Level Vision

• Recognition

Its a chair!
7
Big Question 1 Who Cares?

• Applications of computer vision
• In AI vision serves as the input stage
• In medicine understanding human vision
• In engineering model extraction

8
Vision and Other Fields
Computer Vision
9
Big Question 2 Does It Work?

• Situation much the same as AI
• Some fundamental algorithms
• Large collection of hacks / heuristics
• Vision is hard!
• Especially at high level, physiology unknown
• Requires integrating many different methods
• Requires reasoning and understandingAI
  completeness

10
Computer and Human Vision

• Emulating effects of human vision
• Understanding physiology of human vision
• Analogues of human vision atlow, mid, and high
  levels

11
Image Formation

• Human lens forms image on retina,sensors (rods
  and cones) respond to light
• Computer lens system forms image,sensors (CCD,
  CMOS) respond to light

12
Low-Level Vision
Hubel
13
Low-Level Vision

• Retinal ganglion cells
• Lateral Geniculate Nucleus function
  unknown(visual adaptation?)
• Primary Visual Cortex
• Simple cells orientational sensitivity
• Complex cells directional sensitivity
• Further processing
• Temporal cortex what is the object?
• Parietal cortex where is the object? How do I
  get it?

14
Low-Level Vision

• Net effect low-level human visioncan be
  (partially) modeled as a set ofmultiresolution,
  oriented filters

15
Low-Level Depth Cues

• Focus
• Vergence
• Stereo
• Not as important as popularly believed

16
Low-Level Computer Vision

• Filters and filter banks
• Implemented via convolution
• Detection of edges, corners, and other local
  features
• Can include multiple orientations
• Can include multiple scales filter pyramids
• Applications
• First stage of segmentation
• Texture recognition / classification
• Texture synthesis

17
Texture Analysis / Synthesis
Multiresolution Oriented Filter Bank
OriginalImage
Image Pyramid
18
Texture Analysis / Synthesis
Original Texture
Synthesized Texture
Heeger and Bergen
19
Low-Level Computer Vision

• Optical flow
• Detecting frame-to-frame motion
• Local operator looking for gradients
• Applications
• First stage of tracking

20
Optical Flow
Image 1
Optical FlowField
Image 2
21
Low-Level Computer Vision

• Shape from X
• Stereo
• Motion
• Shading
• Texture foreshortening

22
3D Reconstruction
TomasiKanade
Forsyth et al.
Phigin et al.
Debevec,Taylor,Malik
23
Mid-Level Vision

• Physiology unclear
• Observations by Gestalt psychologists
• Proximity
• Similarity
• Common fate
• Common region
• Parallelism
• Closure
• Symmetry
• Continuity
• Familiar configuration

Wertheimer
24
Grouping Cues
25
Grouping Cues
26
Grouping Cues
27
Grouping Cues
28
Mid-Level Computer Vision

• Techniques
• Clustering based on similarity
• Limited work on other principles
• Applications
• Segmentation / grouping
• Tracking

29
Snakes Active Contours

Contour Evolution forSegmenting an Artery
30
Histograms
Birchfeld
31
Expectation Maximization (EM)
Color Segmentation
32
Bayesian Methods

• Prior probability
• Expected distribution of models
• Conditional probability P(AB)
• Probability of observation Agiven model B

33
Bayesian Methods

• Prior probability
• Expected distribution of models
• Conditional probability P(AB)
• Probability of observation Agiven model B
• Bayess Rule P(BA) P(AB) ? P(B) / P(A)
• Probability of model B given observation A

Thomas Bayes (c. 1702-1761)
34
Bayesian Methods
black pixels
black pixels
35
High-Level Vision

• Human mechanisms ???

36
High-Level Vision

• Computational mechanisms
• Bayesian networks
• Templates
• Linear subspace methods
• Kinematic models

37
Template-Based Methods
Cootes et al.
38
Linear Subspaces
39
Principal Components Analysis (PCA)
Data
New Basis Vectors
PCA
Kirby et al.
40
Kinematic Models

• Optical Flow/Feature tracking no constraints

• Layered Motion rigid constraints

• Articulated kinematic chain constraints

• Nonrigid implicit / learned constraints

41
Real-world Applications
Osuna et al

42
Real-world Applications
Osuna et al

43
Course Outline

• Image formation and capture
• Filtering and feature detection
• Motion estimation
• Segmentation and clustering
• PCA
• 3D projective geometry
• Shape acquisition
• Shape analysis

44
3D Scanning
45
Image-Based Modeling and Rendering
Debevec et al.
Manex
46
Course Mechanics

• 65 4 written / programming assignments
• 35 Final group project

47
Course Mechanics

• BookIntroductory Techniques for 3-D Computer
  VisionEmanuele Trucco and Alessandro Verri
• Papers

48
COS 429 Computer Vision

• Instructor Szymon Rusinkiewicz
  smr_at_cs.princeton.edu
• TA Nathaniel Dirksen ndirksen_at_cs.princeton.e
  du
• Course web page http//www.cs.princeton.edu/cours
  es/cs496/