Computer Vision

1
Computer Vision

• by
• Paul Bates

2
Introduction

• Human vision
• Complex the way we perceive, analyze, and
  classify images we see continues to be a mystery
• General idea of sight is understood
• Retina and first layer of cells break down images
  into simple elements like edges and line segments
• How the brain recognizes an entranceway or the
  face of a friend is puzzling
• Brain stores different elements of objects in
  different parts of the brain (size, color,
  texture) and links them
• Process is not understood

3
Introduction(contd)

• Computer vision
• Same problems of recognition remain
• Where do we start in order for a computer to
  see an image?
• Modeled after human vision but we need algorithms
  to imitate the brains way of recognizing things
• Far off from human vision and understanding of
  objects but we are advancing

4
Introduction(contd)

• Ancient temple
• example
• How does the brain
• perceive this image?
• It must break it into simpler elements such as
• 2 horizontal slabs
• 2 vertical bars
• 1 horizontal bar
• Before recognizing this as an archway on a
  platform or an ancient temple, there must be
  preliminary steps taken to break then image down
• A computer will need to do the same thing

5
Vocabulary

• Polyhedral scene an assembly of solids each of
  which is bounded by plane faces
• Convex bulging outward
• Concave going inward

6
Overview

• History of computer vision
• Pioneers
• Concept of labeling 3-D shapes
• How it works
• Examples
• Breakdown of ancient temple example
• Applications
• Where can we use computer vision?

7
History

• David A. Huffman was formerly a professor at MIT,
  known for Huffman Coding, interests were
  efficient coding algorithms, scene analysis, the
  physical and mathematical properties of surfaces
  such as paper3, and many other areas such as
  electrical engineering, died at 74 from cancer in
  19992.
• He contributed to computer vision with a labeling
  algorithm

8
History (contd)

• Maxwell Clowes was a British researcher in
  artificial intelligence who did pioneering work
  on interpretation of pictures by computers,
  Clowes and Huffman together came up with their
  labeling scheme in images5.
• David Waltz (pictured on right) is the director
  of the Center for Computational Learning Systems,
  he extended Huffman and Clowes work in labeling
  3-D shapes in 1971 at MIT1

9
Huffman-Clowes Labeling Scheme

• This algorithm labels edges and vertices to
  represent a 3-D shape4
• A requirement of the algorithm is that the
  drawing must be well formed
• Every line must connect to at least one other
  line at each of its endpoints
• Ambiguous and impossible shapes will not be able
  to be interpreted with the Huffman-Clowes
  labeling scheme as feasible 3-D forms
• Since solids meet along straight line segments,
  they show only a finite number of relationships
  where two or more of them meet

10
Huffman-Clowes Labeling Scheme (contd)

• In the temple example, there are essentially only
  5 relationships where lines meet which are
  pictured below1

11
Huffman-Clowes Labeling Scheme (contd)

• These relationships are called junctions

12
Huffman-Clowes Labeling Scheme (contd)

• and signs are needed to give the viewer the
  perspective of an edge
• Arrows ? are used for obscuring edges (edges
  which represent the visual boundary of a
  particular solid within a scene
• C is meant for cracks in the solids

13
Huffman-Clowes Labeling Scheme (contd)

• For example, you could have 3 obscuring edges
  coming together at a point and that would be
  called a fork (one of the previous junctions)
• To find out which way it faces, the and signs
  come in
• A fork that faces towards the viewer would be
  and an edges that faces away from the viewer
  would be - - -

14
Huffman-Clowes Labeling Scheme (contd)

• As far as the temple example is concerned, there
  are only 13 labeled junctions that are possible
  in this scene1

15
(No Transcript)
16
Huffman-Clowes Labeling Scheme (contd)

• Since most drawings are unambiguous, it is
  possible to arrive at a labeling of the vertices
  that can provide a description of the 3-D shapes
  vertices and edges
• Ambiguous and impossible shapes can cause
  problems which I will go into later

17
Combining Relationships

• Waltzs analysis program is virtually ensured
  because of the interaction between the junctions
  which carry a line segment1
• The reason is because combining relationships
  does not change their physical identity in going
  from one junction to another
• Combining these junctions is allowed as long as
  it does not change their identity being their
  direction or which way they face

18
Combining Relationships (contd)

• For example, if a fork shares a line segment with
  a T forming its upright, there are many
  possible combinations of this, BUT only 4 of
  these can actually occur as far as the list is
  concerned
• (F2, T1), (F2, T5), (F3, T3), (F3, T4)

19
Combining Relationships (contd)

• (F2, T1), (F2, T5), (F3, T3), (F3, T4)

• As long as the line segments match, the
  combination is possible

20
Combining Relationships (contd)

• A good example of how different relationships
  might not work together take F2 T3 for example

21
Combining Relationships (contd)

• If we combine these as before with T3 as an
  upright the combination is not possible

22
Analyzing an Image

• Take the temple example

• To analyze this image, the program needs a place
  to start

23
Analyzing an Image (contd)

• The program breaks down the image into every
  simple junction that was possible at all points
  and then starts at an arbitrary junction to start
  labeling

24
Analyzing an Image (contd)

• The starting junction is a fork and has 3 labels
  to start off that it could possibly be F1, F2,
  or F3
• Below the fork is an arrow junction with the
  possible labels of A1 and A2

25
Analyzing an Image (contd)

• The program tries to label both junctions and
  when comparing, discovers that only will work
  for the line segment

26
Analyzing an Image (contd)

• This means that the fork junction list is reduced
  from 3 possibilities to only 1, F2

27
Analyzing an Image (contd)

• The arrows junction list only gets reduced to A2
  when the adjacent T junctions are analyzed

28
Analyzing an Image (contd)

• When the program analyzes the L junction adjacent
  to the T junction it cannot reduce its list with
  the given information leading to an ambiguity

29
Analyzing an Image (contd)

• This particular ambiguity is cleared up by
  Waltzs special device that recognizes that this
  junction lies on the visual boundary of the
  entire image and is given a ? to signify the edge
  of the image leading to the label of L1

X
30
Analyzing an Image (contd)

• Continuing from junction to junction we
  eventually end up with this image

31
Analyzing an Image (contd)

• With this restricted version of the program we
  get unique labels for all line segments and have
  15 ambiguities

Ambiguities are lines segments with multiple
labels possible since the program could
not decide what they were
32
Analyzing an Image (contd)

• This version has limited knowledge of polyhedral
  scenes and knows nothing of support relationships

33
Analyzing an Image (contd)

• Possible ambiguities

Do we know if the 2 pillars are really floating a
couple inches above the platform?
Should the outermost edges be cracks or obscuring
edges?
34
Waltzs Complete Program

• Waltzs complete program has extra features with
  a much larger set of junctions including shadows
  to help remove ambiguities
• These junctions are symbolized by an arrow
  crossing the appropriate line segment towards a
  shadow

35
Waltzs Complete Program (contd)

• A few of these shadow junctions include the
  picture below adding T6, A3, and K2 to the list
  of junctions possible

36
Waltzs Complete Program (contd)

• When we include these few shadow junctions and
  view the image with shadows, it appears like this

37
Waltzs Complete Program (contd)

• Including shadows into the program gives rise to
  many more junctions than this but with the few
  added, all but 2 previous ambiguities are cleared
  up

38
Waltzs Complete Program (contd)

• With larger, more complex sets, many more images
  will be able to be analyzed
• The input to Waltzs program was a list of the
  images junctions, line segments, and regions
• 3 things that could give this list was
• A crisp, well-lighted polyhedral scene
• A digitizing camera, interface, and computer
• And an effective line-finding program

39
Waltzs Complete Program (contd)

• It is interesting to note that this idea was
  entirely topological considering there was no
  input of distances at all
• His program gives a single or possibly a multiple
  label for each line segment inputted to the
  program
• It would then be possible for another program to
  identify objects in a scene according to the
  labels that the program outputs

40
Waltzs Complete Program (contd)

• Waltzs program is far too complex to list
  everything here but it uses a large database of
  labeled junctions and line segments as well as
  various selection rules to aid in eliminating
  impossible combinations of labels assigned to
  junctions and line segments
• To summarize how his program eliminates labels
  for junctions
• It creates a list of all possible labels for a
  junction of that type
• It looks at adjacent junctions, using their
  current label lists to restrict the possibilities
  at this junction
• Using the reduced list of labels obtained, it
  removes any newly impossible labels from adjacent
  junctions and continues to propagate such
  restrictions outward as far as they can occur

41
Problems for Waltzs Program

• Sometimes even our own eyes get tricked with
  different images
• The same will happen for the program when faced
  with different images such as these

42
Problems for Waltzs Program (contd)

• Other algorithms will have to be sought out for
  impossible shapes

43
Applications

• Drawing objects into the computer4

44
Applications (contd)

• Industrial robot vision (assembly line)

45
Applications (contd)

• Robotic vision

• And many, many others

46
Summary

• Computers need to break down images into simpler
  segments in order to see them just as humans do
• One early method of labeling segments was the
  Huffman-Clowes labeling scheme soon after
  expanded by Waltz
• Impossible images will not be able to be decoded
  by this method
• There can be a variety of uses for computer
  vision

47
References

• Dewdney, A.K. The New Turing Omnibus. Owl Books,
  New York. 1993 (pages 121-130)
• http//www.soe.ucsc.edu/people/faculty/huffman.htm
  l
• http//www.sgi.com/misc/grafica/huffman/
• http//depts.washington.edu/dmgftp/publications/pd
  fs/acadia98-mdg.pdf
• http//www.cs.bham.ac.uk/research/cogaff/sloman-cl
  owestribute.html