Visual Object Recognition

1
Visual Object Recognition

• Bastian Leibe
• Computer Vision Laboratory
• ETH Zurich
• Chicago, 14.07.2008

Kristen Grauman Department of Computer
Sciences University of Texas in Austin
2
Identification vs. Categorization
?
???
3
Object Categorization

• How to recognize ANY car
• How to recognize ANY cow

4
What could be done with recognition algorithms?
There is a wide range of applications, including
Navigation, driver safety
Autonomous robots
Situated search
Content-based retrieval and analysis for images
and videos
Medical image analysis
5
Object Categorization

• Task Description
• Given a small number of training images of a
  category, recognize a-priori unknown instances of
  that category and assign the correct category
  label.
• Which categories are feasible visually?
• Extensively studied in Cognitive Psychology,e.g.
  Brown58

6
Visual Object Categories

• Basic Level Categories in human categorization
  Rosch 76, Lakoff 87
• The highest level at which category members have
  similar perceived shape
• The highest level at which a single mental image
  reflects the entire category
• The level at which human subjects are usually
  fastest at identifying category members
• The first level named and understood by children
• The highest level at which a person uses similar
  motor actions for interaction with category
  members

7
Visual Object Categories

• Basic-level categories in humans seem to be
  defined predominantly visually.
• There is evidence that humans (usually) start
  with basic-level categorization before doing
  identification.
• ? Basic-level categorization is easier and
  faster for humans than object identification!
• ? Most promising starting point for visual
  classification

Abstract levels



Basic level
Individual level


8
Other Types of Categories

• Functional Categories
• e.g. chairs something you can sit on

9
Other Types of Categories

• Ad-hoc categories
• e.g. something you can find in an office
  environment

10
Levels of Object Categorization

• Different levels of recognition
• Which object class is in the image? ? Obj/Img
  classification
• Where is it in the image? ? Detection/Localizatio
  n
• Where exactly ? which pixels? ? Figure/Ground
  segmentation

cow
car
motorbike
11
Challenges robustness
Illumination
Object pose
Clutter
Occlusions
Viewpoint
K. Grauman, B. Leibe
12
Challenges robustness

• Detection in Crowded Scenes
• Learn object variability
• Changes in appearance, scale, and articulation
• Compensate for clutter, overlap, and occlusion

13
Challenges context and human experience
K. Grauman, B. Leibe
14
Challenges context and human experience
Context cues
Dynamics
Video credit J. Davis
Image credit D. Hoeim
15
Challenges scale, efficiency

• Thousands to millions of pixels in an image
• Estimated 30 Gigapixels of image/video content
  generated per second
• About half of the cerebral cortex in primates is
  devoted to processing visual information
  Felleman and van Essen 1991
• 3,000-30,000 human recognizable object categories
• 30 degrees of freedom in the pose of articulated
  objects (humans)
• Billions of images indexed by Google Image Search
• 18 billion prints produced from digital camera
  images in 2004
• 295.5 million camera phones sold in 2005

K. Grauman, B. Leibe
16
Challenges learning with minimal supervision
More
Less
Unlabeled, multiple objects
Cropped to object, parts and classes labeled
Classes labeled, some clutter
K. Grauman, B. Leibe
17
Rough evolution of focus in recognition research
1980s
K. Grauman, B. Leibe
18
This tutorial

• Intended for broad AAAI audience
• Assuming basic familiarity with machine learning,
  linear algebra, probability
• Not assuming significant vision background
• Our goals
• Describe main approaches to recognition
• Highlight past successes and future challenges
• Provide the pointers (to literature and tools)
  that would allow you to take advantage of
  existing techniques in your research
• Questions welcome

19
Outline

• Detection with Global Appearance Sliding
  Windows
• Local Invariant Features Detection Description
• Specific Object Recognition with Local Features
• ? Coffee Break ?
• Visual Words Indexing, Bags of Words
  Categorization
• Matching Local Features
• Part-Based Models for Categorization
• Current Challenges and Research Directions

19
K. Grauman, B. Leibe