Intelligent Robotics

1
Intelligent Robotics

• Today Vision
• Time Space Complexity

2
Color Perception in Human

• Two kinds of light receptors rods and cones
• Rods register the relative light and dark
• Cones perceive color
• Three kinds of cones red, green, blue

3
Computer Vision

• Uses the electromagnetic spectrum to produce an
  image.
• Visible light, x-rays, thermal, infrared

4
Representation

• Image picture like format where there is a
  direct physical correspondence to the scene being
  viewed.
• There are multiple readings in a grid
• Pixels picture element
• Measure depends on the type of spectra being used
• Image function converts a signal to a pixel value

5
CCD Cameras

• Charged-Couple Device detects visible light
• Light fall on an array of metal-oxide
  semiconductor capacitor (MOS)
• Line transfer or frame transfer
• Frame grabber
• A/D conversion
• A/D conversion
• Slower than CCD
• Frame buffers

6
Representations

• Grayscale
• 8-bit
• 256 discrete gray values
• 0 black, 255 white

7
Representations

• Different method for representing color
• RGB space red, green, blue
• HSV space hue, saturation, and value

8
RGB

• Color Space
• 24 bit color (8 bits per color)

9
RGB Representations

• Interleaved
• RGB values stored together
• Red imagerowcol0
• Green imagerowcol1
• Blue imagerowcol2
• Separate
• RGB values stored as separate 2D arrays
• Red image_redrowcol
• Green image_greenrowcol
• Blue image_Bluerowcol

10
Region Segmentation
11
Region Segmentation

• Identifying a region in an image with a
  particular color
• Thresholding
• Binary image

12
HSV Representations

• HSV space hue, saturation, and value
• Hue is the dominant wavelength
• Saturation is the lack of whiteness
• Value is intensity or brightness the light
• Linear transformation between RGB and HSV

13
HSV Color Space
14
Comparison of Region Segmentation
RGB segmentation
Original Image
HSV Segmentation
15
IC Color Vision System

• Color Space
• Finding color blobs in an image
• Making a color model
• Tuning the camera

16
Color Selection Plane
Sat0 Val224
Sat224 Val0
Sat224 Val224
Hue0
Note 224 is the range of values the camera
pixels put out in each of R, G B
Hue360
17
Color Blobs

• For color tracking, a rectangular piece of the
  color selection plane is selected. All of the
  pixels in the image whose color falls within that
  piece are selected.
• Selected pixels that are contiguous are combined
  as blobs
• Each blob has a size, position, number of pixels,
  major and minor axis, etc.
• The blobs correspond to objects seen in the image
  that are the desired color (as given by the
  specified piece of the of the color selection
  plane).

18
Color Models

• The XBC can segment the image using three
  different pieces of the color selection plane
  (each is called a color model) simultaneously
• It can track a number of blobs from each color
  model
• It can display the video in any one of three ways
• Raw (live video)
• Processed (pixels not selected by a color model
  are dark)
• Combined (processed combined with raw show you
  can see where the blobs are on the live video)

19
More on Color Models

• A Color Model-HSV specifies a bounding box in the
  color selection plane
• Moving either edge towards the center line
  constrains the range of accepted color values to
  only include more vivid colors.
• If everything you want is being accepted but so
  is a lot of other junk you don't want, move the
  corners closer to the center.

20
Onscreen Instructions

• for setting color models

21
Onscreen Instructionsfor setting color models
22
Move Mode
Symmetric Move or Resize mode (M or R) is
selected by the START button
Mode Designator
Corner move mode is selected by the L or R button
and is indicated by a highlighted corner
Press the SELECT button to view the onscreen
instructions
23
Processed Video
The A button cycles through live, processed, and
combined video
24
XBC Camera Check

• Create a color model for channel 0 that sees a
  red screw driver
• Load xbctest.ic onto your XBC
• Run the program
• Select the vision test
• Select the correct channel/model to see red
• Follow onscreen directions to get data on the
  blobs
• If you like your model, save it to flash
• For more info, see XBC Camera in IC Help

25
Which Color Model?

• The XBC stores 3 channels of color (0, 1, 2)
• There are 3 models maintained for each channel
• The currently active model (the one you see in
  the modify model menu)
• The model in flash (you can save the currently
  active model and camera settings using the save
  to flash menu item or make them active by loading
  from flash)
• The default model and camera settings which are
  made active by selecting restore to default

26
Tuning the Camera

• By default cameras turn on Auto White Balance
  (AWB) and dynamically adjust their color
  temperature to keep the amounts of red and blue
  in the field of view roughly balanced.  This is
  fine if the content of the field of view is
  roughly balanced between red and blue and if the
  goal is to look good to humans in changing,
  arbitrary lighting situations.  However, when
  trying to do color tracking, dynamically changing
  color temperature is generally counter
  productive. 
• Instead, you should use the Vision/Camera Config
  menu and press the Start button to interactively
  calibrate the white balance while pointing the
  camera at a white sheet of paper.  This will
  adjust the Red and Blue components of the color
  temperature until the amount of each in the scene
  balances, then turn off AWB to lock those values
  in.  After this procedure, it's a good idea to go
  to the Vision/Flash Memory menu, select "Setting
  lt Camera Config gt", and then "Save to Flash".

27
Tuning the Camera

• Focus the un-focus the lens
• Depending on what you are looking for it can help
  to have the image a little out of focus