Overview of Our Sensors For Robotics - PowerPoint PPT Presentation

About This Presentation
Title:

Overview of Our Sensors For Robotics

Description:

Overview of Our Sensors For Robotics Machine vision Computer vision To recover useful information about a scene from its 2-D projections. To take images as inputs and ... – PowerPoint PPT presentation

Number of Views:526
Avg rating:3.0/5.0
Slides: 83
Provided by: Mar151
Learn more at: http://web.cecs.pdx.edu
Category:

less

Transcript and Presenter's Notes

Title: Overview of Our Sensors For Robotics


1
Overview of Our Sensors For Robotics
2
Machine vision
  • Computer vision
  • To recover useful information about a scene from
    its 2-D projections.
  • To take images as inputs and produce other types
    of outputs (object shape, object contour, etc.)
  • Geometry Measurement Interpretation
  • To create a model of the real world from images.

3
Topics
  • Computer vision system
  • Image enhancement
  • Image analysis
  • Pattern Classification

4
Related fields
  • Image processing
  • Transformation of images into other images
  • Image compression, image enhancement
  • Useful in early stages of a machine vision system
  • Computer graphics
  • Pattern recognition
  • Artificial intelligence
  • Psychophysics

5
Vision system hardware
6
  • Image Processing System

7
(No Transcript)
8
Image Representation
9
Image
  • Image a two-dimensional array of pixels
  • The indices i, j of pixels integer values
    that specify the rows and columns in pixel values

10
(No Transcript)
11
Sampling, pixeling and quantization
  • Sampling
  • The real image is sampled at a finite number of
    points.
  • Sampling rate image resolution
  • how many pixels the digital image will have
  • e.g.) 640 x 480, 320 x 240, etc.
  • Pixel
  • Each image sample
  • At the sample point, an integer value of the
    image intensity

12
  • Quantization
  • Each sample is represented with the finite word
    size of the computer.
  • How many intensity levels can be used to
    represent the intensity value at each sample
    point.
  • e.g.) 28 256, 25 32, etc.

13
Color models
  • Color models for images,
  • RGB, CMY
  • Color models for video,
  • YIQ, YUV (YCbCr)
  • Relationship between color models

14
(No Transcript)
15
6.7. Digital Cameras
16
Digital Cameras
  • Technology
  • CCD (charge coupled devices)
  • CMOS (complementary metal oxide semiconductor)
  • Resolution
  • 60x80 black/white up to
  • several Mega-Pixels in 32bit color
  • However Embedded system has to have computing
    power to deal with this large amount of data!

17
Vision (camera framegrabber)
18
Digital Cameras
  • Performance of embedded system 10 - 50 of
    standard PC

19
Interfacing Digital Cameras to CPU
  • Interfacing to CPU
  • Completely depends on sensor chip specs
  • Many sensors provide several different
    interfacing protocols
  • versatile in hardware design
  • software gets very complicated
  • Typically 8 bit parallel (or 4, 16, serial)
  • Numerous control signals required

20
Interfacing Digital Cameras to CPU
  • Digital camera sensors are very complex units.
  • In many respects they are themselves similar to
    an embedded controller chip.
  • Some sensors buffer camera data and allow slow
    reading via handshake (ideal for slow
    microprocessors)
  • Most sensors send full image as a stream after
    start signal
  • (CPU must be fast enough to read or use hardware
    buffer or DMA)
  • We will not go into further details in this
    course. However, we consider camera access
    routines

21
Simplified diagram of camera to CPU interface
22
Problem with Digital Cameras
  • Problem
  • Every pixel from the camera causes an interrupt
  • Interrupt service routines take long, since
    they need to store register contents on the stack
  • Everything is slowed down
  • Solution
  • Use RAM buffer for image and read full image
    with single interrupt

23
  • Idea
  • Use FIFO as image data buffer
  • FIFO is similar to dual-ported RAM, it is
    required since there is no synchronization
    between camera and CPU
  • When FIFO is half full, interrupt is generated
  • Interrupt service routine then reads FIFO until
    empty
  • (Assume delay is small enough to avoid FIFO
    overrun)

24
(No Transcript)
25
(No Transcript)
26
Bayer Pattern
27
De-Mosaic
28
(No Transcript)
29
Conversion in Digital Cameras
  • Bayer Pattern
  • Output format of most digital cameras
  • Note
  • 2x2 pattern is not spatially located in a
    single point!
  • Can be simply converted to RGB (drop one green
    byte)
  • 160x120 Bayer ? 80x60 RGB
  • Can be better converted using demosaicing
    technique
  • 160x120 Bayer ? 160x120 RGB

30
(No Transcript)
31
  • CMUCAM2 CAMERA www.seattlerobotics.com
  • The camera can track user defined color blobs at
    up to 50 fps (frames per second)
  • Track motion using frame differencing at 26 fps
  • Find the centroid of any tracking data
  • Gather mean color and variance data
  • Gather a 28 bin histogram of each color channel
  • Manipulate horizontal pixel differenced images
  • Arbitrary image windowing
  • Adjust the cameras image properties

This camera can do a lot of processing
32
This camera can do a lot of processing
  • Dump a raw image
  • Up to 160 X 255 resolution
  • Support multiple baud rates
  • Control 5 servos outputs
  • Slave parallel image processing mode off of
    single camera bus
  • Automatically use servos to do two axis color
    tracking
  • B/W analog video output (Pal or NTSC)
  • Flexible output packet customization
  • Multiple pass image processing on a buffered image

33
(No Transcript)
34
Vision Guided Robotics
  • and Applications in Industry and Medicine

35
Contents
  • Robotics in General
  • Industrial Robotics
  • Medical Robotics
  • What can Computer Vision do for Robotics?
  • Vision Sensors
  • Issues / Problems
  • Visual Servoing
  • Application Examples
  • Summary

36
Industrial Robot vs Human
  • Robot Advantages
  • Strength
  • Accuracy
  • Speed
  • Does not tire
  • Does repetitive tasks
  • Can Measure
  • Human advantages
  • Intelligence
  • Flexibility
  • Adaptability
  • Skill
  • Can Learn
  • Can Estimate

Robot needs vision
37
Industrial Robot
  • Requirements
  • Accuracy
  • Tool Quality
  • Robustness
  • Strength
  • Speed
  • Price Production Cost
  • Maintenance

Production Quality
38
Medical (Surgical) Robot
  • Requirements
  • Safety
  • Accuracy
  • Reliability
  • Tool Quality
  • Price
  • Maintenance
  • Man-Machine Interface

39
What can Computer Vision do for (industrial and
medical) Robotics?
  • Accurate Robot-Object Positioning
  • Keeping Relative Position under Movement
  • Visualization / Teaching / Telerobotics
  • Performing measurements
  • Object Recognition
  • Registration

Visual Servoing
40
Vision Sensors
  • Single Perspective Camera
  • Multiple Perspective Cameras (e.g. Stereo Camera
    Pair)
  • Laser Scanner
  • Omnidirectional Camera
  • Structured Light Sensor

41
Vision Sensors
  • Single Perspective Camera

Single projection
42
Vision Sensors
  • Multiple Perspective Cameras (e.g. Stereo Camera
    Pair)

43
Vision Sensors
  • Multiple Perspective Cameras (e.g. Stereo Camera
    Pair)

44
Vision Sensors
  • Multiple Perspective Cameras (e.g. Stereo Camera
    Pair)

45
Vision Sensors
  • Laser Scanner

46
Vision Sensors
  • Laser Scanner

47
Vision Sensors
  • Omnidirectional Camera

48
Vision Sensors
  • Omnidirectional Camera

49
Vision Sensors
  • Structured Light Sensor

Figures from PRIP, TU Vienna
50
Issues/Problems of Vision Guided Robotics
  • Measurement Frequency
  • Measurement Uncertainty
  • Occlusion, Camera Positioning
  • Sensor dimensions

51
Visual Servoing
  • Vision System operates in a closed control loop.
  • Better Accuracy than Look and Move systems

Figures from S.Hutchinson A Tutorial on Visual
Servo Control
52
Visual Servoing
  • Example Maintaining relative Object Position

Figures from P. Wunsch and G. Hirzinger.
Real-Time Visual Tracking of 3-D Objects with
Dynamic Handling of Occlusion
53
Camera Configurations for Visual Servoing
End-Effector Mounted
Fixed
Figures from S.Hutchinson A Tutorial on Visual
Servo Control
54
Visual Servoing Architectures
Figures from S.Hutchinson A Tutorial on Visual
Servo Control
55
Position-based vs Image Based control in Visual
Servoing
  • Position based
  • Alignment in target coordinate system
  • The 3D structure of the target is rconstructed
  • The end-effector is tracked
  • Sensitive to calibration errors
  • Sensitive to reconstruction errors
  • Image based
  • Alignment in image coordinates
  • No explicit reconstruction necessary
  • Insensitive to calibration errors
  • Only special problems solvable
  • Depends on initial pose
  • Depends on selected features

End-effector
target
Image of end effector
Image of target
56
EOL and ECL control in Visual Servoing
  • EOL endpoint open-loop only the target is
    observed by the camera
  • ECL endpoint closed-loop target as well as
    end-effector are observed by the camera

EOL
ECL
57
Visual Servoing
  • Position Based Algorithm
  • Estimation of relative pose
  • Computation of error between current pose and
    target pose
  • Movement of robot
  • Example point alignment

p1
p2
58
Visual Servoing
  • Position based point alignment
  • Goal bring e to 0 by moving p1
  • e p2m p1m
  • u k(p2m p1m)
  • pxm is subject to the following measurement
    errors sensor position, sensor calibration,
    sensor measurement error
  • pxm is independent of the following errors end
    effector position, target position

59
Visual Servoing
  • Image based point alignment
  • Goal bring e to 0 by moving p1
  • e u1m v1m u2m v2m
  • uxm, vxm is subject only to sensor measurement
    error
  • uxm, vxm is independent of the following
    measurement errors sensor position, end effector
    position, sensor calibration, target position

p1
p2
u1
v1
v2
u2
d1
d2
c1
c2
60
Visual Servoing
  • Example Laparoscopy

Figures from A.Krupa Autonomous 3-D Positioning
of Surgical Instruments in Robotized Laparoscopic
Surgery Using Visual Servoing
61
Visual Servoing
  • Example Laparoscopy

Figures from A.Krupa Autonomous 3-D Positioning
of Surgical Instruments in Robotized Laparoscopic
Surgery Using Visual Servoing
62
Registration
  • Registration of CAD models to scene features

Figures from P.Wunsch Registration of CAD-Models
to Images by Iterative Inverse Perspective
Matching
63
Registration
  • Registration of CAD models to scene features

Figures from P.Wunsch Registration of CAD-Models
to Images by Iterative Inverse Perspective
Matching
64
Summary on tracking and servoing
  • Computer Vision provides accurate and versatile
    measurements for robotic manipulators
  • With current general purpose hardware, depth and
    pose measurements can be performed in real time
  • In industrial robotics, vision systems are
    deployed in a fully automated way.
  • In medicine, computer vision can make more
    intelligent surgical assistants possible.

65
Omnidirectional Vision Systems
  • CABOTO Robots task
  • Building a topological map of an unknown
    environment
  • Sensor
  • Omnidirectional vision system
  • Works aim
  • Prove effectiveness of omnidirectional sensors
    for Spatial Semantic Hierarchy SSH

66
Spatial Semantic Hierarchy...
  • ... A model of the human knowledge of large
    spaces
  • Layers
  • Sensory Level
  • Control Level
  • Causal Level
  • Topological Level
  • Metrical Level

Interface with the robots sensory system
Control Laws, Transition of State,
Distinctiveness Measure
Minimal set of Places, Paths and Regions
View, Action, Distinct Place Abstracts Discrete
from Continous
Distance, Direction, Shape Useful, but seldom
essential
67
Tracking
  • Instrument tracking in laparoscopy

Figures from Wei A Real-time Visual Servoing
System for Laparoscopic Surgery
68
Omnidirectional Camera
  • Composed of
  • Standard Color Camera
  • Convex Mirror
  • Perspex Cylinder

69
Pros e Cons
  • Advantages
  • Wide vision field
  • High speed
  • Vertical Lines
  • Rotational Invariance
  • Disadvantages
  • Low Resolution
  • Distortions
  • Low readability

70
Omnidirectional Vision and SSH
  • View Omnidirectional image
  • Exploring around the block
  • Robot should discriminate between turns and
    travels
  • We need an Effective Distinctiveness measure

71
Assumptions for vision system
  • Man-made environment
  • Floor flat and horizontal
  • Wall and objects surfaces are vertical
  • Static objects
  • Constant Lighting
  • Robot translates or rotates
  • No encoders

72
Features and Events
  • Feature
  • Vertical Edges
  • Events
  • A new edge
  • An edge disappears
  • Two edges 180 apart
  • Two pairs of edges 180 apart

73
Experiments
  • Tasks of Caboto robot
  • Navigation
  • Map building
  • Techniques
  • Edge detection
  • Colour marking

74
Cabotos Images
75
(No Transcript)
76
Results
  • Correct tracking of edges
  • Recognition of actions
  • Calculation of the turn angle

The path segmentation
77
Mirror Design
Mirror shape should depend on robot task!
  • Design custom mirror profile
  • Maximise resolution in ROIs

Mirror Profile
78
The new mirror
79
Conclusion on Omnivision camera
  • Omnidirectional vision sensor is a good sensor
    for map building with SSH
  • Motion of the robot was estimated without active
    vision
  • The use of a mirror designed for this application
    will improve the system

80
Omnidirectional Cameras
  • Compound-eye camera
  • (from Univ. of Maryland, College Park. )
  • Panoramic cameras (from Apple)
  • Omnidirectional cameras
  • (from University of Picardie - France)

81
Student info.
  • of lab marks can be deducted if rules and
    regulation are not followed
  • ex by not cleaning up your bench or sliding
    your chairs back
  • underneath bench top.
  • For more technical information on boards, devices
    and sensors check out my web page at
    www.site.uottawa.ca/alan
  • Students are responsible for their own extra
    parts ex if you want to add a sensor or device
    that the dept. doesnt have you are responsible
    for the purchase and delivery of that part, on
    rare occasion did the school purchase those
    parts.
  • Back packs off bench tops
  • TAs will have student based on station
  • Important issue regarding the design of a new
    project is to do a current analysis before the
    start of your design
  • Setup a leader among your team so that you are
    better organized
  • Do not wait, before starting your project start
    now !
  • Prepare yourself before coming to the lab
  • It doesnt work ! Ask yourself is it software or
    hardware, use the scope to trouble shoot
  • Fuses keeps on blowing, stop and do some
    investigation.
  • Do not cut any servo, battery and other device
    wire connectors. If you must please come and see
    me
  • No design must exceed 50 volts, ex do not work
    with 120 volts AC
  • I can give you what I have regarding metal, wood
    and plastic recycled pieces and do some cuts or
    holes with my band saw and drill press for you,
  • PLEASE DO NOT ask me to barrow my tools. If you
    need to do a task with a special tool that I have
    then I shall do it for you.

82
Problems for students
  1. Hardware and software components of a vision
    system for a mobile robot
  2. Image representation for intelligent processing
  3. Sampling, pixeling and Quantization
  4. Color models
  5. Types of digital cameras.
  6. Interfacing digital cameras to CPU.
  7. Problems with cameras.
  8. Bayer Patterns and conversion.
  9. What is good about CMUCAM?
  10. Use of vision in industrial robots.
  11. Use of multiple-perspective cameras.
  12. Use of omnivision cameras.
  13. Types of visual servoing.
  14. Applications of visual servoing
  15. Visual servoing in surgery
  16. Explain tracking applications of vision.

83
References
  • Photos ,Text and Schematics Information
  • www.acroname.com
  • www.lynxmotion.com
  • www.drrobot.com
  • Alan Stewart
  • Dr. Gaurav Sukhatme
  • Thomas Braunl
  • Students 2002, class 479
  • E. Menegatti, M. Wright, E. Pagello
Write a Comment
User Comments (0)
About PowerShow.com