The Role of Sensors in Robotics

1
The Role of Sensors in Robotics
2
Review Why is robotics hard?

• sensors are
• limited
• inaccurate
• noisy
• effectors are
• limited
• crude
• the state (internal and external, but mostly
  external) of the robot is partially-observable,
  at best
• the environment
• often dynamic (changing over time)
• full of potentially-needed information

3
Sensors

• Sensors are one of the key elements as well as
  limitations in robotics.
• Sensors constitute the perceptual system of a
  robot.
• Sensors do not deliver state!
• Sensors are physical devices that measure
  physical quantities, such as
• physical property -gt technology
• contact -gt bump, switch
• distance -gt ultrasound, radar, infra red
• light level -gt photo cells, cameras
• sound level -gt microphones
• strain -gt strain gauges
• rotation -gt encoders

• magnetism -gt compasses
• smell -gt chemical
• temperature -gt thermal, infra red
• inclination -gt inclinometers, gyroscopes
• pressure -gt pressure gauges
• altitude -gt altimeters
• and others...
• Note the same property can be measured with
  different sensors

4
Mobile Robotics Sensors that we used in the past

• magnetism -gt compasses (PSUBOT)
• smell -gt chemical (fire detector)
• temperature -gt thermal, infra red
• inclination -gt inclinometers, gyroscopes
• pressure -gt pressure gauges

• contact -gt bump, switch
• distance -gt ultrasound, sonar, infrared
• light level -gt photo cells, cameras
• sound level -gt microphones
• strain -gt strain gauges
• rotation -gt encoders

5
Simple and Complex Sensors

• Sensors range from simple to complex in the
  amount of information they provide
• a switch is a simple on/off sensor
• a human retina is a complex sensor consisting of
  more than a hundred million photosensitive
  elements (rods and cones)
• Sensors provide raw information, which can be
  treaded in various ways,
• i.e., can can be processed to various levels.
• For example, we can simply react to the sensor
  output
• if the switch is open, stop, if the switch is
  closed, go.
• More complex sensors both require and allows to
  do more complex processing.

6
Simple and Complex Sensors

• We can ask the following question
• "given the sensory reading I am getting, what was
  the world like to make the sensor give me this
  reading."
• This is what is done in computer vision, for
  example, where
• the sensor (a camera lens) provides a great deal
  of information (for example, 512 x 512 pixels
  262,144 pixels of black white, or gray levels,
  or color), and
• we need to compute what those pixels correspond
  to in the real world (i.e., a chair, a phone?).

7
Signals -gt Symbols(States)

• Sensors do not provide state/symbols, just
  signals
• A great deal of computation may be required to
  convert the signal from a sensor into useful
  state for the robot.
• This process bridges the areas of
• electronics,
• signal processing, and
• computation.

Real state
measurement
Model of real State in robot memory
8
Levels of Processing

• Example 1. just to figure out if a switch is open
  or closed, you need to measure voltage going
  through the circuit that's electronics
• Example 2. now suppose you have a microphone and
  you want to recognize a voice and separate it
  from noise that's signal processing
• Example 3. now suppose you have a camera, and you
  want to take the pre-processed image
• (suppose by some miracle somebody has provided
  you with all the edges in the image, so you have
  an "outline" of the objects),
• and now you need to figure out what those
  objects are,
• perhaps by comparing them to a large library of
  drawings
• that's computation

9
Processing sensory data needs brain

• As you can see, sensory data processing is
  challenging and can be computationally intensive
  and time consuming.
• Why does that matter?
• Because it means that your robot needs a brain to
  do this processing.

10
What does the brain have to have to process
sensors

• analog or digital processing capabilities (i.e.,
  a computer)
• wires to connect everything
• support electronics to go with the computer
  batteries
• to provide power for the whole thing
• Thus perception requires
• sensors (power and electronics)
• computation (more power and electronics)
• connectors (to connect it all)

11
What does the brain have to have to process
sensors
Not a good idea to separate and reconstruct

• It is not a good idea to separate
• what the robot senses,
• how it senses it,
• how it processes it, and
• how it uses it.
• If we do that, we end up with a large, bulky, and
  ineffective robot.
• Historically, perception has been treated poorly
  
• perception in isolation
• perception as "king"
• perception as reconstruction.
• Traditionally these approaches came from computer
  vision, which provides the most complex data.

12
The best is Sensor Integration Approach

• Instead, it is best to think about these as a
  single complete design
• the task the robot has to perform
• the best sensors for the task
• the best mechanical design that will allow the
  robot to get the necessary sensory information to
  perform the task
• the body shape of the robot,
• the placement of the sensors,
• etc.

13
New and Better Approaches to Perception

• Perception in the context of action and the task
• Action-oriented perception
• Expectation-based perception uses knowledge about
  the world as constraints on sensor interpretation
  
• Focus-of-attention methods provide constraints on
  where to look
• Perceptual classes partition the world into
  useful categories

14
A New and Better Way
Use ideas from Nature for Perception

• Nature is very clever in the way it solves
  perception/sensing problem
• it evolves special sensors with special geometric
  and mechanical properties.
• Facetted eyes of flies, or
• polarized light sensors of birds have, or
• horizon/line sensors of bugs have, or
• the shape of the ear, etc.
• All biological sensors are examples of clever
  mechanical designs
• These designs maximize the sensor's properties
• it's range
• correctness.

15
Proprioception - internal state

• Origin of received sensory information divides
  perception into
• Proprioception sensing internal state (e.g.,
  muscle tension, limb position)
• Exteroception sensing external state (e.g.,
  vision, audition, smell, etc.)
• Examples of proprioception
• path integration (dead-reconning)
• balancing
• all movements...

16
Affordances

• Affordances are "potentialities for action
  inherent in an object or scene" (Gibson 1979,
  psychology)
• The focus of affordances is the interaction
  between the robot and its environment
• Perception is biased by what needs to be done.
• Robot thinks what is my task?
• I see a chair because I want to sit on it
• I see a chair as something to avoid.
• I see a chair as something to throw at my enemy.

17
Affordances

• As a robot designer, you may not get the chance
  to make up new sensors.
• But you will always have the chance to design
  interesting ways of using the available sensors
  to get the job done.
• It is often not only a chance but a necessity!
• Utilize the interaction with the world
• and always keep in mind the task.
• Food for thought
• how would you detect people in an environment?

18
How to detect people?

• For example, how would you detect people? Some
  options include
• temperature pyroelectric sensors detect special
  temperature ranges
• movement if everything else is static
• shape now you need to do complex vision
  processing
• color if people are unique colored in your
  environment
• Let's think about something even more simple how
  would you measure distance
• ultrasound sensors give you distance directly
  (time of flight)
• infra red through return signal intensity
• two cameras (i.e., stereo) can give you
  distance/depth
• a camera can compute it from perspective
• use a laser and a fixed camera, triangulate
• structured light overlying grid patterns on the
  world
• frequency and phase modulation
• interferometry

19
Sensor Fusion

• Another clever thing to do is to combine multiple
  sensors on a robot to get better information
  about the world.
• This is called sensor fusion.
• Sensor fusion is not simple
• Different sensors give different types, accuracy
  and complexity of information
• processing is necessary to put them together in
  an intelligent and useful way,
• and in real-time.
• The brain processes information from many sensors
  (vision, touch, smell, hearing, sound).
• The processing areas are distinct in the brain
• For vision, they are further subdivided into the
  "what" and "where" pathways.
• Much complex processing is involved in combining
  the information.

20
Information Filters

• Sensory organs act as information filters.
• Extract only part of the total information
  available
• form a representation or physical encoding which
  facilitates the answers to some questions while
  making others impossible to answer
• Simple light sensors function like a set of
  goal-oriented detectors.
• For instance, frog eyes
• are designed to detect motion not interpret
  static images.

21
Vision

• Vision is the process of converting sensory
  information into the knowledge of
• shape,
• identity or
• configuration of objects.
• Other sensors besides light sensors can also
  provide similar information
• bat sonar
• pit viper heat detector
• touch

22
Vision (more)
Characteristics of Vision sensor

• Current processing of sensory data can be greatly
  affected by
• previous input and its interpretation and
• pre-wired processing
• Seeing is the physical recording of the pattern
  of light energy received from the environment.
• It consists of
• selective gathering in of light
• projection or focusing of light on a
  photoreceptive surface
• conversion of light energy into a pattern of
  chemical or electrical activity

23
Costs and Benefits of Sensing

• A cost of sensing of a system in terms of
• 1. energy,
• 2. organizational complexity and
• 3. the possibility of malfunction.
• The nature of useful information is related to
  organisms needs and goals.
• For example, plants only need information on
  light direction.
• Their system compares the light energy received
  on differently oriented surfaces.

24
Receptors in biological organisms

• Sensitivity to environmental influences is a
  general characteristic of living cells.
• In addition to general sensitivity, most animals
  develop a range of specialized receptor cells
• These often form parts of multi cellular sense
  organs.
• Types of senses are called sensory modalities.

Tell the story of bacteria foraging
25
Sensory Modality

• Classifications of sensors
• 1. Exteroceptors - sensitive to external
  influences
• 2. Interoceptors - respond to internal factors
• 3. Proprioceptors - signal movements or positions
  of muscles, joints, etc.
• Classification can be based on the physical
  characteristic of the stimulus concerned, e.g.
• light,
• mechanical,
• chemical.
• Phasic receptors respond to changes in the
  environment.
• Tonic receptors relate to the absolute level of
  stimulation.
• Some receptors are a combination of phasic and
  tonic.
• Sensitivity to one modality can be exploited to
  provide information about another.

26
Sensory Modality
Sensory Modality

• Classifications (more)
• Receptors sensitive to gravity are called
  statocysts.
• These receptors function by using sensory cilia
  in a vesicle which contains one or more dense
  bodies to sense the position of these bodies.
• These organs can also sense acceleration.
• Note
• insects lack these specialized organs,
• instead, they depend on the information from many
  sense organs associated with their joints to
  provide relevant information.

27
Specialist and Generalist Receptors

• 1. Receptors which are specialists respond only
  to a restricted range of whatever they are
  sensing.
• For example, olfactory specialists have a
  restricted spectrum of response to odors
• with an acute sensitivity to only a single
  compound such as a pheromone.
• 2. Generalist receptors respond to a wide variety
  of stimuli within the modality
• but each generalist has its own pattern of
  sensitivity
• so a substance can be recognized by the unique
  combination of receptors activated.

28
Intensity Coding in biological sensors

• Information from sensors is usually not just ON
  or OFF, but also includes how much''.
• The range of stimulation intensity to which an
  organism is sensitive is often a controllable
  factor.
• Also different cells can operate across different
  parts of a wide range.

29
Sensory Processing Example

• In the locust, simple light sensing organs on the
  top of the head produce a poorly focused image.
• A massive amount of receptor information (about
  1000 receptors) in each organ is funneled through
  a small number of second-order neurons (25).
• During flight, the ocelli provide a rapid,
  overall assessment of the position of the horizon.

30
Another Example

• When a male hoverfly has a possible mate in its
  field of vision, it sets a course to intercept.
• To plot a course, it needs distance, velocity and
  course information of target
• probably not determined from observation.
• The fly assumes'' that the object in the visual
  field is
• 1.the size of one of its own kind
• 2.travelling at approximately the same velocity
• The size assumption leads to a determination of
  distance.
• The direction and speed at which the object moves
  across the visual field indicate then its course
  and the intercept can begin!

31
Convergence

• Convergence occurs when multiple sources of
  information are compressed into a much smaller
  domain.
• A sensory field is an array of receptors which
  provide sensory input to a cell or centre in a
  nervous pathway.

32
Divergence

• Divergence is the conveying of information from a
  single receptor cell, or group of cells, into the
  nervous system via multiple or parallel pathways.
• These pathways can be used to extract and
  segregate different types of information.
• Divergence also covers the concept of a system
  responding to a single sensory modality, but
  providing out to different centers and thus
  influencing different types of behavior.

33
Labeled Lines

• This principle works on the premise that similiar
  signals from different receptors are handled as
  if they were labeled'' by their origin.
• An example is the escape response of the
  cockroach.
• The lunging attack of a toad creates a current of
  air which is detected by sensory hairs on the
  anal cerci of the insect,
• The hairs are arranged in a number of columns
  which are sensitive to wind from different
  directions.
• The different columns form distinct combinations
  of connections with processing neurons so that
  the insect is aware of the location of the
  threat.
• The combinations of sensory input trigger
  appropriate movements.

34
The Photopine

• Sensors distributed over vehicle body
• As the sensor is touched, the reflex response is
  immediate and it determines the area of contact.

35
Sources

• A. Ferworn
• Maja Mataric
• Fred Martin

36
Questions for students

1. List all robotics sensors that you are familiar
   with.
2. List some inexpensive sensors that one can
   purchase in Radio Shack or Home Depot and how
   they can be used in robotics.
3. What is the concept of sensor fusion. Write a
   simple program to fuse 2 types of sensors.
4. What is proprioception and exteroception?
5. What are affordances, give examples for our robot
   theatre. Why affordances are so important.
6. Discuss simple vision system for a humanoid
   robot, what can it do?
7. Give examples of complex and simple sensors and
   their role.
8. Give examples of perception system of some simple
   animal and how this system can be modeled in our
   robots, both stationary and mobile robots. What
   kind of experiments would you implement with this
   system?
9. Give examples of divergence and convergence of
   sensors.
10. How would you implement a vision system for a
    simple mobile robot that would act according to
    three principles (1) if a moving item is small,
    it is a potential food, attack it. (2) if a
    moving item is large, it is a potential enemy,
    escape. (3) if a moving target is your size, it
    is a potential mating partner, mate.