A Lightweight ComputerVisionbased Electronic Travel Aid

1
A Lightweight Computer-Vision-based Electronic
Travel Aid

• Andrew B. Raij (raij_at_cs.unc.edu)
• Enabling Tech Project Final Report
• 4/17/2003

Department of Computer Science, UNC-Chapel Hill
2
Long-term Goals

• Design and build a wearable, inconspicuous device
  that assists people with visual disabilities with
  orientation and mobility (OM)
• Gathers info from the environment with one or
  more cameras and presents the data to the user in
  a useful form
• Focus on gathering depth info this semester
• Explore useful ways to present depth to user

3
Previous ETAs (1)

• Cane
• A depth and acoustics probe for nearby things
• No long distance probing
• No probing up high
• Requires scanning back and forth across world

4
Previous ETAs (2)

• NavBelt
• Spatial audio
• possibly interferes with users auditory feedback
  from the world
• Uses sonars to build a local, 2D, birds-eye map
• Vertical resolution of depth is poor
• Modes
• Guidance Mode guides users around obstacles,
  but requires dead-reckoning
• Image Mode scans across map, increases pitch
  and volume as depth decreases. Too much info,
  requires lots of training

5
Previous ETAs (3)

• GuideCane
• Cane attached to robot that turns at obstacles
  but maintains users desired course
• Guidance tech based on NavBelt Guidance Mode
• Very intuitive, natural force feedback
• Restricted to relatively flat surfaces
• How does the user feel about being led by a
  robot? Control issues?
• Concerns about appearing different?

6
Previous ETAs (4)

• vOICe (http//www.seeingwithsound.com)
• Scans across images (like the NavBelt image mode)
  and translates images to different sound cues.
• Uses images - Inherently good vertical and
  horizontal resolution, scans world for user
• Auditory feedback could be lots of information to
  deal with at once
• Can require lots of time practicing to get used
  to it
• Some users claim it almost makes them feel like
  they are seeing
• Can also work with depth images

7
Depth Extraction

• We want to determine X given the two views
• First find the fundamental matrix F ?
  xFx 0
• Use the fundamental matrix to find the projection
  matrices P and P for C and C

• Use least-squares linear triangulation method to
  calculate X fast
• This method is more prone to errors from noise

8
Estimating F

• Estimating the fundamental matrix F
• 7 or more corresponding points
• Insert known points in scene
• Structured light
• Checkerboard
• robust searching for corresponding points in 2
  views
• possible, but not at every frame
• Need a scene with good features
• if geometric relationship between views is
  constant, so is F, P, P
• Only need to estimate F, P, P once, or maybe
  just every once in a while

9
Depth Sampling

• At every frame we need to
• choose points to sample for depth
• All
• can be fast if epipolar lines are parallel to
  scanlines
• Edges and surrounding areas?
• Object extraction?
• Correspond them in 2 views
• Points that do not correspond well in both images
  are not processed further

10
Spatial Perception (Ungar)

• Near-Space (Haptic) vs. Far-Space (Locomotor)
• Both spaces directly connected through vision
• Early and congenitally blind generally code the
  world in terms of their own bodies, i.e.
  near-space
• Spatial relationships are understood but spatial
  inference, rotation can be difficult
• Recent research suggests people who are blind
  tend to not learn/use strategies for
  understanding locomotor space and its connection
  to haptic space
• Hypothesis tactile maps can bridge the gap
  between locomotor and haptic space

11
Tactile Display

• Convey information through touch
• Two kinds of tactile displays I read about
• Vibration
• Thermal
• Others
• Pin Arrays
• Electric

12
Vibration

• Vibration for phones, PDAs, other small form
  factor devices
• Phone
• Ambient Touch
• Tactile Virtual Buttons
• Gloves, vests with piezo-electric buzzers
• Lots of others

13
Thermal

• Not enough work here
• Mostly for VE research
• blowing hot air on skin
• Thermal Phantom, Gloves
• Peltier cell, water-cooled heat sink
• Temperature Perception
• Spatial resolution is poor
• Interesting spatial summation properties
• Slow temp changes not very noticeable but fast
  changes are
• Body adapts to temperature change over time
• Unlikely to perceive clearly several areas on
  body changing temp
• Safety issues!