Tracking

1
Tracking

• Sherman Craig, pp. 75-94.
• Welch, Greg and Eric Foxlin (2002). Motion
  Tracking No Silver Bullet, but a Respectable
  Arsenal, IEEE Computer Graphics and
  Applications, special issue on Tracking,
  November/December 2002, 22(6) 2438..
  (http//www.cs.unc.edu/tracker/media/pdf/cga02_we
  lch_tracking.pdf)

2
Motivation

• Methods to interact with the virtual world
• More natural
• Higher level of immersion
• Task performance
• Control navigation
• Control interaction
• Ex. Training soldiers w/ a gun
• How do we track the gun?
• How do we determine what the user sees?
• This requires
• Signaling (button presses, etc.)
• Location. lt- this is tracking!

3
Tracking

• http//www.sv.vt.edu/future/vt-cave/apps/detour
• Pose
• Position
• Orientation
• What do we want to track?
• Head pose
• Hand pose
• Other body part
• Other objects (e.g. spider)
• So what does it mean if a tracking system reports
  your head at 2.5,3.3, 1.9?

4
Common Tracking Methods

• GPS
• Wii

http//www.directionsmag.com/images/articles/GPS_a
rticles/realtime_diff_GPS.jpg
5
Basic Idea
Trackers provide location and/or position
information relative to some coordinate
system. (x,y,z) (rx,ry,rz)
(0,0,0) Receiver coordinate system
(0,0,0) Origin for tracker coordinate system
6
Degrees of freedom

• The amount of pose information returned by the
  tracker
• Position (3 degrees)
• Orientation (3 degrees)
• There are trackers that can do
• only position
• only orientation
• both position and orientation

7
Question

• Given that I want to track your head, I attach a
  new tracker from NewTracker Corp. it returns 6
  degrees of freedom (6 floats). What questions
  should you have?
• What are some evaluation points for a tracking
  system?

8
Evaluation Criteria

• Data returned
• Spatial distortion (accuracy)
• Resolution
• Jitter (precision)
• Drift
• Lag
• Update Rate
• Range

• Interference and noise
• Mass, Inertia and Encumbrance
• Number of Tracked Points Durability
• Wireless
• Price

Which of these are most important?
9
Performance Measures

• Registration (Accuracy)
• Difference between an objects pose and the
  reported pose
• Location
• Orientation
• What are determining factors?
• Resolution
• Granularity that the tracking system can
  distinguish individual points or orientations
• What are determining factors?
• Jitter
• Change in reported position of a stationary
  object
• What are determining factors?
• Drift
• Steady increase in error with time
• What are determining factors?

10
Performance Measures

• t0 time when sensor is at point p
• t1 time when sensor reports p
• Lag or Latency t1 - t0
• What makes up latency?
• Acquisition
• Transmission
• Filtering

11
Performance Measures

• t0 time when sensor is at point p
• t1 time when sensor reports p
• Lag or Latency t1 - t0
• What makes up latency?
• Acquisition
• Transmission
• Filtering
• What is a minimum?

12
Update Rate

• Number of tracker position/orientation samples
  per second
• High update rate ! accuracy
• Poor use of update information may result in more
  inaccuracy
• Communication pathways and data packet size are
  important

13
Range

• Working volume
• What is the shape?
• Accuracy decreases with distance
• Range is inversely related to accuracy
• Position and orientation range could be different
• Sensitivity not uniform across all axis

14
Interference and Noise

• Interference - external phenomenon that degrades
  systems performance
• Each type of tracker has different causes of
  interference/noise
• Occlusion
• Metal
• Noise
• Environmental (e.g. door slamming, air
  conditioner)

15
Mass, Inertia and Encumbrance

• Do you really want to wear this?
• Inertia
• Tethered

16
Multiple Tracked Points

• Number of potentially tracked points
• Unique
• Simultaneous
• Difficulties
• Interference between the sensors
• Multiplexing
• Time Multiplexing Update rate of S samples per
  second and N sensors results in S/N samples per
  sensor per second
• Frequency Multiplexing Each sensor broadcasts
  on a different frequency. More

17
Price

• You get what you pay for. (30-100k)
• Rich people are a small market.

18
Tracking Technologies

• Different Tracking Technologies
• Goals
• Understand how they work
• Understand tradeoffs
• Know when to use which
• Future directions

19
Mechanical Linkage

• Rigid jointed structure
• One end (base) is fixed
• The other (distal) is free
• Distal is user controlled to an arbitrary
  position and orientation.
• Sensors at the joints detect the angle
• Concatenate translates and rotates
• Determine the position and orientation of the
  distal relative to the base.

20
Mechanical Tracking

• Data returned 6 DOF
• Spatial distortion 0.3381 mm
• Resolution very high
• Jitter (precision) very low
• Drift - none
• Lag gt5ms
• Update Rate - 300 Hz
• Range - 8 ft
• Number of Tracked Points 1
• Wireless - no
• Interference and noise metal, earth
• Mass, Inertia and Encumbrance substantial
• Durability low
• Price high

• Pros
• Accurate
• Fast
• Low lag
• Minimal environmental interference
• No calibration
• Can incorporate force feedback
• Cons
• Low range (effectively 5 does not scale well)
• Cost
• 1 tracked point (body/others are hard to track)

21
Mechanical Tracking Products

• Fake Space Labs BOOM Display (discontinued)
• Sensible Phantom

22
Electromagnetic Trackers

• Emitter
• Apply current through coil
• Magnetic field formed
• 3 orthonormal coils to generate fields
• Sensor
• Strength attenuated by distance
• 3 orthonormal magnetic-field-strength sensors
• Determine the absolute position and orientation
  of a tracker relative to a source.
• Polhemus (a.c.)
• Ascension (d.c.)

23
Basic Principles of EM Trackers

• Pulse the emitter coils in succession
• Sensor contains 3 orthogonal coils
• For each pulse, sensor measures the strength of
  the signal its 3 coils (9 total measurements)
• Known
• Pulse strength at the source
• Attenuation rate of field strength with distance
• Calculate position and orientation of the sensor
  coils

24
EM Trackers

• Data returned 6 DOF
• Spatial distortion 0.6 mm, 0.025
• Resolution 0.00508 mm, 0.025 / inch from
  receiver
• Jitter (precision) mm to cm
• Drift - none
• Lag reported 4 ms
• Update Rate - 120 Hz
• Range - 5 ft
• Number of Tracked Points 16 (divides update
  rate)
• Wireless - yes
• Interference and noise metal, earth
• Mass, Inertia and Encumbrance - minimal
• Durability - high
• Price - 4000

25
EM Trackers

• Data returned 6 DOF
• Spatial distortion 0.6 mm, 0.025
• Resolution 0.00508 mm, 0.025 / inch from
  receiver
• Jitter (precision) mm to cm
• Drift - none
• Lag reported 4 ms
• Update Rate - 120 Hz
• Range - 5 ft
• Number of Tracked Points 16 (divides update
  rate)
• Wireless - yes
• Interference and noise metal, earth
• Mass, Inertia and Encumbrance - minimal
• Durability - high
• Price - 4000

• Pros
• Measure position and orientation in 3D space
• Does not require direct line of sight
• Low encumbrance
• Cost
• Good performance close to emitter
• Lag
• Can be built into devices
• Earth magnetic field good for 3DOF
• Cons
• Accuracy affected by
• DC Ferrous metal and electromagnetic fields.
• AC Metal and electromagnetic fields
• Operate on only one side of the source (the
  working hemisphere)
• Low range (effectively 5 does not scale well)
• Calibration

26
EM Tracking

• Ascension Flock of Birds
• Polhemus Fastrak
• Extremely popular
• Good for many applications
• CAVEs (remove metal)
• HMDs
• Projection displays
• Fishtank

27
Acoustic/Ultrasonic Tracking

• Time of Flight Tracking
• Emitters
• Multiple emitters
• In succession, emit sound (record time)
• Receiver
• Report time of receiving sound
• Frequency tuned
• Calculate time-of-flight (1000 feet/sec)
• Use ultrasonic (high) frequencies
• Similar
• EM tracking
• Radar/sonar
• Phase Coherence tracking
• Orientation only
• Check phase of received signal

28
Ultrasonic Tracking System Setup
How much data does 1 transmitter provide? How
much data do 2 transmitters provide? How much
data do 3 transmitters provide?
Stationary Origin (receivers)
Tracker (transmitters)
distance1
distance2
distance3
29
Acoustic/Ultrasonic Tracking Characteristics

• Pros
• Inexpensive
• Wide area
• Encumbrance
• Cons
• Inaccurate
• Interference
• Requires line-of-sight

• Data returned 3 or 6 DOF
• Spatial distortion low (good accuracy)
• Resolution good
• Jitter (precision) mm to cm
• Drift - none
• Lag very slow
• Update Rate - 120 Hz
• Range 40 (scaling issues)
• Number of Tracked Points numerous
  (spread-spectrum)
• Wireless - yes
• Interference and noise medium, noise,
  environment
• Mass, Inertia and Encumbrance - minimal
• Durability - high
• Price cheap to 12000

30
Ultrasonic Tracking Devices

• Logitech
• Mattel Power Glove
• Intersense
• Used as part of hybrid systems

31
Inertial Tracking

• Electromechanical devices
• Detect the relative motion of sensors
• Measuring change
• Acceleration (accelerometers)
• Gyroscopic forces (electronic gyroscopes piezo
  electric)
• Inclination (inclinometer)
• Frameless tracking
• Known start
• Each reading updates current position

32
Accelerometers

• Mounted on to body parts
• Detects acceleration
• Acceleration is integrated to find the velocity
• Velocity is integrated to find position
• Unencumbered and large area tracking possible
• Difficult to factor out gravity

33
Accelerometer Tracking Errors

• Suppose a constant error ?i, so that measured
  acceleration is ai(t) ?i
• vi(t) ?(ai(t) ?i)dt ? ai(t)dt ?it
• xi(t) ? vi(t)dt ???(? ai(t)dt ?t)dt
• xi(t) ?? ai(t)dtdt 1/2 ?it2
• Errors accumulate since each position is measured
  relative to the last position
• Estimated 10 degrees per minute. How is this
  related to drift?

34
Inertial Tracking

• Inclinometer
• Measures inclination
• Relative to some level position

• Gyroscopes
• Resist rotation
• Measure resistance

35
Inertial Tracking Systems Characteristics

• Data returned 3 or 6 DOF
• Spatial distortion low (good accuracy)
• Resolution good
• Jitter (precision) low
• Drift - high
• Lag very low
• Update Rate - high
• Range very large
• Number of Tracked Points 1
• Wireless - yes
• Interference and noise gravity
• Mass, Inertia and Encumbrance - minimal
• Durability - high
• Price cheap

• Pros
• Inexpensive
• Wide area
• Orientation very accurate
• Minimal interference
• Encumbrance
• Cons
• Position poor
• Need to recenter
• Calibration
• Inaccurate over time
• Drift

36
Optical Trackers

• Use vision based systems to track sensors
• Outside-Looking In
• Cameras (typically fixed) in the environment
• Track a marked point
• PPT tracker from WorldViz (www.worldviz.com)
• Older optical trackers
• Inside-Looking Out
• Cameras carried by participant
• Track makers (typically fixed) in the environment
• Intersense Optical Tracker
• 3rdTech HiBall Tracker

Image from High-Performance Wide- Area Optical
Tracking The HiBall Tracking System, Welch, et.
al. 1999.
37
Outside Looking In Optical Tracking

• Precision Point Tracking by WorldViz
• IR Filtered Cameras are calibrated
• Intrinsics
• Focal length, Center of projection, aspect ratio
• Extrinicis
• Position and orientation in world space
• Each frame
• Get latest images of point
• Generate a ray (in world coordinates) through the
  point on the image plane
• Triangulate to get position

38
Outside Looking In Optical Tracking

• What factors play a role in O-L-I tracking?
• Camera resolution
• Frame rate
• Camera calibration
• Occlusion
• CCD Quality
• How does it do for
• Position
• stable, very good
• Orientation
• Unstable, poor
• Latency
• Cameras are 60Hz

39
Orientation

• How to compensate for poor orientation?
• Combine with orientation only sensor (ex.
  Intersenses InertiaCube)
• Also known as
• Hybrid tracker
• Multi-modal tracker
• Position vision
• Orientation inertial

40
Inside-Looking-OutOptical Tracking

• Tracking device carries the camera
• Tracks markers in the environment
• Intersense Tracker
• 3rdTech HiBall Tracker

Images from High-Performance Wide- Area Optical
Tracking The HiBall Tracking System, Welch, et.
al. 1999.
41
HiBall Tracker

• Position
• Pretty good
• Orientation
• Very good
• Latency
• LEPDs can operate at 1500 Hz

Six Lateral Effect Photo Dioides (LEPDs)
in HiBall. Think 6 cameras.
42
LED Optical Trackers

• Sensors
• Webcameras
• Photodiodes
• Track
• LEDs
• Reflected LED light
• Why LEDs?
• Easy to track
• Grab your webcam and point a remote at it
• Super cheap
• P5 Glove
• Nintendo Wii
• WorldViz PPT
• Virtual Patients

43
Optical Tracking Review

• Data returned 6 DOF
• Spatial distortion very low (very good
  accuracy)
• Resolution very good
• Jitter (precision) very good
• Drift - none
• Lag moderate
• Update Rate low - high
• Range very large (40 x 40 )
• Number of Tracked Points 4
• Wireless - yes
• Interference and noise occlusion
• Mass, Inertia and Encumbrance - moderate
• Durability low - high
• Price cheap to very expensive

• Pros
• Inexpensive
• Wide area
• Very accurate
• Cons
• High quality is very expensive
• Occlusion
• Calibration

44
Hybrid Approaches

• Nintendo Wii

45
Angle Measurement

• Measurement of the bend of various joints in the
  users body
• Used for
• Reconstruction of the position of various body
  parts (hand, torso).
• Measurement of the motion of the human body
  (medical)
• Gestural Interfaces
• Sign language

46
Angle Measurement Technology

• Optical Sensors
• Emitter and receiver on ends of sensor
• As sensor is bent, the amount of light from
  emitter to receiver is attenuated
• Attenuation is determined by bend angle
• Examples Flexible hollow tubes, optical fibers
• VPL Data Glove

47
Angle Measurement Technology (cont.)

• Strain Sensors
• Measure the mechanical strain as the sensor is
  bent.
• May be mechanical or electrical in nature.
• P5 Glove 25 (!)
• Cyberglove (Virtual Technologies)

48
Joints and Cyberglove Sensors
Proximal Inter- phalangeal Joint (PIP)
Interphalangeal Joint (IP)
Metacarpophalangeal Joint (MCP)
Metacarpophalangeal Joint (MCP)
Abduction Sensors
Thumb Rotation Sensor
49
Angle Measurement Technology (cont.)

• Exoskeletal Structures
• Sensors mimic joint structure
• Potentiometers or optical encoders in joints
  report bend
• Exos Dexterous Hand Master

50
Other Techniques

• Pinch Gloves
• Have sensor contacts on the ends of each finger

51
Technology
Mechanical motion capture

• Dataglove
• Low accuracy
• Focused resolution

• Monkey
• High accuracy
• High data rate
• Not realistic motion
• No paid actor

52
Technology

• Exoskeleton angle sensors
• Analogous
• Tethered
• No identification problem
• Realtime
• No range limit
• Rigid body approximation

53
Body Tracking Technology

• Position Tracking
• Orthogonal Electromagnetic Fields
• Measurement of Mechanical Linkages
• Ultrasonic Signals
• Inertial Tracking
• Optical Tracking
• Inside Looking Out
• Outside Looking In

• Angle Measurement
• Optical Sensors
• Strain Sensors
• Exoskeletal Structures
• http//www.measurand.com/videos/ShapeTapeTheMovie.
  m1v

54
Recap Tracking Table

• Focusing on Head and Hand Tracking
• Data returned
• Magnetic 6 DOF
• Acoustic 3 DOF per sensor (need 3 to get 6 DOF)
• Inertial 3 DOF
• Optical 6 DOF
• Spatial distortion (accuracy)
• Magnetic good close to emitter, degrades quickly
• Acoustic okay close to emitter
• Inertial short time very good, poor due to drift
• Optical okay (webcam) to very good accuracy
• Resolution
• Magnetic good close to emitter, degrades quickly
• Acoustic okay close to emitter
• Inertial very good
• Optical okay (webcam) to very good accuracy
• Jitter (precision)
• Magnetic good close to emitter, degrades quickly
• Acoustic okay close to emitter

55
Recap Tracking Table

• Drift
• Magnetic none
• Acoustic none
• Inertial substantial
• Optical none
• Lag
• Magnetic low
• Acoustic moderate
• Inertial low
• Optical low to moderate
• Update Rate
• Magnetic good
• Acoustic poor
• Inertial good
• Optical poor to very good
• Range
• Magnetic 5
• Acoustic 15
• Inertial excellent

56
Recap Tracking Table

• Number of Tracked Points
• Magnetic 16
• Acoustic 16
• Inertial 1
• Optical lt4
• Wireless
• Magnetic yes
• Acoustic yes
• Inertial yes
• Optical yes
• Interference and noise
• Magnetic metal, Earth
• Acoustic environment, occlusion
• Inertial none
• Optical occlusion

57
Recap Tracking Table

• Mass, Inertia and Encumbrance
• Magnetic low
• Acoustic low
• Inertial low
• Optical low to high
• Durability
• Magnetic high
• Acoustic high
• Inertial high
• Optical low
• Price
• Magnetic 4000
• Acoustic 4000
• Inertial very cheap
• Optical cheap (wecams) - 180k (motion capture
  systems)