Jay Summet

1
Jay Summet
BurningWell Micro-controller based sensor
localization and tracking system using projected
light patterns

• -Intel Mentor Rahul Sukthankar

GVU Brownbag September 30th, 2004
2
Motivation

• Displaying an image on an arbitrary surface
  requires that you know where the surface is
  within the projection area.

Bottom photos courtesy of www.ShaderLamps.com
3
Motivating Movie
4
Previous Methods

• Manual Calibration Select control points
  manually.
• Requires user intervention.
• Slow.
• Possible user error.
• Camera Based Calibration Recognize object or
  detect fiducial marks.
• Error Prone.

5
Our Solution

• Hardware light sensor embedded in the display
  surface.
• Structured light cast from the projector encodes
  a position.
• Sensor decodes it's own position.
• Position can then be broadcast via radio (or used
  locally).

6
Closely Related Work

• Automatic Projector Calibration with Embedded
  Light Sensors Johnny C. Lee, Paul H. Dietz, Dan
  Maynes-Aminzade, Ramesh Raskar and Scott Hudson
  - To Appear, UIST 2004.
• RFIG Lamps Interacting with a Self-describing
  World via Photosensing Wireless Tags and
  Projectors - Ramesh Raskar, Paul Beardsley,
  Jeroen van Baar, Yao Wang, Paul Dietz, Johnny
  Lee, Darren Leigh, Thomas Willwacher -Proceedings
  of SIGGRAPH 2004.
• No autonomous operation.
• Locates the sensors with a pure Gray code
• Disadvantages May be wrong.
• Advantages Always provides a location.
• No tracking.

7
Hardware

• Microchip rfPIC 12F675
• 4 Mhz RISC Processor.
• 1024 program words.
• 64 bytes of RAM.
• 128 bytes of EEPROM.
• Built in Analog-to-Digital converter.
• UHF ASK/FSK 10dBm radio transmitter.

• Photo-transistor based light sensor.

8
Hardware (cont.)

• Microchip ASK receiver module
• Microchip 12F629 microprocessor.
• Detects/decodes radio signals, converts to RS-232
  Serial signal.

• Hardware cost
• Prototype Sensor/Transmitter 40 - Receiver
  50
• In quantities Sensor/Transmitter lt 10 -
  Receiver lt 15

9
Sensor Software

• Hand coded assembly (not by Russian programmers).
• 858 14bit program words (out of 1024 possible).
• Tightly integrated with the rfPic 12F675
  hardware.
• Analog-to-Digital converter.
• Interrupt Timer.
• Instruction Counter.
• UHF Transmitter.

10
Sensor Software (cont)

• Sample Light Sensor.
• Average of 16 samples.
• Sync to 60Hz signal.
• Find edges.
• Sample in the middle of bits.
• Threshold bit
• Store sampled bits in a queue.
• Search for a valid data packet.
• Looking for two different data packet types.
• Using FEC error detection signatures.
• Transmit data in background.
• Interrupt driven while doing the above.

11
RS-232 Receiver

• PIC 12F629
• Samples Radio.
• Decodes Data Packets.
• Transmits via RS-232 at 19,200 bps.

12
Terms

• Frame
• Projectors can update what they project 60 times
  per second.
• Pattern
• One frame of video.
• Packet
• Sequence of bits (one/zero or black/white) spread
  over multiple frames.

13
Location Pattern

• Packets consist of
• Gray Code Patterns
• SECDED checksum Patterns
• 21 bits - 16 data, 5 checksum
• 2.85 packets a second (3Hz)

14
Location Packet

• Check bits supports correction of 1 bit error, or
  detection of up to 3 bits of error.
• Limited in complexity of error detection code by
  what can be implemented on an 8 bit
  micro-controller with RAM measured in bytes.
• Current SECDED implementation uses 115 program
  words and 5 bytes of RAM, including the 3 byte
  buffer which holds the last 24 bits received.

15
Hexagonal Tracking Pattern

• 8 total bits
• Inverse replication of 3 data bits to 6 bits plus
  2 shared bits of framing.
• End of last packet is beginning of next.

16
Tracking Packets

• 10 bits are cross checked
• Framing bits must be the same (e.g. 11 or 00) and
  sequential frames must alternate (e.g.
  00......11...... 00).
• Check bits must be inverse of data bits.
• Still not as reliably distinguishable from noise
  as the location pattern, but much faster (7.5 /
  second).

17
Pattern/Packet Generation

• C program using OpenGL and the GL Utility Toolkit
  (GLUT).
• Allows easy image warping.
• RS232-gtUSB-gt /dev/ttyUSB0 serial interface used
  for tracking feedback.

18
Retrace Problem Example
19
Tracking Strategies

• Static
• Sensor is located where it was last observed.
• Static Variable Size
• Resize tracking pattern depending upon confidence
  of sensor location. (Less confidence, larger
  size.)
• Linear
• Linear model of sensor motion.

20
Demo Movie
21
Contributions

• Sensor able to operate independently.
• Error detecting code indicates when the location
  is successfully determined.
• Tracking of sensor while using rest of the
  display for output.
• Source code hardware design.

22
Questions?
Jay Summet summetj_at_cc.gatech.edu http//www.cc.gat
ech.edu/summetj/burningwell