Real-time Structured Light Depth Extraction

1
Real-time Structured Light Depth Extraction

• Kurtis Keller Jeremy Ackerman
• Department of Computer Science
• University of North Carolina
• Chapel Hill, North Carolina, USA

EI2000
2
Divisions of talk

• Application and reason for research
• Comparisons of various, relevant depth extraction
  systems (structured light, etc..)
• First design with large digital projector
• Second design with real time custom projector
• Improvements...

3
High speed structured light depth extraction for
Augmented reality surgical procedures
4
Application Medical

• Virtual / Augmented Reality view from single
  incision.
• This information can be rendered with textures to
  surgeons viewpoint.
• Must be real-time, small, able to image
  shiny/curved surfaces.

5
Comparison of various, relevant Depth Extraction
Systems

• Stereo laparoscopes
• Traditional viewing of images
• Automated depth extraction poor
• Structured light depth Extraction
• Too slow with standard projector and camera
• Can be used on curved surfaces
• Laser scan
• Too slow for real time use
• Miniaturization

6
Surface Texture and Z depth scanning methods for
real-time medical use
7
Structured light (Binary encoded)

• Start with coarse projected texture, then
  increase number of bars for finer resolution.

1
2
3
8
First System
9
First Design Structured Light with large
digital projector
10
First System

• Not Real time
• Utilized off the shelf (modified) components
• TI micromirror projector
• Custom lens to flex scope to laparoscope
• 3D scans in lab and animal cadaver tests
• Bulky

11
First System(in operation)

• Close to Real time (60Hz frame imaging, 5Hz
  updates)
• Custom components
• Projector Lens
• Custom lens to flex scope to laparoscope
• Off the shelf componets
• TI Micromirror projector
• Laparoscope
• Pulnics progressive scan camera

12
Second System (real time)
13
Second System

• Almost Real time (10 Hz)
• Utilized custom components
• Displaytech ferro-reflective LCD engine in custom
  projector
• Custom lens to flex scope to laparoscope
• Fits in with standard surgical environment
• Custom projector snap onto standard medical
  laparoscope
• Light source 2 meters away from patient by fiber
  light
• Projector about the size of an orange, fitting to
  surgeons hand and light weight
• All exposed surfaces surgical steel and
  autoclaveable.

14
High-speed Projector

• Miniature size (50mm Dia x 75mm long)
• Custom components
• Displaytech 640x480 180Hz. engine
• Custom lens to laparoscope

15
High-speed Miniature Projector

• Attached to Laparoscope

• Exploded view

16

• Real time
• High-speed components
• Custom, 180Hz projector
• 180 Hz 512x512 pixel camera
• High speed graphics processor

17
Hardware Improvements

• Color additive scanning by synchronized color
  wheel
• Single tube Stereo Laparoscope
• Reduction of Specular rejection problems
• Increased light level bit depth
• Image processing/rejection

Color Wheel
18
FUTURE RESEARCH inHIGH SPEED STRUCTURED LIGHT

• Obtain Ferro reflective (or other) displays
  which are not hampered by crystal migration
• If possible, purchase graphics processor and high
  speed camera from same vendor
• Full projection imaging is being performed, not
  micro internals that we emphasized.
• Virtual Teleconferencing
• Inspection
• Surveillance