MultiResolution RealTime Stereo on Commodity Graphics Hardware

1
Multi-Resolution Real-TimeStereo on
CommodityGraphics Hardware

• Presented By Narek Manouk

2
Introduction

• Run stereo algorithm on commodity graphics
  hardware in order to free up CPU for higher level
  analysis of stereo results.
• Run real-time stereo algorithm for depth
  perception.
• This stereo algorithm was implemented on an
  NVIDIA GForce4 graphics card.

3
Algorithm Overview

• Previous methods use global optimization for
  depth. Not practical for real-time applications
  (e.g. visual obstacle avoidance for a robot).
• Uses correlation-based techniques to provide
  real-time, per-pixel, depth map.
• Uses sum of square differences (SSD)
  dissimilarity measures, using different sized
  windows.
• Can run efficiently on todays GPUs.

4
Method

• Uses plane sweep method to generate
  correspondence map between two images.
• Compares intensities of pixels at the same
  location on two different images, using square
  difference.
• (Ix,y Ix,y)2

Larger disparities equates to a denser
correspondence map (i.e. more detail)
f
Object
Stereo cameras
b
z
z0
z1
z2
zi
Disparity value, Z-fb/zi
5
Method

• Calculate the Sum-of-Square-Differences (SSD),
  for all disparities
• A low SSD indicates the true depth.
• n is the window size.

6
Window Size

• So what size n (window) should be used?
• Large n
• Reduces the probability of a mismatch
• A strong single minimum (low SSD), which
  corresponds to the true depth
• Loss of accuracy (true depth less accurate).
  Strong SSD minimum within the neighborhood of the
  true depth.
• Small n
• Multiple minima exist
• Minima well localized (more accurate depth)
• Use both. Combine the robustness of a large
  window with the accuracy of a small one.

7
MML

• Combination of using different window sizes leads
  to the Multiple Mip-map level (MML). Each level
  corresponds to a window size.
• A Single Mip-map level (SML) uses only one window
  size.

6-level SSD 6 different window sizes (multi-resolu
tion)
8
Algorithm

• for (i0 iltsteps i)
• //the scoring stage, which computes the SSD
• computeSSD()
• //the aggregation stage, which sums up SSD
  scores from different mipmap level
• if (MMLgt0) //MML is the maximum mipmap level
• sumAllMipLevels(MML)
• //selection stage, which selects the depth with
  the minimum SSD score
• selectMinimumSSD()
• //optional filter to filter disparity map
• MinFilter()

9
Disparity Map

• Depth value encoded in the RGB channel
• SSD score encoded in the alpha channel
• SML method causes resolution to drop as window
  size increases
• MML method generates better results.
• Little gain with MMLgt4

MML
SML
Increased window size
10
Results

• MML produces more detail but at a lower frame
  rate
• SML produces less detail but at a higher frame
  rate
• Algorithm exhibits good linear performance with
  respect to image size

MML 4x4
SML
11
Conclusion

• Algorithm runs on NVIDIA GForce4 graphics card
  having equivalent performance to the fastest
  commercial CPU implementation.
• 6 to 8 frames/sec. achieved using 100 disparity
  search range, 256x256 size images, and
  un-optimized C code.