Accelerated Image Orthorectification in NASA WorldWind

1
Accelerated Image Orthorectification in NASA
WorldWind

• By Craig Collins

2
Topics covered

• A low cost method of georeferencing data
• Projectively texturing on 3D terrain
• Shadow mapping
• Lens Distortion correction

3
Photogrammetry with a UAV

• Camera is a frame sensor
• Various rigorous sensor models exist including
  pushbroom, scanline, and synthetic aperture radar
• DEM digital elevation model (essential for
  orthorectification)
• Orthorectification a type of 2D map projection
  where there is no perspective distortion

4
OSSIM ossimImageMosaic

• Performs histogram matching, tonal balancing, and
  layering of data
• Supports Applanix POS/AV DG sensor model frame
  sensor (a calibrated camera)
• Blends photos using layering, closest-to-center,
  feather blending, etc, etc
• Requires a pretty good DEM

5
The old method of georeferencing

• Space resection using ground control, tie points,
  and camera model geometry
• The camera model data is reconciled by ground
  control, however no biases are seen in the data

6
The Kalman filter

• The Kalman filter is an efficient recursive
  filter which estimates the state of a dynamic
  system from a series of incomplete and noisy
  measurements.
• An example of an application would be to provide
  accurate continuously-updated information about
  the position and velocity of an object given only
  a sequence of observations about its position,
  each of which includes some error. It is used in
  a wide range of engineering applications from
  radar to computer vision.

7
The new method of direct georeferencing

• The combination of these GPS and IMU, with a
  Kalman filter allows the accurate determination
  of both attitude (roll, pitch, heading) and
  position (x, y, z) of the camera at the time of
  exposure. This results in the same information
  that is normally obtained in photogrammetric
  block or bundle adjustments used in film
  collections. The primary operational difference
  is that the Emerge approach does not require
  ground control to obtain the solution of the
  attitude and position data. This saves large
  amounts of labor and schedule time, resulting in
  a low cost, georeferenced image product.

8
(No Transcript)
9
Accuracy of direct georeferencing of RMK images
(scale 113000, hg2000m)1
10
UAV demo for SGI workstation

• UAV demo shows an unusual use of projective
  texturing and shadow testing for accelerated
  image orthorectification.

11
SGIs UAV demo technology

• Written in OpenGL for the SGI workstation
• Undos perspective distortion of an image on a per
  pixel basis through perspective divide
• Uses shadow mapping techniques to occlude regions
  of terrain not covered by image

12
(No Transcript)
13
Shadow Mapping with HLSL

• Draws an image of the scene from the lights
  perspective
• Depth buffer stores information of distance to
  closest surface for each pixel represented in the
  scene
• The depth buffer is projectively textured to the
  scene
• The scene is drawn from the cameras perspective.
  Parts of scene where depth value is less than
  true value of depth to camera are in shadow

14
(No Transcript)
15
(No Transcript)
16
Pixel Shader 2.0 Released 2004

• Small programs are written in HLSL (High Level
  Shader Language), a language resembling C
• The program is ran on the GPU rather than the CPU
• Allows DirectX developer to use more complex
  rendering techniques (shadow mapping, projective
  texturing, etc, etc)

17
Lens distortion terminology

• Tangential lens distortion "decentering", or
  imperfect centering of the lens components and
  other manufacturing defects in a compound lens
• Radial lens distortion Distortion present in
  fisheye lenses
• Principal point The center point of an aerial
  photograph

18
"Plumb Bob" model of lens distortion

• Contains 5 distortion coefficients (K)
• Other inputs Focal length, principal point, and
  skew coefficient compose the camera matrix
• Not advised to use 5th coefficient, 3rd and 4th
  are generally not used either
• It is claimed that the error in the principal
  point of todays cameras is negligible

19
Matlab camera toolbox

• r is radius from principal point to pixel

20
UAV Plugin 1.1 uses K1 and K2

• Using only the first two is much quicker and
  results are acceptable
• Bilinear interpolation eliminates aliasing
  effects that cause image to look rough and blocky
• All 5 coefficients can be found using the Matlab
  camera toolbox
• OpenCV supports camera calibration

21
UAV for World Wind - Additional features

• Code exists as a plugin (no more compilation
  required by the developer)
• Real-time modification of individual image
  parameters
• Coexists with WW images
• Support for pre-processing of images on pixel
  level (for histograms)

22
(No Transcript)
23
Sources

• GPS/inertial data in aerial photogrammetry -
  http//www.ifp.uni-stuttgart.de/forschung/photo/ge
  oref-Dateien/georef.en.html
• SGI UAV demo http//www.sgi.com/products/software/
  performer/brew/uav.html
• American Society for Photogrammetry and Remote
  Sensing http//www.asprs.org/publications/pers/200
  2journal/may/georef.html
• OSSIM http//www.ossim.org/
• Real-time Lens distortion correction
  http//ieee.stanford.edu/ecj/docs/ECJ_demo.pdf
• Matlab Camera Calibration script
  http//www.vision.caltech.edu/bouguetj/calib_doc/h
  tmls/parameters.html
• Kalman filtering
• http//www.cs.unc.edu/welch/kalman/