Offical CFM Viewgraph

1
Multi-sensor Integration and Calibration Aspects
Dorota A. Grejner-Brzezinska Civil and
Environmental Engineering and Geodetic Science
The Ohio State University 470 Hitchcock
Hall Columbus, OH 43210 Tel. (614)
292-8787 E-mail dorota_at_cfm.ohio-state.edu
OSU
2
Presentation Outline

• Multi-sensor system
• AIMS Calibration experiences
• Camera calibration
• Lever arm calibration
• OTF INS calibration (GPS)
• Boresighting misalignment (AT)
• Summary, outlook

3
Multi-sensor System

• Kinematic platform, upon which multiple sensors
  have been integrated and synchronized to a common
  time base
• Provide three-dimensional near-continuous
  positioning of both the platform and
  simultaneously collected geo-spatial data
• In general direct georeferencing is used
• In principle, no external information, such as
  ground control, is needed except for the GPS base
  station
• Calibration of the components and the link
  between components -- a challenging task

4
Multi-sensor System Objectives

• Automation of the map making process the
  ultimate goal of digital photogrammetry
• Sensors based on different physical principles
  record complementary and often redundant
  information, leading ultimately to a more
  consistent scene description
• Offer a feasibility of automation of the
  photogrammetric tasks

5
Multi-sensor Systems Design Level

• Specifications of the accuracy requirements
• Sensor specifications
• Architecture of the sensor mount
• Time synchronization circuit (crucial)
• Most challenging
• Algorithms for sensor calibration
• Algorithms for data processing
• Methods for testing the performance

6
Multi-sensor Systems

• Variety of new spatial data acquisition sensors
• CCD-based cameras
• LIDAR (light detection and ranging)
• multi/hyper-spectral sensors
• SAR/IFSAR
• Optimal sensor fusion
• more consistent scene description
• complementary information
• redundant information

7
Direct georeferencing (direct platform
orientation DPO)

• Geometric data fusion ? time-space registration
  (georeferencing)
• Basis for higher level fusion (multi-sensor
  imaging systems)
• GPS/INS integration
• complementary and competitive data
• high accuracy (5-20 cm, 10-30 arcsec)
• high reliability
• fault-resistant
• cost effective (less ground control)
• mandatory for new spatial data sensors (LIDAR,
  SAR, multi/hyperspectral)
• experimental systems (University of Calgary, OSU
  AIMS?)
• commercial (Applanix)
• GPS multi-antenna systems for less demanding
  applications

8
Effects of Errors in Direct georeferencing on the
3D Ground Point Positioning

• Depend on
• Quality of GPS/INS error modeling
• Quality of boresight transformation
• Quality of GPS/INS lever arm estimation
• Quality of individual sensor calibration
• Rigidity of the mount (platform)

9
3D Ground Point Positioning Error Resulting from
Errors in the Exterior Orientation Simulation
10
3D Ground Point Error as a Function of Increasing
Errors in Attitude for Variable Error Levels in
Exposure Center Location (300 m altitude )
11
3D Ground Point Error as a Function of Increasing
Errors in Exposure Center Location for Variable
Error Levels in Attitude (300 m altitude )
12
Calibration of a Multi-sensor System Based on
AIMS? GPS/INS/CCD
13
OSU Center for Mapping AIMS?

• Over 20 airborne tests
• Several land-based tests
• Typical standard deviations (covariance analysis)
• position 2-4 cm
• attitude
• 5-6 arcsec for pitch and roll
• 7-10 arcses for heading
• Typical fit to ground truth
• 2-20 cm for flight altitude 300m
• 0.2-3 cm for land-based applications (10-20 m
  object distance)
• Calibration crucial component

14
System Calibration Concept
GPS Rover
OTF GPS Error Estimation and Ambiguity
Resolution
Lever Arm Calibration
CCD
INS
Camera Calibration
INS OTF Calibration
Sensor Mount
Boresight Misalignment
GPS Base
15
Hardware Configuration for Land-based Tests
16
Integrated System Calibration

• Camera calibration
• GPS/INS lever arm calibration
• Boresight calibration between the camera body
  and the INS body frames
• linear offsets and the rotation matrix
• determined from the imagery containing images of
  the ground control points

17
Integrated System Calibration

• Camera calibration
• focal length, principle point coordinates, and
  radial distortions
• performed at the indoor test range
• in-flight calibration (limited)
• USGS polynomial model (36)
• especially important for non-metric digital
  cameras
• must to be repeated any time after the CCD was
  detached

18
Lever Arm Test (2)
19
Radial Symmetric Distortion SurfaceCarl Zeiss
Distagon 4/50 mm
microns
20
Radial Distortion Difference Between Two
Calibrations
microns
21
50-mm lens equipped 4K by 4K CCD frame Camera
Calibration
22
CCD Camera Calibration Experiences

• Recalibration of AIMS digital camera
• Insignificant changes in distortion surfaces
• Linear parameters vary (especially principal
  point coordinates)
• No check on changing radiometric behavior
• Dust accumulates (factory cleaning)
• System performance unaffected

23
Lever Arm Determination

• Linear offsets from GPS phase center to INS body
  frame center (usually pre-surveyed)
• Have to be known with sufficient accuracy
• especially important for embedded systems where
  INS directly aids the carrier-phase-tracking loop
• Could be determined by the integrated filter
• speed of estimation depends on the vehicle
  dynamics

24
Calibration of the GPS Lever Arm Errors
GPS Lever Arm Error Estimation
GPS Lever Arm Error Estimation
10
4
5
2
0
0
Bias Estimate (x-axis, cm)
Bias Estimate (z-axis, cm)
-5
-2
-10
-4
0
500
1000
1500
0
500
1000
1500
time(s)
time(s)
Standard Deviation
Standard Deviation
10
10
8
8
6
6
Standard Deviation (x-axis, cm)
Standard Deviation (z-axis, cm))
4
4
2
2
0
0
0
500
1000
1500
0
500
1000
1500
time(s)
time(s)
25
Effects of GPS Lever Arm Errors
DD Carrier Phase Residuals level arm errors not
calibrated
DD Carrier Phase Residuals level arm errors
estimated
20
20
15
15
10
10
5
5
0
0
Residual (cm)
Residual (cm)
-5
-5
-10
-10
-15
-15
-20
-20
0
200
400
600
800
1000
1200
1400
1600
0
200
400
600
800
1000
1200
1400
1600
time(s)
time(s)
26
Lever Arm Test

• Lever arm offsets
• pre-surveyed 0, 0.95, -0.04 m
• distorted 0, 0.50, 0 m
• INS x-axis oriented towards south during the
  initial 50-sec stationary period
• 200-sec vehicle maneuvering

27
Van Trajectory
start
28
Difference between solutions with correct and
incorrect lever arm components
Motion starts
29
Difference between solutions with correct and
incorrect lever arm components
30
Difference between solutions with correct and
incorrect lever arm components (no initial
maneuvering)
31
Difference between solutions with correct and
incorrect lever arm components (no initial
maneuvering)
32
Camera/INS Boresight Misalignment
33
Death Valley, JPL Test
34
INS/Camera Mount
35
Boresight Misalignment

• Since DPO rotational components are naturally
  related to the INS body frame, they must be
  transformed to the imaging sensor frame
• The angular and linear misalignments between the
  INS body frame and the imaging sensor frame are
  known as boresight components
• The boresight transformation must be determined
  with sufficiently high accuracy

36
Boresight Misalignment Calibration

• Angular and linear misalignments between the INS
  body frame and the imaging sensor frame
• Resolved by comparison of the GPS/INS
  positioning/orientation results with independent
  AT solution
• or as a part of a modified bundle adjustment with
  constraints
• Should be performed at a specialized test range
• No flex or rotation of the common mount of the
  imaging and the georeferencing sensors can occur

37
Direct Georeferencing
38
Error in Object Coordinates Due to Errors in
Boresighting
39
Boresight Calibration Example

• Aerotriangulation Results for Boresight
  Calibration
• SoftPlotter data reduction and adjustment
  packages
• 2 cm accuracy assumed for control points
• 7 ? for image coordinate observations

• GPS/INS Results for Boresight Calibration
• 5-8 arcsec estimated standard deviations of
  attitude components
• 1-2 cm estimated standard deviations of INS
  center position

40
Boresight Matrix Estimation Example
RPht - photogrammetrically derived attitude
matrix RINS - GPS/INS derived attitude matrix S
- reflection matrix to approximately align INS
and camera frame axes (could be absorbed directly
by RB RB - boresight matrix
41
Boresight Estimation Quality

• Performed on ground control points (natural
  objects)
• The standard deviations for the boresight
  components
• linear displacements 0.22, 0.08 and 0.06 m
• rotation angles ?, ?, ? 0.01, 0.03 and 0.04 deg
• Possible reasons for modest quality
• Photogrammertic processing accuracy
• affected by poor signalization of control points
• Mechanical problems with the camera body/mount
• Image time tagging (rather unlikely)

42
IMU OTF Error Calibration

• The main sources of errors in an inertial
  navigation are due to the following factors
• The time rates of change of the velocity errors
  are driven chiefly by accelerometer errors and
  gravity disturbance
• The attitude error rates are driven primarily by
  gyroscope errors
• In the aligned INS the main errors in tilt are
  due to accelerometer bias and the gravity
  disturbance vector
• It is important that the IMU errors are properly
  estimated and applied by the feedback loop
• Platform maneuvers are needed to separated error
  sources

43
IMU Error Calibration
44
IMU Error Calibration
45
IMU Error Calibration Flight Trajectory
46
Accelerometer Scaling Factor for Different
Atmospheric Conditions
GPS time s
GPS time s
47
Positioning Error Growth During GPS Losses of
Lock Due to Obstruction or Interference(forward
direction, end of the gap)
48
Summary

• Direct orientation can be achieved with high
  accuracy
• Proper system calibration is crucial
• AT is needed for boresight calibration
• Radio interference poses problems (AT is needed)
• Continuity of the GPS lock can pose serious
  problems in land-based applications
• Rigidity of the multi-sensor system mount --
  embedded systems are preferred
• Precise GPS/INS time synchronization is crucial