TLS Data Processing Modules

1
TLS Data Processing Modules
Armin Gruen Zhang Li Institute of Geodesy
Photogrammetry Swiss Federal Institute of
Technology Zurich, Switzerland E-Mail
agruen,zhangl_at_geod.baug.ethz.ch
2
Contents ? Introduction ? Georeferencing by
Aerial-Triangulation ? Image Matching ? DTM
Generation ? Ortho Image Generation ?
Conclusions
3
IntroductionOverview of some line-scanner
digital sensor systems
4
Introduction ? Main Features of Airborne Digital
Sensors pushbroom three-line scanner
principle. High area coverage performance
(field of view and stereo angle). High
resolution and accuracy (spatial radiomatic).
Stereo Imaging capability.
Multispectral imaging capability. Direct
digital workflow avoiding the film development
and scanning. Accuracy oriented
sensors, using GPS/INS integration.
5
Introduction? TLS (Three Line Scanner) System ?
A new airborne digital line-scanner developed by
STARLABO Corporation of Japan. ? Imaging
system Focal length 60.0mm Field of
view 61.5deg Stereo lines 3 MS
lines 3 CCD-line pixels 10200
Pixelsize 0.007mm Stereo angle
21.0deg
6
Introduction? Sensor configuration ? JAE (Japan
Aviation Electronics Industries) ACE3000TLS
stabilizer. (500HZ attitude data) ? a
Trimble MS750 serves as Base GPS. (L1/L2
kinematic data at 5HZ) ? a Trimble MS750 serves
as Rover GPS. ? Software Trimble Geomatics
Office calculate the kinematic position. ?
Trimble TanzVector serve as Aircraft attitude
sensor data.
7
Introduction? Characteristics of TLS system ?
TLS system can obtain three different viewing
directional Seamless high-resolution (about
5-10cm in air-borne situation) images. ? The
raw imagery of TLS stereo channel can be directly
used for stereo measurement and image matching
procedure due to the employee of high quality
stabilizer and no additional image rectification
procedure is needed. ? TLS system contains the
GPS/INS integrated system, this system can
produce an accurate estimates for the sensor
position and altitude, so much less ground
control points is needed. After the precise
recovery of system calibration parameters, it
enables the direct georeferencing of TLS images.
? TLS system records digital image directly,
which enable users to easily process and analyze
them on the real-time basis and help minimize the
processing errors. ? TLS can also acquire the
multi-spectrum images.
8
Introduction
9
Georeferencing by Aerial-Triangulation? Indirect
Georeferencing In traditional photogrammetric
triangulation, the georeferencing problem is
solved indirect using some well-distributed
ground control points and applying geometric
constraints such as colinearity equations between
the image points and object points.
10
Georeferencing by Aerial-Triangulation? Direct
Georeferencing The integration of INS/GPS can
reach a very high absolutive accuracy. For GPS,
using the differential phase observations with
rover-master receiver separation below 30 km, 10
cm or even better absolute positional accuracy in
airborne kinematic environments can be achieved.
For high quality INS 15 arc sec can be achieved.
Translational offsets between GPS,
INS and camera Rotational offset between
axes of INS and camera
11
Georeferencing by Aerial-Triangulation Interior
Orientation Parameters of TLS ? Focal length
f, ? Central points coordinate x0, y0 for
each CCD arrays attached to the focal
plane, ? Inclination angle ? for each CCD
arrays to the image y axis, and ? Lens
distortion correction coefficients a1,a3 and a5.
Central Point
12
Georeferencing by Aerial-Triangulation
Georeferencing of the TLS Imagery ? The
geometry of the TLS imagery is weaker compared
with the traditional film-based frame
imagery. There are only 3 image lines be
recorded simultaneously and share the same
exterior orientations ? The orientations
of All image lines should be recovered
Piecewise polynomial model Interpolation
model ? Airborne digital sensors have to be
integrated with high accuracy INS GPS
13
Georeferencing by Aerial-Triangulation TLS
Sensor Configuration
Boresight misalignment between INS and camera
14
Georeferencing by Aerial-Triangulation
GPS/INS/Camera Translational Displacement
Correction ? GPS/INS displacement vector
correction using the aircraft attitude data
(Tanzvector data, RMS of directional error is 0.3
deg) ? INS/Camera displacement vector
correction using the output of INS
15
Georeferencing by Aerial-TriangulationModel 1
Direct Georeferencing Model (DGR)
X0-2,Y0-2,Z0-2 are unknowns to model the residual
errors after the GPS/INS/Camera displacement
correction.
rotation matrix from INS body to ground
coordinate frame, ?INS, ?INS, ?INS is the
original altitude which from INS sensor. unknowns
?1, ?1, ?1 is used to model INS draft errors.
16
Georeferencing by Aerial-TriangulationModel 1
Direct Georeferencing Model (DGR)
Colinearity equations (X,Y,Z) (x,y)
17
Georeferencing by Aerial-TriangulationModel 1
Direct Georeferencing Model (DGR)
Vectors of unknowns
Coefficient martices for unknown vectors Weight
martices Observation vectors
18
Georeferencing by Aerial-TriangulationModel 2
Piecewise Polynomial Model (PPM)
coordinates of perspective center denoted in the
ground coordinate system, XSi0,YSi0,ZSi0XSi1,YSi1
,ZSi XSi2,YSi2,ZSi2 are 0,1 and 2-order unknown
polynomials coefficients of ith segment to model
the perspective center.
rotation matrix from ground frame to camera frame
19
Georeferencing by Aerial-Triangulation Model 2
Piecewise Polynomial Model (PPM)
There are 2 kinds of constraints that may be
applied to each parameters at the segment
boundaries. The zero order continuity constraints
ensure that the value of the function computed
from the polynomial in each 2 neighboring
segments is equal at their boundaries,
i.e. The first order continuity constraint
required that the slope, or first order
derivative, of the functions in 2 adjacent
segments be forced to have the same value at
their boundary, i.e.
20
Georeferencing by Aerial-Triangulation Model 3
Cubic Spline Interpolation Model (CSI)
The Cubic Spline Interpolation model
computes the orientation parameters of reference
image lines at regular intervals, then calculates
the parameters of intermediate image lines by
cubic polynomial interpolation. These reference
image lines are so-called "orientation fixes"
21
Georeferencing by Aerial-Triangulation Test
Flight Area ? The GSI Geographical Survey
Institute) testing area. ? The testing area
covers about 650?2500m2 ? The image scale is
18000 and the footprint is about 5.6cm. ? All
control points are signalised.
22
Georeferencing by Aerial-Triangulation
Signalised Control Points
23
Georeferencing by Aerial-Triangulation
Semi-automatic Tie Point extraction
? Feature point extraction. The user can select a
point in one image , software can automatically
derive the nearest feature points in a image
window sized 41?41 pixels using Forstner interest
operator. ? Image pyramid generation and pixel
level accuracy conjugate points generation. ?
Subpixel accuracy matching by Least Squares
Matching. 6 geometric parameters are used in the
adjustment. Using the above semi-automatic
approach, several hundards of tie points
(standard deviation of 0.2-0.3 pixel) are
extracted in an interactive way.
24
Georeferencing by Aerial-Triangulation
Semi-automatic Tie Point extraction
25
Georeferencing by Aerial-Triangulation
Semi-automatic Tie Point extraction
Forward Nadir
Backward
26
Georeferencing by Aerial-Triangulation
Semi-automatic Tie Points extraction
Forward Nadir
Backward
27
Georeferencing by Aerial-Triangulation Accuracy
of the aerial-triangulation
? By modeling the remaining errors after GPS-INS,
INS-Camera displacement vector correction as a
constant offset value, about 0.10, 0.07m and
0.08m absolute accuracy in planar and height was
achieved using DGR model. ? Using PPM and CSI
model, better results, about 0.08, 0.05m and
0.07m absolute accuracy in planar and height was
achieved, especially the 20 sections PPM model,
about 1 pixel accuracy was achieved. ? Better
accuracy results can be achieved by using more
sections in PPM model and more orientation fixes
in CSI model.
28
Georeferencing by Aerial-Triangulation Accuracy
of the aerial-triangulation
29
Georeferencing by Aerial-Triangulation Accuracy
of the aerial-triangulation
30
Georeferencing by Aerial-Triangulation Accuracy
of the aerial-triangulation
31
Georeferencing by Aerial-Triangulation Accuracy
of the aerial-triangulation
32
Georeferencing by Aerial-Triangulation Accuracy
of the aerial-triangulation
33
Image Matching Image Pyramid Generation
? Enhancement of the texture in original image
(level 0) using the Wallis filter.
? Image pyramid generation by reduction factor
2, including the original image, the
pyramid level is fixed to 4.
Level 3
upper level
Level 2
lower level
Level 0
34
Image Matching Feature Points Extraction in
Upper Level Image
35
Image Matching Feature Points Extraction in
Upper Level Image
36
Image Matching Search Area Determination in
Upper Level Image ? Precise position/attitute
data after aerial triangulation procedure. ?
Coarse DEM/TIN data generated from the measured
tie control points.
Flight Trajectory
Nadir Image
Forward Image
Backward Image
Approximate Height
dh
-dh
37
Image MatchingPixel level accuracy conjugate
point generation by cross- correlation method
? Several candidate points are selected according
to their normalized correlation
coefficients using epipolar line
constraints ? Pixel level accuracy
conjugate points determined by forward
intersection
forward
nadir backward
incorrect match
correct match
38
Image MatchingTranformation of the matching
result from upper level pyramid image to lower
level image ? Feature point extraction in lower
level image. ? Search area
determined from parallax results at upper level
pyramid image
lower level feature
upper level feature
39
Image MatchingLeast squares matching in the
original image ? After the correlation in
highest level pyramid image, blunders are deleted
by forward intersection procedure. ? All 6
geometric parameters (shift shape parameters)
are used in adjustment, some bundles are deleted
in this stage. ? As a result, 12994 point
triples were extracted by using the test imagery
of GSI. The standard deviation is about 0.2-0.6
pixels.
40
Image Matching Some examples
41
Image Matching Some examples
42
DTM Generation Each matched point is a result
of 3 imaging rays. Using the forward
intersection procedure, the object coordinates of
these matching points can be obtained. TIN
data can be generated by a sufficient number of
object points, then using two-dimensional
interpolation algorithms, grid DTM data can be
generated.
43
DTM Generation
44
DTM Generation
45
Ortho-Image Generation Direct method
Xf(x,y) Yg(x,y)
y
Y
X
x
Original Image
Ortho-Image
46
Ortho-Image Generation Indirect
method This is our ortho-image
generation method of choice
x
47
Ortho-Image Generation Back-projection by an
iterative search algorithm The location
of the corresponding scan line of a certain
object point can be found with an iterative
search algorithm. After starting with an
initially approximate scan line in TLS image, the
final corresponding scan line can be found by
minimizing the perpendicular distance to the
corresponding CCD line.
48
Ortho-Image Generation Back-projection by an
iterative search algorithm
x-coordinate
Scan line number
Trajectory
P (X,Y,Z)
Ortho-image pixel
49
Ortho-Image Generation Examples
Original Image
Orthorectified Image
50
Ortho-Image Generation Examples
Original Image
Orthorectified Image
51
Conclusion
? An experimental software package has been
developed to allow the processing of the TLS raw
images including georeferencing, image matching,
DEMs generation, ortho-image generation and
stereo measurement. ? From our experimental
results, the geometric accuracy of TLS system is
about 1-3 pixels and not obtained the sub-pixel
accuracy, these may attribute to the
non-precise/fully sensor modeling. ? Untill now
only one strip of GSI testing area has been
evaluated in our experiments, the TLS images of
different image scale, different terrain type and
multi-strip need to be evaluated. Further more,
the stability of TLS system calibration
parameters need to be evaluated.