Generating Raster DEM from Mass Points via TIN Streaming

1
Generating Raster DEMfrom Mass Pointsvia TIN
Streaming
raster DEM
LIDAR points
Jonathan ShewchukUC Berkeley
Martin IsenburgUC Berkeley
Yuanxin LiuUNC Chapel Hill
Jack SnoeyinkUNC Chapel Hill
Tim ThirionUNC Chapel Hill
2
Generating DEMs from LIDAR
LIDAR points
3
Billions of Points
courtesy of www.ncfloodmaps.com
4
DEM Generation via TIN Streaming
read points from disk
5
Example Result
500,141,313 Points 11 GB(binary, xyz, doubles)
LIDAR points
? on a household laptop with two harddisks ?
? in 67 minutes ?
? 64 MB of main memory ?
? 270 MB temporary disk space ?
6
Related Work
7
Rasterizing Billions of Points

• popular GIS packages
• ArcGIS 9.1 (limit 25 million points, 1 GB)
• QTModeller 4 (limit 50 million points, 1 GB)
• GRASS (limit 25 million points, 1 GB)

Agarwal et al. 06
8
Interpolation Methods

• Inverse Distance Weighting (IDW)
• Natural Neighbors
• Kriging
• Splines
• TIN
• linear
• quintic

9
External Memory Quadtree
From Point Cloud to Grid DEM A Scalable
Approach Agarwal et al. 06

• rearrange points on disk into quadtree

• process cell
• find its neighbors
• interpolate all (i,j)
• write rasters to temporary file

• sort temporary file on (i,j) write DEM

10
Our Approach TIN Streaming

• reorganize computations not points

• compute TIN in places where all points have
  already arrived
• raster, output deallocate
• keep parts that miss neighbors

• enhance points with spatial finalization

11
Our Approach TIN Streaming

• reorganize computations not points

• compute TIN in places where all points have
  already arrived
• raster, output deallocate
• keep parts that miss neighbors

• enhance points with spatial finalization

12
Our Approach TIN Streaming

• reorganize computations not points

• compute TIN in places where all points have
  already arrived
• raster, output deallocate
• keep parts that miss neighbors

• enhance points with spatial finalization

13
Delaunay Triangulation

• standard algorithm for TIN construction
• good properties for interpolating points

• a triangulation in which every triangle has an
  empty circumscribing circle

14
Incremental Point Insertion
Lawson 77 Bowyer 81 Watson 81

• locate triangle enclosing the point

• find and remove all triangles with non-empty
  circumcircles

• triangulate by connecting new point

15
Spatial Finalization Tags

• inject tags intothe point streamthat inform
  whengrid cells are freeof future points

finalized region
future pointsonly arrive inthese cells

• use tags to certify triangles as beingpart of
  the finaltriangulation andoutput them

16
Streaming TIN Generation
output to disk
input from disk
small bufferin main memory
Streaming Computation of Delaunay
Triangulations Isenburg et al. 06

• triangulate point stream seamlessly
• output completed parts reuse memory

17
Rasterization of Streaming TIN
input (pipe)streaming TINfrom triangulator
yet anothersmall buffer

• points from single UTM time zone
• triangles with long edges are killed
• output to a single row-ordered DEM

18
Streaming Rasterization Pipeline
spfinalize -i points.raw -ospb spdelaunay ispb
osmb -ospb tin2iso ismb o dem.bil
19
The Finalizer(spfinalize)
20
Spatial Finalization of Points
? compute bounding box
21
Spatial Finalization of Points

• ? compute bounding box
• ? create finalization grid
• count number of points per cell

22
Spatial Finalization of Points

• ? compute bounding box
• ? create finalization grid
• count number of points per cell
• ? output finalized points
• buffer per grid cell
• if full, output points in one randomized
  chunk followed by finalization tag

23
Spatial Finalization of Points

• ? compute bounding box
• ? create finalization grid
• count number of points per cell
• ? output finalized points
• buffer per grid cell
• if full, output points in one randomized
  chunk followed by finalization tag

24
Spatial Finalization of Points

• ? compute bounding box
• ? create finalization grid
• count number of points per cell
• ? output finalized points
• buffer per grid cell
• if full, output points in one randomized
  chunk followed by finalization tag

25
Spatial Finalization of Points

• ? compute bounding box
• ? create finalization grid
• count number of points per cell
• ? output finalized points
• buffer per grid cell
• if full, output points in one randomized
  chunk followed by finalization tag

26
Spatial Finalization of Points

• ? compute bounding box
• ? create finalization grid
• count number of points per cell
• ? output finalized points
• buffer per grid cell
• if full, output points in one randomized
  chunk followed by finalization tag

27
Spatial Finalization of Points

• ? compute bounding box
• ? create finalization grid
• count number of points per cell
• ? output finalized points
• buffer per grid cell
• if full, output points in one randomized
  chunk followed by finalization tag

28
Spatial Finalization of Points

• ? compute bounding box
• ? create finalization grid
• count number of points per cell
• ? output finalized points
• buffer per grid cell
• if full, output points in one randomized
  chunk followed by finalization tag

29
Spatial Finalization of Points

• ? compute bounding box
• ? create finalization grid
• count number of points per cell
• ? output finalized points
• buffer per grid cell
• if full, output points in one randomized
  chunk followed by finalization tag

30
Exploit the Existing Coherence

• document with finalization tags
• enhance by releasing chunks of points
• ? benefits of global sort, but cheaper

raw point stream
finalized point stream
31
Exploit the Existing Coherence

• document with finalization tags
• enhance by releasing chunks of points
• ? benefits of global sort, but cheaper

raw point stream
finalized point stream
32
Unfinalized Region as Quadtree

• as tags finalize cell after cell, maintain
  remaining cells in adaptive quadtree
• ? used for efficient overlap tests by the
  triangulator

33
The Triangulator(spdelaunay)
34
Modified Incremental Delaunay

• insert points

• when reading afinalization tag
• find certified triangles
• output them to disk or pipe
• deallocate data structure

35
Modified Incremental Delaunay

• insert points

• when reading afinalization tag
• find certified triangles
• output them to disk or pipe
• deallocate data structure

36
Modified Incremental Delaunay

• insert points

• when reading afinalization tag
• find certified triangles
• output them to disk or pipe
• deallocate data structure

37
The Rasterizer(tin2dem)
38
Rasterizing the Streaming TIN
39
Results
40
Neuse-RiverBasin
41
Pipeline Details for the 20 ft DEM
6 min
10 MB60,388 triangles
5 MB512 x 512 grid
6 min
35 MB 2,249,268 points
43 min
270 MB 239 files
8 min
19 MB444 million rasters
46 MB128 rows per file
42
Comparison to (20 ft)
Agarwal et al. 06

• 53 hours
• 3.4 GHz desktop
• 10,000 RPM disks
• scratch space
• quadtree 8 GB
• rasters 2.5 GB

• 67 minutes
• 2.1 GHz laptop
• 5,400 RPM disks
• scratch space
• compressed rasters 270 MB

• 86 of CPU time for expensive RST
• remaining 7.5 hours
• constructing quadtree 1.1 hours
• finding cell neighbors 5.6 hours

43
Demos Discussion
44
Demos

• streaming generation of DEM
• extraction of elevation contours

45
Streaming DEM Rasterization
raw points
46
Main Features

• order of magnitude faster than method that uses
  external memory

• avoid temporary query structure on disk
• data-driven, no gathering of neighbors

• interleave input computation output
• 100 CPU usage while interpolating

• no global sort
• dont fight the data, use existing coherence

47
Future Work

• breaklines / barriers (e.g. rivers, roads)
• use Constraint Delaunay Triangulation

• other interpolation methods
• natural neighbors
• inverse distance weighting

• stream other tasks
• thinning of points
• compression of raw LIDAR data (useful?)
• hydrological enforcement (streamable?)

48
Acknowledgements

• data
• Kevin Yi, Duke
• support
• NSF grants 0429901 0430065
• "Collaborative Research Fundamentalsand
  Algorithms for Streaming Meshes."
• NGA award HM1582-05-2-0003
• Alfred P. Sloan Research Fellowship

49
Thank You
executables, video, slides, paper, and soon also
source code http//www.cs.unc.edu/isenburg/