Compression of Mesh Connectivity and Video

1
Compression of Mesh Connectivity and Video

• Advancement to Candidacy Talk
• Pablo Diaz-Gutierrez
• University of California, Irvine
• March 24th, 2005

2
Introduction

• Larger and larger data sets are generated,
  transferred and stored daily
• Need for interactive visualization and
  processing
• Medical diagnosis
• CAD
• Meteorology/Satellite imagery
• GIS
• Even art! (Digital Michelangelo project)
• Local processing power is limited
• For how long will Moores law apply?
• But network bandwidth grows much faster
• Distributed systems are the future of
  supercomputing
• Optiputer project (Optical networking, Internet
  Protocol, computer storage, processing and
  visualization techniques)

Courtesy of Digital Michelangelo Project
3
ContextOptiputer project

• New approach to large scale computing where main
  element is fast optical networking, not
  computers.
• Thrive on huge bandwidth due to parallelism in
  optical networking (different wavelengths)
• Computing clusters distributed worldwide
• San Diego, Chicago, Amsterdam
• Idea Learn to waste bandwidth to save scarce
  computing
• Still, data sets grow beyond current capabilities
• Need for some level of compression
• But high compression is not the only goal anymore
• Shift in compression requirements

Optiputer Project Logo
4
ContextObservations Desirable compression
features

• Compress once decompress many times
• Extra compression time is affordable
• High compression rates should not compromise fast
  decompression
• Decompression time lower bound is O(n). This is
  mandatory.
• Large datasets need to be displayed/processed on
  the fly
• Incremental, online decompression
• Use application-specific knowledge about data
• i.e. decompress data in the order it will be used
• MPEG assumes nothing about the compressed video,
  but can we do better if we know its a prefixed
  fly-around sequence?

C
D
D
D
D
D
D
D
Compress once, decompress many times


Decompression should be incremental
5
ContextRelevant types of data

• Video sequences of real world realistic
  reconstructions
• Digital museums
• Historic archiving
• Marketing
• Virtual catwalk
• Geometric models
• Tessellated surfaces
• 3D scanners
• CAD applications
• Character modeling
• Volumetric data (regular grids or tetrahedral
  meshes)
• Atmosphere density, temperature, etc
• Medical scans
• Seismologic studies

Courtesy of VRVis-GmbH
Institute of Computer Graphics and
Algorithms Vienna University of Technology
6
Talk Overview

• Compression of Video Sequences
• Compression of Geometric Models

7
Talk Overview

• Compression of Video Sequences
• Compression of fly-around sequences with
  Geometric Hash Tables
• Compression of Geometric Models

8
Fly-around Video Compression withGeometric Hash
Tables

• Standard approach to video compression (MPEG,
  DivX, etc.) is content-independent
• No information other than the video itself is
  used
• Lots of redundancy (same objects from different
  viewpoints)
• Opposite extreme uses/retrieves total
  information Geometric reconstruction
• Expensive
• Not always possible (highly unstructured/transluce
  nt data)
• Intermediate approach
• Loose reconstruction of scene geometry
• Direct geometry texturing from input

Courtesy of http//sprott.physics.wisc.edu/sprott.
htm
9
Problem Description
Input Calibrated video sequence. This is a set
of images associated to the position and
orientation from where they were taken.
Output Set of billboards that look like the
input when seen from input viewpoints. We encode
the video sequence as a set of textured
billboards and a sequence of camera positions and
orientations.
Output
Input
10
Related WorkPaul Debevecs Façade

• Image-based modeling system
• The user interactively selects edges in
  calibrated images
• System constructs a boxy approximation to the
  modeled geometry
• User help the system match image edges to
  geometry
• Automatic texturing of model from photographs

Images by Paul Debevec (http//www.debevec.org)
11
Description of our method

• We exploit the existing redundancy in video
  sequences.
• If a fixed point in the geometry has the same
  color in images from different viewpoints, the
  corresponding input pixels can be hashed into
  this point in the geometry (hash point).
• Collections of hash points on a plane define a
  hash plane.
• The colors of the hash points give the texture
  map of the hash plane (compression).
• Rendering the textured hash planes from the input
  viewpoints is equivalent to playing the video
  (decompression).
• Idea Hash input pixels to points in the
  geometry.
• Find set of planes approximating the unknown
  geometry within e distance
• e determines the compression ratio
• Hash input pixels to points on the planes
• As long as output looks like input, we dont
  need geometric precision

Redundancy in video sequences
12
Applications of our method

• Fly-around sequences with static models
• Background capture
• Quick and cheap scene reconstruction
• In general, the applications of a low cost
  image-based modeling tool
• Not intended for general purpose video compression

13
Finding Hash Points
Optic flow A measure of the motion of pixels
between two video frames. It is a 2D vector
field. Wherever the flow is null, there is a hash
point.
Compute optic flow on dense set of parallel
planes in front of the input cameras. We get a
set of hash points near the model surface, and
then triangulate them.
Optic flow in two video sequences. A Looking
back from a forward moving train. B View from
cockpit of landing airplane.
14
Finding Hash Planesfrom Hash Points

• We find candidate planes using a stochastic
  procedure called RANSAC (Random Sample
  Consensus)
• Method for robust fitting of models in the
  presence of outliers.
• Randomly pick a number of data items that
  describe a fitting model.
• After several iterations, there is high
  probability of finding a good model.

15
Texturing the hash planes
Finally, we texture the hashed regions of the
planes with the projected textures from the most
front-facing cameras, or alternative ones if
there is occlusion. For this, we use a per-pixel
color voting procedure, where each camera that
sees a hash point votes with a color, and the
most voted color wins that point.
Color voting where 7 cameras see two pixels
Equivalent geometry of a textured sphere
16
Results, Problems and Possible Solutions

• Problems
• Holes in texture
• Non convergence of iterative process

• Possible Solutions
• Compact surface growing.
• Adding some view-dependent planes to hash the
  un-hashed pixels.
• Results expected soon.

17
Talk Overview

• Compression of Video Sequences
• Compression of fly-around sequences with
  Geometric Hash Tables
• Compression of Geometric Models
• Creation of a strip to capture mesh properties
• Hand Glove connectivity compression

18
Why compressing meshes?

• Large polygonal meshes are generated, transferred
  and stored daily
• 3D scanners, CAD, GIS, videogames
• Efficient storage and transmission is necessary
• Current techniques consider compression ratio
  alone, but we also need to consider
• Decompression efficiency.
• Compression of associated data (texture
  coordinates, color, material, etc.).

Rendering of Michelangelos David. 8 million
triangles. Digital Michelangelo Project
19
What is mesh compression?
element vertex 8 element face 12 1.0 1.0 1.0 1.0
1.0 -1.0 -1.0 1.0 -1.0 -1.0 1.0 1.0 1.0 -1.0
1.0 1.0 -1.0 -1.0 -1.0 -1.0 -1.0 -1.0 -1.0 1.0 3
2 1 1 2 5 2 6 5 4 5 6 7 4 6 2 3 7 7 6 2 1 5 4 4 7
3 4 3 0 0 1 4 3 1 0

• Polygonal 3D models are defined by two
  fundamental pieces of information, each of which
  can be compressed independently
• Vertex coordinates
• Generally lossy compression
• Variants of generic techniques usually work
• Quantization, location prediction, etc
• Connectivity information (polygons, tetrahedra)
• Obligatory lossless compression
• Most uncompressed storage is connectivity
• No generic techniques available

20
Contribution
element vertex 8 element face 12 1.0 1.0 1.0 1.0
1.0 -1.0 -1.0 1.0 -1.0 -1.0 1.0 1.0 1.0 -1.0
1.0 1.0 -1.0 -1.0 -1.0 -1.0 -1.0 -1.0 -1.0 1.0 3
2 1 1 2 5 2 6 5 4 5 6 7 4 6 2 3 7 7 6 2 1 5 4 4 7
3 4 3 0 0 1 4 3 1 0
2 6 5 4 7 10 11 1 12 9 8 3

• Current techniques obtain strips (linear ordering
  of the mesh components triangles, quads), only
  as a byproduct.
• Instead of getting strips accidentally, we start
  finding a single-strip and then compress it. Mesh
  connectivity is implicit in the encoding of the
  strip.
• But why are strips so useful?

21
Why compressing strips?

• Triangle strips render faster than separate
  triangles (save 66 bandwidth)
• Single-strips can be used for
• Mesh simplification
• Visibility culling
• Vertex-cache optimization
• Etc.
• Single-strips improve lossy compression of vertex
  coordinates and other data (i.e. Context-based
  Space Filling Curves)
• In general, many algorithms are cleaner and more
  efficient when using a strip, but calculating the
  strip is expensive (Hamiltonian Cycle
  NP-complete). Pre-compute and store.

Mesh simplification
Improved lossy compression
22
Previous WorkFinding a Single-strip

• This technique (by D. Eppstein M. Gopi) finds a
  single-strip of a triangle mesh.
• We will modify it to have the strip capture some
  properties of the model.
• The first step is to find a perfect matching in
  the dual graph of the mesh. If we connect the
  unmatched edges of the triangles, we get a set of
  disjoint loops.
• Second, we join the loops using a set of
  operations as shown below.

From 2 disjoint loops to a Single-strip
Perfect matching is the first step of
stripification
23
Constraining the Strip

• The obtained strip is mostly determined by the
  matching found.
• There are many different matchings for a dual
  graph of a triangulated manifold.
• We influence the choice by giving weights to the
  edges and using a weighted perfect matching
  algorithm.
• The weighing scheme depends on the mesh
  properties we want to capture with our strip.
• We demonstrate strip constraining with two
  different applications
• Back-face culling
• Vertex cache optimization

24
First Application Back-face CullingWeighing
Scheme

• Back-face culling techniques benefit from
  grouping similarly oriented faces together. The
  visibility of these face clusters can be checked
  at little cost.
• In order to be useful, our triangle strip should
  map long segments onto regions of low normal
  variance.
• Weighing scheme should form barriers to keep the
  strip within planar areas.
• We cluster the faces in the input model according
  to their normal deviation, and assign weights to
  the edges as follows
• Edge is within a cluster ? 0
• Edges joins clusters A, B ? angle (A,B)
• With proper management, this strip enhances
  rendering speed.

25
First Application Back-face CullingStrip
Management

• Segments in this strip are the smallest pieces of
  geometry that will be tested for visibility.
• We build a segment hierarchy and do recursive
  visibility tests while rendering.

Visibility test
The segment tree data structure
26
Second Application Vertex Cache Optimization
Weighing Scheme

• To maximize cache performance, a vertex should
  appear many times in the strip within a distance
  that depends on the cache size.
• This is a property of space filling curves.
  Therefore, if the strip follows a s.f.c., it
  should have good cache behavior.
• To do this, we generate a vertex spanning tree in
  the triangle mesh, such that no triangle touches
  three vertices in the tree. Then assign edge
  weights
• Edge in tree ? 1
• Edge not in tree ? 0

Sierpinski curve (a space filling curve) and its
medial axis.
Cache-optimized strip on a sphere.
27
Results
Rendering is between 33 and 300 faster
Frames/second when rendering different models
with unconstrained and constrained single-strips.
28
Strips in other geometric objects

• We can calculate single-strips in
• Triangulated manifolds
• Of any genus
• With boundary (work in progress)
• Manifolds of quadrilaterals (work in progress)
• More work is required in
• Hybrid (quads triangles) manifolds
• Tetrahedral meshes
• However, today well assume we have solved those
  problems.

Surfaces of genus 0 and 1
29
Talk Overview

• Compression of Video Sequences
• Compression of fly-around sequences with
  Geometric Hash Tables
• Compression of Geometric Models
• Creation of a strip to capture mesh properties
• Hand Glove connectivity compression

30
Connectivity CompressionPrevious work

• Topological Surgery, by J. Rossignac.
• Cuts the manifold and unfolds it on a plane.
• Encode the unfolding and the reconstruction
  operations.
• Improved in Edge-breaker, by J. Rossignac.
• Encodes spirals of triangles on the manifold.
• A 5-word codebook describes all possible
  situations with 1 word per vertex.

31
Summary of strip-based connectivity compression
techniques

• Surface meshes Hand-and-Glove compression
• Triangulated manifolds
• Genus 0
• Genus gt 0
• With boundaries
• Meshes of quadrilaterals
• Hybrid meshes (triangles quads)
• Tetrahedral meshes

32
Hand and Glove compressionGenus-0 triangulated
manifolds

• Key observation A single strip on a triangulated
  sphere splits the vertices in two spanning trees
  (the 2 medial axes of the strip).
• We call these trees hand and glove because
  they conceptually wrap around each other.
• Each face in the strip contains vertices from
  both trees.
• The strip follows an out-order traversal of
  spanning trees.

Glove
Hand
33
Hand and Glove compression (2) Genus-0
triangulated manifolds

• We can thus encode the mesh as
• Two vertex spanning trees
• A bit string describing how to zip the trees
  together along the strip
• Encoding details
• Vertex spanning trees 2 bits per node
• Bit 1
• 1 has children
• 0 has no children
• Bit 2
• 1 has siblings
• 0 has no siblings
• Encoding the strip We need one bit per triangle,
  indicating where the next vertex comes from
• 0 (h) Next vertex is on hand tree
• 1 (g) Next vertex is on glove tree

1
0
1
0
1
0
1
0
Encoding of a strip as a zipping of two trees
34
Hand and Glove compression (3) Genus-0
triangulated manifolds Example
35
Hand and Glove compression (4) Genus-0
triangulated manifolds Compression results

• Raw storage need 4 bits/vertex.
• We can further improve compression
• Predicting the next hand/glove bit when the strip
  reaches a tree leaf save 1 bit/leaf (see prev.
  slide).
• Generic entropy/pattern based encoder on output
• We used arithmetic encoding
• Final storage 2.5-3.5 bits/vertex
• Compare to 6log2V bits/vertex uncompressed
• (V vertices in mesh)

36
Hand and Glove compression (5)Genus gt 0
triangulated manifolds

• When genus gt 0, strip might not separate vertices
  in 2 connected components, but only one.
• Further, vertex spanning trees might loop
• We dont have a spanning trees anymore, but
  spanning sub-graphs

Planar unfolding of a torus, showing a single
strip that produces a vertex spanning sub-graph
with a loop.
37
Hand and Glove compression (6)Genus gt 0
triangulated manifolds

• To cut the loop(s) in spanning sub-graph, we need
  two integers (2log2V bits) per loop.
• But loops 2genus of the object
• Increase in storage negligible for common data
  sets.
• If there is only one vertex spanning sub-graph,
  we encode the strip using two iterators on the
  tree, starting from different positions.
• This requires one integer to store starting
  position of second iterator.

Loop cut-points (equivalent nodes)
Traversal jumps here
Tree traversal with two iterators
38
Hand and Glove compression (7)Manifolds with
boundaries

• Each boundary (hole in the surface) generates an
  extra loop in the spanning sub-graph(s).
• Need 2 more integers per boundary.
• Same method works with no further changes.

b
3
Planar unfolding of a torus with a hole, showing
the corresponding loop in the spanning sub-graph.
a
a
b
39
Hand and Glove compression (8)All-quad and
hybrid meshes

• Encoding an all-quad mesh requires 2 extra bits
  to determine how the next quad is attached to the
  last added quad in the strip
• 0 front
• 10 left
• 11 right

?
?
0
11
0
10
?
0

• If the mesh is hybrid, one bit determines the
  next faces type
• 0 Triangle
• 1 Quad
• and appropriate face encoding (for either face
  type) is used.

40
Tetrahedral mesh compressionIdea Extending
Hand Glove

• Hand Glove for surfaces (2D) relies on
  classifying edges (1D) as matched/unmatched
  disjoint subsets.
• Then encode the structure of these subsets and
  how to zip them together.
• Hand Glove for tetrahedral meshes (3D) would
  classify faces (2D) in disjoint subsets as well
• Matched (strip does not cross them)
• Unmatched (between consecutive tetrahedra)
• The structure of these subsets is not as simple
  as with surfaces
• Forest of triangle-fans?
• Filling the gap between 2D-3D could show the path
  to arbitrary dimensions
• Just speculating

41
Talk Overview

• Compression of Video Sequences
• Compression of fly-around sequences with
  Geometric Hash Tables
• Compression of Geometric Models
• Creation of a strip to capture mesh properties
• Hand Glove connectivity compression

42
Conclusion

• High bandwidth changes compression needs
• Pure compression ratio is less important
• Decompression efficiency is crucial
• We exploit knowledge about data to improve
  compression
• Restriction on type of video
• Stripified geometric objects

43
Conclusion (2)

• Our connectivity compression method has
• Efficient compression
• Linear time decompression
• Incremental decompression
• After some further development, we would like to
  transparently apply it to visualization of huge
  simulation data sets (Terabytes).
• It would be interesting to support random access
  to compressed data
• Hardware support for compression
• Applicable to more algorithms (not just
  visualization)

44
Future Work

• Stripification of
• Hybrid meshes
• Tetrahedral (and higher dimensional) meshes
• Homogeneous approach to geometric model
  compression

45
Cest fini!Thank you.

• Questions?
• Comments?
• Suggestions?