Surface Reconstruction from Point Clouds by Transforming the Medial Scaffold

1
Surface Reconstructionfrom Point Clouds by
Transforming the Medial Scaffold

• Ming-Ching Chang
• Frederic Fol Leymarie
• Benjamin B. Kimia

2
Problem surface reconstruction with minimal
assumptions

• Context reconstruct a surface mesh from
  unorganized
• points, with a minimal set of
  assumptions the samples are nearby
  a possible surface (thick
  volumetric traces not considered here).
• Benefit reconstruction across many types of
  surfaces.

3
Goal surface reconstruction with minimal
assumptions

• To find a general approach, applicable to various
  topologies,
• without assuming strong input constraints,
  e.g.
• No surface normal information.
• Unknown topology (with boundary, for a solid,
  with holes, non-orientable).
• No a priori surface smoothness assumptions.
• Practical sampling condition non-uniformity,
  with varying degrees of noise.
• Practical large input size (gt millions of points).

4
Goal surface reconstruction with minimal
assumptions

• Surface normal not accurate, or problem locally
  solved
• Unknown topology practical (e.g., holes, in
  CAD)
• No smoothness practical (sharp features)
• Non-uniformity, noise practical acquisition
• Large input size scalable

5
How Literature Overview

• Implicit distance functions
• Locally approximate the distance function by
  blending primitives.
• Globally approximate the distance function by
  volumetric propagation.
• Propagation based (region growing) methods
• Voronoi / Delaunay geometric constructs
• Incremental surface-oriented.
• Volume-oriented.

• Many methods have additional assumptions in
  addition to unorganized points
• Surface normal imply knowing the surface
  locally.
• Surface enclose a volume (distance field) a
  strong global information.

6
Implicit Methods

• Locally blending primitives
• Blinn82, linear combination of Gaussian blobs
  Muraki91,
• bounded polynomial in 3 variables Taubin94,
• blended union of spheres Lim95,
• globally/compactly supported RBFs Carr03,
  Samozino06,
• MPUs Ohtake03,06, etc.
• Mainly differ in (i) how clusters are generated,
  (ii) type of implicit function used, (iii) how
  local functions are blended together.
• Global volumetric
• Hoppe92, volumetric integration Curless,
  Levoy96,
• level-set methods Zhao01,
• active surface model Terzopoulos91, Sharf06,
  etc.

Pro re-meshed water-tight surfaces Con need
surface normal or assume surface enclosing volume
7
Propagation-Based Methods

• Boissonnat84,
• Taubin99 Ball pivoting algorithm (BPA),
• Huang, Menq02 consider k-N-N samples, locally
  vary ball radius,
• Lin04 Intrinsic Property Driven (IPD),
• Gopi02 restrictive assumptions to solve local
  ambiguities.

BPA
BPA
Pro simple and efficient, large input. Con
topological errors. surface holes, problem
when surfaces come close, or near sharp
ridge/corner with low sampling.
Difficult to select candidate and detect /
recovery error Some use Delaunay triangles
(Hybrid methods).
8
Voronoi/Delaunay Based Methods

• Incremental surface-oriented
• Amenta, Bern99 Voronoi filtering, Crust.
• pole (farthest Voronoi vertex) to approximate
  normal.
• Local feature size (LFS) min. dist. from a
  sample to theoretical MA.
• Amenta, Dey et al.01 Co-cone.
• Petitjean, Boyer01 r-regularity.
• Cohen-Steiner, Da04 greedy, postpone
  difficult decisions.
• Volume-oriented
• Boissonnat84, Attali,98 Volume sculpting.
• Amenta01 Power Crust, Kolluri04
  EigenCrust.
• Dey, Goswami03,05 Tight Cocone, Robust Cocone.

Pro water-tightness, some theoretical
constraint. Con some need surface normal or
volume.
9

• How we solve it Find an Inverse of Sampling
• Relate the sampled surface with the underlying
  (unknown) surface and try to invert (recover) the
  sampling process

10
How Overview of Our Approach (2D)

• Not many clues from the assumed loose input
  constraints.
• Work on the shape itself to recover the sampling
  process.

• Key ideas
• Relate the sampled shape with the underlying
  (unknown) surface by a sequence of shape
  deformations (growing from samples).
• Represent (2D) shapes by their medial shock
  graphs. Kimia et al.
• Handle shock transitions across different shock
  topologies to recover gaps.

11
How Sampling / Meshing as Deformations
Schematic view of sampling infinitesimal holes
grows, remaining are the samples.
We consider the removing of a patch from the
surface as a Gap Transform.
2D
3D
12
How Medial Scaffolds for 3D Shapes A graph
structure for the 3D Medial Axis

• Classify shock points into 5 general types, and
  organized into a hyper-graph form Giblin, Kimia
  PAMI04
• Shock Sheet A12
• Shock Curves A13 (Axial), A3 (Rib)
• Shock Vertices A14, A1A3

Akn contact (max. ball) at n distinct points,
each with k1 degree of contact.
A special case of input of points the Medial
Scaffold consists of only A12 Sheets,
A13 Curves, A14 Vertices.
A14 Vertex
A12 Sheet
A13 Curve
13
How Medial Scaffolds for 3D Shapes A graph
structure for the 3D Medial Axis

• Augmented Medial Scaffold (MS) hyper-graph
  Leymarie PAMI07
• Reduced Medial Scaffold (MS) 1D graph structure
• Shock sheets are seen as redundant (loops in the
  graph).

Easter island statue point data courtesy of
Yoshizawa et al.
14
Background in Shape Deformation

• Kimia et al. represent shape as a member of an
  equivalent class (shape cell), each defined as
  the set of shapes sharing a common shock graph
  (in 3D, Medial Scaffold) topology.

• Medial Axis (MA) loci of centers of maximal
  spheres bi-tangent to shape boundary.
• Shocks Medial Axis points endowed with dynamics
  of flow, which arises when the MA is considered
  as singularities of the quench points of a
  propagating grass file from the contour samples.

15
How Organise/Order Deformations (2D)
A
B
NB A B share object symmetries. Symmetries due
to the sampling need to be identified.
Deformation in shape space
16
How Organise/Order Deformations (3D)

• Recover a mesh (connectivity) structure by using
  Medial Axis transitions modelled via the Medial
  Scaffold (MS).
• Meshing as shape deformations in the shape
  space.
• The Medial Scaffold of a point cloud includes
  both the symmetries due to sampling and the
  original object symmetries.
• Rank order Medial Scaffold edits (gap transforms)
  to segregate and to simulate the recovery of
  sampling.

Object symmetry
Sampling recovery
Meshed Surface Organized MA
Shock Segregation Leymarie, PhD02
17
Summary
Recover a mesh (connectivity) structure by using
MA transitions modeled via the MS - MA
transitions discretise and organise the shape
space (of deformations) such that we can hope to
invert the sampling process. The MS is itself a
compact form of the MA (and the associated VD)
qualitatively, similar shapes have a similar MS
ease of computation and manipulation/management.
Key idea the MS of a point cloud represents
together symmetries of both the surface sampling
and the original object symmetries (MA of the
object's surface(s) itself) our goal is also to
disambiguate these two classes of symmetries by
rank-ordering the used MA transitions (what we
called a segregation process). - NB important
difference with other common approaches we use
the underlying MA symmetries from beginning to
end, without assumptions of smoothness (e.g.,
feature-size of Amenta et al.) There are many
ways to rank order the selected transitions (and
thus navigate the shape-space) the following is
the result of our experimentation.
18
Algorithmic Method

• Enough theory
• Here is how we order symmetries (and thus gap
  transforms) in practice.

19
Algorithmic Method

• Consider Gap Transforms on all A13 shock curves
  in a ranked-order fashion
• best-first (greedy) with error recovery.
• Cost reflects
• Likelihood that a shock curve (triangle)
  represents a surface patch.
• Consistency in the local context (neighboring
  triangles).
• Allowable (local surface patch) topology.

A13 shock curve
3 Types of A13 shock curves (dual Delaunay
triangles) Represented in the MS by singular
shock points (A13-2)
(unlikely to be correct candidate)
20
Algorithmic Method

• How we order gap transforms
• Favor small compact triangles.
• Favor recovery in nice (simple) areas, e.g.,
  away from ridges, corners, necks.
• Favor simple local continuity (similar
  orientation).
• Favor simple local topologies (2D manifold).
• BUT allow for error recovery!

21
Ranking Isolated Shock Curves (Triangles)
Triangle geometry
(Herons formula)
(Compactness, Gueziecs formula, 0ltClt1)
Cost favors small compact triangles with large
shock radius R.
The side of smaller shock radius is more salient.
R minimum shock radius dmax maximum expected
triangle, estimated from dmed
Surface meshed from confident regions toward the
sharp ridge region.
22
Estimate the Sampling Scale
The maximum expected triangle size (dmax) can be
estimated from shock radius distribution analysis.

• Distribution of the A13-2 radii of all shock
  curves corresponding to

All triangles of shock curves of type I and type
II in the (full) Medial Scaffold of the point
cloud.
All triangles in the original Stanford bunny
mesh.
The median of the A13-2 distribution (dmed)
approximates its peak.
23
Cost Reflecting Local Context Topology
Cost to reflect smooth continuity of
edge-adjacent triangles
Typology of triangles sharing an edge
Typology of mesh vertex topology
Point data courtesy of Ohtake et al.
24
Strategy in the Greedy Meshing Process

• Problem Local ambiguous decisions ? errors.
• Solutions
• Multi-pass greedy iterations
• First construct confident surface triangles
  without ambiguities.
• Postpone ambiguous decisions
• Delay related candidate Gap Transforms close in
  rank, until additional supportive triangles
  (built in vicinity) are available.
• Delay potential topology violations.
• Error recovery
• For each Gap Transform, re-evaluate cost of both
  related neighboring (already built) candidate
  triangles.
• If cost of any existing triangle exceeds top
  candidate, undo its Gap Transform.

Queue of ordered triangles
25
Summary of Our Approach

• We relate an object and its sampling by
  navigating the shape space (of deformations).
• We organize this navigation by gap transforms on
  the Medial Scaffold.
• We select a path by ordering these transforms and
  allowing for error recovery.

26
Show Time!

• Some results
• Other issues
• Validation,
• Using a priori information,
• Dealing with large inputs,
• Sampling quality,
• Running time.
• Conclusions

27
Result Meshing a Toy Sheep Model (5K Points)
Top 50 of candidate triangles in the shock queue
Q1.
10
20
30
28
Contd
75
50
Final result water-tight surface mesh.
90
29
Results Surface with Various Types
Non-orientable
Water-tight surface
With boundary
With sharp ridges (discontinuous curvature)
Multiple components
With multiple holes
Multiply punctured
Closely knotted
Gold water-tight surface Blue mesh boundary.
Dataset are courtesy of Cyberware, Stanford data
repository, Stony Brook archive, H. Hoppe.
30
Result Videos on Meshing Algebraic Surfaces
Mobius strip
Costa minimum surface (courtesy of H. Hoppe)
31
Result Video on Meshing the Rocker Arm
Flat smooth regions are meshed prior to the
ridges/corners.
The rocker arm data courtesy of Cyberware.
32
Result Video on Meshing Stanford Bunny
33
Validation

• Superimpose our meshing result on the original
  mesh.

Color Original mesh in gray.
Difference of reconstructed triangles in green.
34
Other Issues

• Validation,
• Using a priori information,
• Dealing with large inputs,
• Sampling quality,
• Running time.

35
Re-mesh / Repair a Partial Mesh

• In the case that existing triangles (in addition
  to the points) are know a priori
• Assign high priority to existing triangles.
• Let candidates compete in the greedy algorithm.
• Similar if surface normal is available.

RESULTS
Meshing result of an implementation of ball
pivoting algorithm (BPA) containing holes /
topological errors.
4,102 points sampling a mechanical part (courtesy
of H. Hoppe)
Re-mesh results of our algorithm (a solid)
36
Handle Large Datasets (Millions of Points)

• No strong constraints (topology, boundary,
  volume, etc.) on input.
• Divide input into buckets (or any full partition
  of space).
• Mesh surfaces in each bucket.
• Stitch surfaces by applying the same algorithm
  again.

Meshing Stanford Asian Dragon (3.6M points).
Related to Dey et al.01 Super Cocone.
37
Result of Stitching After Meshing in Buckets
38
Result Bucketing Stitching Video
120,965 points, divided into 20,000 points per
bucket.
Sapho dataset courtesy of Stony Brook archive.
39
Meshing Stanford Thai Statue (5M points)
40
Dealing with sampling quality

• Input of non-uniform and low-density sampling

Response to additive noise
50
100
150
41
Surface Meshing Running Time

• Roughly linear to the number of samples.
• Performance similar to other recent Delaunay
  filtering methods.

42
Compute Medial Axis for Arbitrary Shapes
Applications Shape Analysis, Reconstruction,
Segmentation, Manipulation
Multiple components
Curve skeleton (Centerline) detection
Corresponding surface patches of a Medial
Scaffold
43
Conclusions

• Surface reconstruction from point clouds.
• Handle a great variety of surfaces of practical
  interest.
• With little restrictions on input.
• Mesh surface by applying min. cost Gap Transforms
  in best-first manner, considering
• Geometrical suitability of candidate Delaunay
  triangles.
• Shock type, shock curve radius.
• Continuity from neighbors.
• Mesh topology.
• Multiple-pass greedy algorithm with error
  recovery.
• Potential to handle arbitrarily large datasets.

44
Future Work Discussions

• Additional Shock Transforms to handle all shock
  transitions.
• Better greedy error recovery.
• Medial Axis regularization application to shape
  manipulation, segmentation, recognition.
• Surface meshing theoretical guarantees.

Medial Scaffold (MS) Corresponding
surface patches Regularized MS
Acknowledgments Support from NSF. Coin3D
(OpenInventor) for visualization/GUI.
Stanford, Cyberware, MPII, Stony Brook archive
for 3D data.
45
Shape is..
About dynamics.
We seek informative ways to navigate the shape
space.
46
Shape is..
about dynamics as much as it can be about the
reconstruction of some static features and
traces. Shape (problems) can be understood by
the recovery of the history of events that can
provide some useful explanations. We seek
intelligent / informative ways of navigating the
shape space.