Texture Sprites: Texture Elements Splatted on Surfaces

1
Texture Sprites Texture Elements Splatted on
Surfaces

• Sylvain LefebvreSamuel HornusFabrice Neyret
• GRAVIR/IMAG-INRIAACM SIGGRAPH 2005 Symposium on
  Interactive 3D Graphics and Games 2005

2
Abstract
3
Introduction

• Texturing 3D models with very high resolutions
  create two major difficulties
• Storage cost of high resolution texture maps can
  easily exceed the available texture memory
• Creating high resolution textures proves to be a
  difficult and tedious task
• Lack of representation for textures composed of
  sparse elements
• Wastes a lot of memory
• Discontinuities
• Flickering due to Z-fighting

4
Introduction

• Texture sprites
• The texture element live on the object surface
  and can be updated dynamically
• Provides a general and convenient scheme for
  sprite-based texturing
• Homogeneous appearances obtained by overlapping
  many sprites
• Interactive addition of local details
• Geometry and composite texture are independent
• Not require 1) mesh to conform to any constraint
  2) global planar parameterization
  3) modify the initial mesh

5
Contributions

• A parameterization-free texture representation
  allowing efficient storage and filtered rendering
  of composite textures
• An efficient solution to dynamic decal management
  and animated texture elements
• New appealing texturing effects
• Complete GPU-implementation

6
Sprite-Based Textures

• Key idea
• Store the sprite attributes (position,
  orientation, etc) in a 3D hierarchical structure
  surrounding the mesh
• Follow the idea of octree textures
• The sprites attributes in the leaves of the
  hierarchical grid
• The entire data-structure is compact
• Many sprites share a same texture pattern
• Information is stored only around the surface

7
Sprite-Based Textures

• Easy to author
• add, drag, drop, change size, and orientation
• The system is very flexible
• Sprite ordering is controllable
• Sprite overlapping can be mixed
• Each sprite can be independently animated
• For instance react to surface deformations

8
Difficulties

• How to manage the hierarchical 3D grid on the GPU
  (fragment program)
• How to use the hierarchical 3D grid to store and
  retrieve sprites attributes
• How to filter the composite texture and blend
  overlapping sprites together

9
3D Hierarchical Grid (in GPU)

• Problem
• Store the hierarchical grid in texture memory
• Retrieve data from the grid at a given 3D
  location
• (This is implemented in a fragment program)
• We call this structure N3-tree
• N in each dimension
• Octree ? N 2
• Implementation N 4

10
N3-tree Memory Layout

• Tree structure
• Node indirection grid consisting of N3 cells
• Cell content (cell type is determined by a flag
  stored in the A-channel)
• A 1, data (leaf)
• A ½ , a pointer
• A 0, empty
• Store our structure in a 3D texture called the
  indirection pool
• Data values and pointer values are stored as
  RGB-triples

11
Notations

• NNN resolution of the indirection grid
  corresponding to each tree node
• Nl NlNl resolution of virtual 3D grid at level
  l
• At level l a point M in space ( M 0, 1)3 )
  lies in a cell located atand has a local
  coordinate within this cell. We have

12
Notations

• Node are packed in the indirection pool
• S (Su, Sv, Sw) size of indirection pool
• NSuNSvNSw resolution of the texture hosting
  the indirection pool
• A texture value in the indirection pool is
  indexed by a real valued P 0, 1)3
• P PiPf

13
Notations

• Pi identifies the node which is stored in the
  area
• Pi is called the node coordinates
• Pf is the local coordinate of P within the nodes
  indirection grid
• (!! Packing the cells does not change the local
  coordinates.)

Equation(1)
14
Tree Lookup

• For rendering, we need to retrieve the data
  corresponding to a given location M in space
  (more precisely, on the object surface)
• This is done per-pixel in a fragment program
• Start from level 0 in the tree
• At level l let Pl,i be the coordinate of the
  current node in the indirection pool
• The RGBA value corresponding to M is found in
  Pl,i at the location Pl,f , obtained by
  Equation(1)
• A in a leaf (return), an empty node (discard) or
  an internal node (continue to the next level
  (l1) )

15
Implementation Details

• To avoid precision issues, and minimize storage
  requirements we store S?Pl,i instead of Pl,i in
  the indirection cells
• S?Pl,i is an integer
• Optimized the encoding choose the number of bits
  to stored in the RGB channels to exactly
  represent at most S indices
• The values of the pointers do not depend on the
  size of the indirection pool

16
Precision Limitations

• The limited precision of 32 bits floating point
• Limit on the depth, resolution, number of nodes
  of the hierarchical grid

17
Managing the Texture Sprites

• Each sprite is associated with
• various parameters
• a bounding volume V defining the spatial limit of
  the sprite influence on the composite texture
• Bounding volume V is used to
• determine which cells are covered by sprite
• detect whether two sprites overlap

18
Storing Sprite Parameters

• Sprite parameter table is a texture containing
  all the parameters for all the sprites
• Each sprite corresponds to an index into this
  texture each texel encodes a parameter set
• Store the index of the sprite parameters in the
  tree leaf data field
• The RGB value encodes a pointer to the sprite
  parameter table

19
Dealing with Overlapping Sprites

• Leaf vectors table
• This 3D texture implements a table of vectors
  storing the sprite parameters associated with a
  leaf
• 2 dimensions the vector
• 1 dimension to index thevector of sprite
  parameters
• Instead of directly storing the index of one
  sprite in the leaves, we now store an index to
  the leaf vector

20
Adding a Sprite to the Tree

• Leaf splitting occurs only in full leaves (Omax)
• Cmax lt Omax
• Insertion failures
• The maximum depth level is reached only when more
  than Omax sprites are crowded in the same very
  small region
• Ordering of sprites
• Order the sprites in the leaf vectors

21
Positioning the Sprites on the Surface

• Associate a frame to each sprite and use a
  parallel projection to surface
• The positioning parameters
• A center point c (sprite handle)
• Two vectors l, h
• defining the plane, scale, orientation of the
  pattern
• A normal vector n (direction of projection)
• M (l, h, n)
• U M-1?(P-c)
• U (u, v, w) texture coordinates within the
  pattern

22
Positioning the Sprites on the Surface

• Deformation-proof texturing
• Store a 3D parameterization (u, v, w) at each
  vertex and interpolated as usual for any fragment
• Blending sprites
• When multiple sprite overlap, the resulting color
  is computed by blending together their
  contributions
• Non standard blending modes
• Blobby-painting cellular textures

23
Filtering

• Linear interpolation (for close viewpoint)
• standard texture unites of the GPU
• MIP-mapping of sprites
• standard texture unites of the GPU

24
Applications and Result

• Texture authoring
• Interactively scaled, rotated, or moved above or
  below the already existing sprites
• Lapped texture approximation
• The sample is stored only once
• N3-tree minimizes the memory space required for
  positioning information
• Rendering does not suffer from filtering issues
  created by atlases

25
Applications and Result

• Animated sprites
• Modify the positioning parameters (position,
  orientation, scaling)
• A sprite can cycle over a set of texture
  patterns, simulating an animation in a
  cartoon-like fashion
• Snake scales
• Estimate the local geometric distortion and scale
  the sprites accordingly to compensate for the
  deformation of the animated mesh

26
Performance
27
Storage Requirements
28
Conclusions

• High texturing resolution at low memory cost
• Without need for a global planar parameterization
• The sprites attributes are efficiently stored in
  a hierarchical grid surrounding the objects
  surface
• Flexible
• Each sprite can be independently animated, moved,
  scaled, rotated
• Create new texturing effects, such as animated
  texture, textures reacting to mesh deformations
• Complete GPU implementation, real-time performance

29
Future Work

• Painterly rendering
• Sprite projector functions
• For point-based and volumetric representation