Compression and Real-time Rendering of Measured BTFs using local-PCA

1
Compression and Real-time Rendering of Measured
BTFs using local-PCA

• Mueller, Meseth, Klein
• Bonn University
• Computer Graphics Group

2
Motivation

• (Real-time) Rendering of complex meso-structure
• Shadowing
• Masking
• Light-Transport
• inter-reflections
• sub-surface scattering
• etc.
• Classical modeling andrendering approach
  infeasible

3
Motivation

• Common work around
• Meso-structure is rendered from one image
• Texture mapping
• Fast and simple
• Hardware support
• Flat appearance
• Only simple relighting

4
Motivation

• Improvement
• Meso-structure is rendered from many images
• Rendering measured BTF
• Bidirectional-Texture-Function (Dana et al. 1999)
• Light and view-dependent Texture
• Apparent BRDF that varies per texel
• Captures all light and view-dependent effects of
  a material

5
Problem

• Accurate samplings of the 6-dimensional BTF(x,
  y, wi, wr) contain many images
• e.g. Sattler et al.(Bonn University, 2003)
• 81 directions for light and view each
• 256x256 texel spatial extend
• 6561 RGB-images 1.2GB
• Interpolation from sampled data impossible in
  real-time
• Memory reduction required

http//btf.cs.uni-bonn.de/
6
Previous Work

• McAllister et al. (2002)
• Fitting analytical BRDF-model (generalized cosine
  lobes - Lafortune) to every texel
• Fast rendering and high compression (1500)
• Limited quality (depth impression!)
• Non-linear fitting required (expensive, lt5 lobes)

McAllister 2002
7
Previous Work

• Daubert et al. (2001)
• Fitting Lafortune to synthetic cloth-BTFs
• Including view-dependent scaling factor
• Increased depth-impression and moderate memory
  requirements (140)
• Modeling abilities still limited

wr
Daubert et al. 2001
wi
Lafortune 2 lobes
scale factor 2 lobes
Lafortune 10 lobes
scale factor 10 lobes
white plaster
8
Previous Work

• Meseth et al. (2003)
• Fitting of analytical functions (polynomials,
  lobes) for fixed measured view-direction
  (reflectance fields)
• Rendering employs view-interpolation
• Masking and shadowing captured
• High memory requirements (115)
• Artificiality of the fitted analytical functions
  still notable

Meseth et al. 2003
9
Previous Work

• Suykens et al. (2003)
• Application of an improved BRDF-factorization
  technique (Chained-Matrix-Factorization, CMF)
• Clustering of resulting factors leads to compact
  representation
• Fast implementation on current graphics hardware
• Tested samples not representative for real
  measured BTFs

Suykens et al. 2003
10
Previous Work

• Sattler et al. (2003)
• Perform PCA on images with fixed view direction
• Combine the resulting Eigen-Textures during
  rendering
• High quality
• Environmental lighting supported
• High memory requirements (110)
• Real-time only for small meshes(CPU-operations
  per vertex)

Sattler et al. 2003
11
Our Approach

• Interpret the measured BTF as set of
  high-dimensional vectors (either images or per
  texel apparent BRDFs)
• Expect correlation between vectors
• Apply data analysis tools for dimensionality
  reduction

wr
wi
reprojected images
per-texel apparent BRDF
12
Our Approach

• Generalize Sattler et al.
• Cluster data to subsets
• Apply Principal Component Analysis (PCA) to data
  in that subsets
• Piece-wise affine-linear approximation

affine-linear approximation
3 piece affine-linear approximation
13
Our Approach

• How should we cluster?
• Generalized Lloyd-algorithm
• Euclidean distance
• Reconstruction error Local-PCA (Kambhatla, Leen
  1997)

14
Analysis
images
BRDFs
no clustering
original
reconstruction (k32, c8)
c30,k1
inverted difference
cluster index map
Average reconstruction error (proposte, c8)

• BRDF-arrangement performs superior
• Represent BTF by sets of Eigen-BRDFs

15
Analysis - Comparison
1.2GB 32 MB 106 MB 60 MB 121 MB 10 MB
corduroy
16
Analysis - Results
raw data
compressed (k32, c8)
17
Analysis - Results
raw data
compressed (k32, c8)
18
Analysis - Results
raw data
compressed (k32, c8)
19
Real-Time Rendering

• Rendering equation for n point-light sources
• Evaluate on hardware

Eigen-BRDF (includes cosine factor)
closest measured light/view-directions
cluster look-up
20
Real-Time Rendering

• Straight-Forward GeForce 5900 FX implementation
• 15 Frames 800x600, P-IV 2.4GHz
• Arranging Eigen-BRDFs in parabolic-maps enables
  built-in view-interpolation
• Factor 3 speed-up

21
Demo
22
Extensions

• Mip-Mapping
• Assigning weights and cluster-indices to scaled
  versions of the BTF
• Environmental Lighting
• Extend Bi-Scale Radiance Transfer (Sloan et al.
  2003)
• Memory savings enable large BTFs
• Lighting integral (dot-product of Spherical
  Harmonics coefficients) could be pre-computed!

23
Preview
24
Conclusions

• Using local-PCA for BTF-compression
• exploits correlations in the materials structure
• especially suited for materials with low- and
  high-frequency content (high spatial resolution
  required)
• stable fitting algorithm
• high quality with affordable memory requirements
  and runtime cost
• implementation on current graphics hardware
• easily extendable and combinable with other
  techniques

25
Acknowledgements

• Funded by the European Union under the project
  RealReflect (www.realreflect.org)
• Funded by the BMBF under the project
  VirtualTry-On (www.virtual-try-on.de)
• HDR-Environments from www.debevec.org