Reflectance Imaging: A Simple Approach to Capturing Surface Detail

1
Reflectance Imaging A Simple Approach
to Capturing Surface Detail
Tom Malzbender
Hewlett-Packard Laboratories
2
Texture Mapping Catmull 74
Bump Mapping Blinn 78

• Pro
• Lighting
• Variations

• Pro
• Photographic
• input
• Simplicity
• Hardware Support

• Con
• Per-pixel lighting
• computation
• Filtering is Problematic
• Procedural Synthesis
• (Not image-based.)
• Rushmeier 97

• Con
• Unrealistic Silhouettes
• Static Lighting

3
PTM Demonstration

• Advantages
• Image based unlike bump mapping
• Simpler to evaluate than bump mapping
• Can leverage mip mapping

4
PTM Demonstration
Top Polynomial Texture Map
Bottom Conventional Texture Map

• Advantages
• Image based unlike Bump Mapping
• Simpler to evaluate than Bump Mapping
• Can leverage Mip Mapping

5
Acquiring PTMs Photographically

• Fixed object, fixed camera.
• Limited to Diffuse Objects.

Third acquisition device being fabricated
currently
6
Modeling Pixel Color Changes Directly
(u,v)
Light Direction Space
7
Modeling Pixel Color Changes Directly
(u,v)
Light Direction Space
8
Polynomial Texture Mapping
PTM Store RGB per pixel and Store polynomial
coefficients (a0-a5) per textel
R L R' G L G' B L B'

• Why Polynomials?
• Compact Representation
• Consist solely of multiplies and adds.
• Cheap to evaluate on both modern CPUs and VLSI

9
Light Direction Parametrization
u,v - texture coordinates a0-a5 - fitted
coefficients stored in texture map lu,lv -
projection of light direction into texture plane
lv
lu
lu,lv can be scan-converted without
normalization.
10
PTM Formats
Per Pixel Storage
Format

• LRGB -
• RGB -
• ENC - Index to L.U.T. storing Polynomial
  Coefficients
• ...

R,G,B
11
2D Applications

• Enhancement of Cuneiform
• Tablets w/ Zuckerman USC
• PTMs for Short Image Sequences
• PTMs for Depth of Focus Effects.

12
Surface Normal Extraction
Yields maximum surface brightness
13
Surface Normal Extraction
For a diffuse object, coordinates of (lu,lv) that
maximize luminance yield local surface normals.
Setting
yields
Providing a surface normal per textel
14
Specular Enhancement
Cuneiform tablet courtesy of Dr. Bruce Zuckerman
at U.S.C.
15
Diffuse Gain - a reflection transformation that

• Keeps the surface normal fixed.
• Increases the second derivative
• of the reflectance function by g.

16
Light Direction Extrapolation

• Input images are collected across a
• hemisphere of light directions, i.e. -1lt
  lu,lv lt1
• PTMs can be evaluated outside of the
  hemisphere,
• ( lu,lv lt -1 or lu,lv gt 1 )

17
Forensic Applications
18
Java PTM viewer

• Written by Clifford Lyon, Harvard University
• Easy to add new datasets.
• http//materialobjects.com/browser/PTMBrowser_1.h
  tml

19
Forensic Applications
20
National Gallery London
Light Arm
Joe Padfield
21

• Basler A504kc 500 F/sec camera.
• Real-time transfer to HP workstation.
• Processing on GPU, Nvidia Gforce 7800 GTX.
• 60 f/sec throughput.
• Normal transforms (unsharp mask), Relighting.

Real Time Enhancement
22
PTM Object Movies - POM

• Viewpoint Variation
• Lighting Variation
• Surface Enhancements
• Currently lacking in view interpolation.

Collaboration w/ Alberto Proenca, Univ. Minho,
Portugal Cultural Heritage Imaging, California
Collection Archéologie du Musée de l'Hospice du
Grand St. Bernard
23
Highlight-based PTMs
Developed with Mark Mudge, Cultural Heritage
Imaging
V
N
L
f
z
Sx
1.0
x
24
Conclusions Web Tools

• Reflectance functions are easy to collect and
  model.
• Transforming reflectance functions can help
  visualization.
• Tools are available at hpl.hp.com/ptm
• sample PTMs
• PTM viewer
• Polynomial Fitter
• PTM format document