Interactive Editing and Modeling of Bidirectional Texture Functions

1
Interactive Editing and Modeling of Bidirectional
Texture Functions

• Frédo Durand
• MIT CSAIL

Solomon Boulos University of Utah
Jan Kautz University College London
2
Introduction

• Bidirectional Texture Functions (BTFs)
• A representation for complex materials Dana99
• Collection of textures under many different
  viewand light directions
• 6D data structure 2D spatial, 2D view, 2D light

view top light top
view top light 50º
view 45º light 50º
view 50º light 60º
3
Introduction

• Bidirectional Texture Functions
• Usually acquired from real materials, e.g.
  Sattler03
• Include real-world effects (scattering,
  occlusions, transparencies, parallax, )
• Very realistic

Lamp
Camera
Sample(on robotic arm)
4
Introduction

• Acquisition of Bidirectional Texture Functions
• Is very expensive (time, setup, )
• University of Bonn BTFs have
• 81 view x 81 light images (6561 images)
• Takes several hours to capture
• Only 10 publicly available

5
Introduction

• Acquisition of Bidirectional Texture Functions
• Is very expensive (time, setup, )
• University of Bonn BTFs have
• 81 view x 81 light images (6561 images)
• Takes several hours to capture
• Only 10 publicly available

Want better return on investment ?
Editing of BTFs Allows to create
variations from single BTF Create
BTF from flat texture
6
Introduction

• How to edit BTFs?

7
Introduction

• How to edit BTFs?
• Everything is baked in
• No explicit knowledge of
• BRDFs
• Geometry
• Transparencies , etc.
• Extraction of these properties is
• Difficult
• How to support editing of high-level properties

8
Related Work

• Most work focuses on
• BTF Acquisition Dana99,Sattler03,Koudelka03,
• BTF Rendering/Compression Sattler03,
  Meseth04,Vasilescu04,
• Very little on BTF editing
• Pasting BTFs onto surfaces Zhou05
• Self-similarity-based editing Dong05
• Structural editing of BTFs Mueller07
• BRDF editing Lawrence06, Colbert06

9
Our BTF Editing Approach

• Simple operators
• Can be applied selectively (with selections)
• Enables powerful editing BTFShop

VIDEO
10
Our BTF Editing Approach

• Philosophy
• Simple operators ? change material appearance
• Rely on simple aspects of material perception
• Contrast ? roughness Koenderink02
• Histograms ? roughness Ginneken99, Leung97
• Sharpness ? subsurface scattering Fleming04
• ? Enables simple operators to be effective

11
Our BTF Editing Operators

• Operators
• E.g., color manipulation, blur/sharpen
• Work on certain dimensions
• The simpler the better
• Use raw data, avoid big inverse problems
• Exploit shape information, when appropriate
• Constrained to user selections
• E.g., highlight pixels, directional range

12
Overview

• Shading
• Shadowing
• Geometry
• Global Effects
• Due to time only highlight one example each

13
Overview

• Shading
• Shadowing
• Geometry
• Global Effects

• Low-level operators
• Blur / sharpen
• Color manipulation
• Give direct control of simple material properties
• Shading roughness

14
Shading Changing Specularity

• Goal
• Specularity relates to frequencies in light-space
  Ramamoorthi01

blur/sharpen in light-space
Light-dependencefor given view and location
Light-dependencefor given view and location
15
Shading Changing SpecularityResults Part 1
(BTF computed from synthetic micro-geometry)
(BTF computed from synthetic micro-geometry)
16
Shading Changing SpecularityResults Part 2
(Measured BTF)
17
Shading Changing Roughness

• Goal modify roughness

?
18
Shading Changing Roughness

• Observation
• Distribution of light vs. dark pixels relates to
  roughness Pont and Koenderink 02
• Distribution can be changed with curve tool
• User-controlledcurve
• Remaps input to output intensities

19
Original
(BTF )
synthetic micro-geometry
20
Overview

• Shading
• Shadowing
• Geometry
• Global Effects

• More complex editing operations
• Requiring selections for effective modification

21
Shading Changing Albedos

• Goal
• Do not want to explicitly extract albedos
• Propagate differences/ratios instead (in HSV)

originals
original
propagate
automaticallypropagated
user-modified
22
Shading Changing Albedos

• Can also edit albedos

23
Shadow Removal

• Goal
• Challenges
• How to find shadow areas?
• How to fill them in automatically?

24
Shadow Removal Finding Shadow Areas

• User-assisted selection
• Fully-lit, frontal texture slice always
  shadow-free
• Check ratio of fully-lit frontal slice /
  each of light-varying
  frontal slices
• If (scaled) ratio lt user-threshold pixel in
  shadow

/

fully-lit, frontal
side-lit, frontal
ratio
thresholdapplied
25
Shadow Removal Finding Shadow Areas
/

/

26
Shadow Removal Finding Shadow Areas
Found Shadow Areas
27
Shadow Removal Automatically Filling in Shadows

• Two-stage process
• Fill in selected areas with data from fully-lit
  frontal texture (never contains shadows)
• Adjust average brightness and saturation to match
  surrounding area

28
Shadow Removal Automatically Filling in Shadows
Original
29
Shadow Removal Automatically Filling in Shadows
Selected Shadow Areas
30
Shadow Removal Automatically Filling in Shadows
Shadow Areas Filled In From Fully-Lit, Frontal
Texture
31
Shadow Removal Automatically Filling in Shadows
Brightness/Saturation Adjusted
32
Shadow Removal Results
Label
Shading is preserved
Shadows Removed
Original
33
Shadow Removal Results
Shadows Removed
Original
34
Shadow Removal

• Limitation
• BTF slices contain spurious lighting variation
• Difficult to perform accurate adjustment
• Input data needs to be consistent

35
Shadow Creation

• Easy to create new shadows
• Select new shadow areas
• Trace rays based on height field and light
  directions (heightfield user-defined, or
  extracted)
• Darken areas using curve tool

36
Shadow Creation
Original
37
Shadow Creation
Heightfield for Shadows
Shadows Added
38
Shadow Creation / Modification

• Easy to create new shadows
• Select new shadow areas
• Trace rays based on height field and light
  directions (user-defined, or extracted)
• Darken areas using curve tool
• Modification removal re-creation

39
Overview

• Shading
• Shadowing
• Geometry
• Global Effects

• Now
• Editing operator requiring inferred information

40
Geometry Parallax

• Parallax is common in complex materials
• Geometric point ? different (x,y)

41
Geometry Parallax

• Shows up in BTF slices
• Consequences
• Spatial editing such as painting needs to take it
  into account
• Change of geometric structure ? warp in texture
  space

Frontal Slice
Oblique Slice
Frontal Slice
Oblique Slice
Original Height
Increased Height
42
Parallax Spatial Editing

Extract Heightfield (Shape from Shadows)
Unwarp Texture Slices (Removes Parallax, Aligns
Texture Slices)
Spatial Editing
Warp Texture Slices (Reintroduce Parallax)
43
Parallax Spatial Editing Result
No Correction
Parallax Correction
44
Geometric Modification

• Parallax Warp can also be used to impose
  different geometric structure

Heightfield
45
Local Geometric Appearance

• Parallax Warp
• Does not modify underlying shading
• Bump mapping Blinn78 modify local view and
  light direction

Diffuse
Heightfield
Bump Mapped
46
Local Geometric Appearance

• Use same idea for BTFs
• Rotate lookup directions w.r.t. heightfield
• Lookup values with rotated directions
• Store values again as BTF

47
Local Geometric Appearance
Original BTF
Heightfield
48
Thickness

• Goal modify apparent thickness

49
Thickness
50
Thickness
51
Thickness
Stretch view directions down
52
Apparent Thickness

• Can also edit apparent thickness

Original (BTF synthetic micro-geometry)
53
Overview

• Shading
• Shadowing
• Geometry
• Global Effects

54
Global Effects

• Asperity Scattering Koenderink03
• Glow at grazing angles
• Simulate using angular selection and curve tool

Fuzz Reference (BTF synth. micro-geo.)
No Fuzz (BTF synth. micro-geo.)
55
Global Effects

• Subsurface Scattering
• E.g., increased brightness in concavities
• E.g., remove using selection and curve tool

Original
BTFShop
Reference(BTF synthetic micro-geo.)
56
Overview

• Shading
• Shadowing
• Geometry
• Global Effects
• More tools / selections see paper

Edit Albedos
Modify Shadows
Edit Thickness
Edit Subsur. Scat.
57
More Editing Results
58
More Results
VIDEO
59
Overview

• Shading
• Shadowing
• Geometry
• Global Effects
• More tools / selections see paper

60
Modeling of BTFs
Single Texture
BTF Created
61
Modeling of BTFs
Single Texture
BTF Created
62
Modeling of BTFs
Single Texture
Created BTF
Reference BTF
63
Modeling of BTFs
VIDEO
64
BTFShop System

• Multi-threaded, tile-based architecture with
  efficient out-of-core caching
• Visual feedback in 2s (touch 5 data)
• Complete BTF (1.6GB data) minutes

65
Discussion

• Approach works well for tweaking BTF appearance
• Shading, Shadowing, Geometry, and Global Effects
• Limitations
• Cannot turn bark into wool
• Resulting BTFs not physically correct
• Operators edit one effect at a time

66
Conclusions

• Simple operators and selections ? surprisingly
  effective
• Mostly work on raw 6D data
• Infer properties when necessary
• BTFShop out-of-core (6D dataset)
• BTF
• Modification
• Creation from single textures

67
Acknowledgements

• Pete Shirley
• Mary Williamson and Omari Dennis for PBRT tools
• Reviewers and referees
• University of Bonn and UCSD for BTF data
• NSF Career Award 0447561, Microsoft Research New
  Faculty Fellowship, and Sloan Fellowship
• University of Utah Brown Fellowship
• German Research Foundation Emmy-Noether
  Fellowship
• Code will be available at http//btfshop.sourcefor
  ge.net