Towards RealTime Texture Synthesis With the Jump Map

1
Towards Real-Time Texture Synthesis With the Jump
Map

• Steve Zelinka Michael Garland
• University of Illinois at Urbana-Champaign
• Thirteenth Eurographics Workshop on Rendering
  (2002)

2
Texture Synthesis
3
Neighbourhood-based

• Compare local causal neighbourhoods Efros and
  Leung (ICCV 99) Wei and Levoy (SIGGRAPH
  2000) Ashikhmin (I3D 2001)

Input
Output
4
Fast Texture Synthesis

• Goal Interactivity
• Want synthesis that is
• Fast
• Simple
• High quality

5
Patch-based Methods

• Copy patches of pixels rather than single pixels

6
Patch-based Methods

• Copy patches of pixels rather than single pixels
• Chaos Mosaic, Xu et al, 1997

7
Patch-based Methods

• Copy patches of pixels rather than single pixels
• Chaos Mosaic, Xu et al, 1997
• Patch-Based Sampling, Liang et al, 2002

8
Patch-based Methods

• Copy patches of pixels rather than single pixels
• Chaos Mosaic, Xu et al, 1997
• Patch-Based Sampling, Liang et al, 2002
• Image Quilting, Efros and Williams, 2001

9
Our Inspiration

• Video Textures
• Schodl et al, SIGGRAPH 2000
• Given a sample video, generate endless video
  without looping
• Generate links between similar frames
• Play video, randomly following links

..
10
Our Approach

• Divide task into two phases
• Analysis
• Once per input texture (need not be fast)
• Generates jump map
• Synthesis
• Uses jump map
• Fast enough for interactive applications

11
What is a Jump Map?

• Same size as input

12
What is a Jump Map?

• Same size as input
• Set of jumps per pixel

13
What is a Jump Map?

• Same size as input
• Set of jumps per pixel
• Jumps are weighted according to similarity
• Need not sum to 1

14
Jump Map Texture Synthesis

• Synthesis becomes a random walk
• Pixel-by-pixel, in scan-line order
• Select an already-synthesized neighbour N
• Select a destination D from Ns jumps
• Copy the pixel neighbouring D

15
Synthesis with Jump Maps
16
Synthesis With Jump Maps
17
Synthesis Order

• Synthesis order influences patch shapes
• Not likely to extend in directions where there
  arent already-synthesized neighbours

18
Synthesis Orders

• Serpentine
• Reverse direction at end of scan-line
• Better than scan-line, just as fast
• Hilbert curve
• Maximizes locality
• Much higher quality
• Adds some overhead

19
Synthesis Issues

• Artifacts may occur if a patch hits an input
  image boundary
• Modify probability of taking a jump
• Increase for jumps from input boundary
• Decrease for jumps to input boundary
• Blend patch boundaries

20
Texture Analysis

• Need best matches for each input pixel
• Pose as high-dimensional ANN problem

21
Multi-resolution Analysis

• Use image pyramid and multi-resolution
  neighbourhood vectors

• Smaller neighbourhood required
• Improves PCA reduction

22
Jump Map Diversity

• Undesirable repitition may occur if jumps cluster
  spatially
• L2 norm is particularly susceptible

23
Poisson Disc Sampling

• Find extra matches
• Iteratively accept matches satisfying Poisson
  disc criterion
• Include a Poisson disc at the source

1
24
Analysis Summary

• Need best matches for each input pixel
• Use multi-resolution neighbourhoods
• Pose as high-dimensional ANN problem
• Reduce dimension with PCA
• Filter matches with Poisson discs
• Normalize similarity values across the jump map

25
Results

• Current implementation 2.1 million
  pixels/second scan-line0.8 million pixels/second
  Hilbert
• Good quality on stochastic textures
• Not so good on structured textures
• TBD demo

26
Future Work

• Analysis phase
• Use perceptual metrics
• Clustering instead of ANN
• Synthesis phase
• Multi-resolution synthesis
• Output control mechanisms

27
Future Work

• Generalization to patches
• Reduce storage used
• Sample size required?

28
Contact Information

• Steve Zelinka
• zelinka_at_uiuc.edu
• Michael Garland
• garland_at_uiuc.edu
• http//graphics.cs.uiuc.edu/zelinka/jumpmaps