NonPhotorealistic Virtual Environment

1
Non-Photorealistic Virtual Environment
CS780 ????
Allison W. Klein Wilmot Li Michael M.
Kazhdan Wagner T. Corrêa Adam Finkelstein Thomas
A. Funkhouser
2
Outline
1. Introduction 2. Motivation 3. Overview 4.
Related works 5. Image-Based Representation 6.
Non-Photorealistic Filtering 7. Runtime
walkthrough 8. Results 9. Conclusion
3
Introduction
A system for real-time NPR of virtual environments
Figure 1 A non-photorealistic virtual
environment
4
Motivation
Why NPR virtual environments? - can convey a
specific mood - can focus the viewers attention
on important information - can suggest
additional semantic information
5
Motivation
What are the challenges for real-time NPR of
virtual environments? - Interactivity - Visual
detail - Controlled stroke size -
Frame-to-frame coherence
6
Overview

• ? Basic Idea
• Image-based rendering of non-photorealistic
  imagery
• NPR texture construction without seams
• NPR mip-maps for constant sized strokes (Art Maps)

Figure 2 Overview of the system
7
Plenoptic Function
Ref McMillan, L., and Bishop, G. Plenoptic
modeling An image- based rendering system.
Computer Graphics (SIGGRAPH 95), 29-46.
8
Image Based Rendering
?If p is known, any view of the scene can be
reconstructed ?An approach to computer-generated
2D-views of natural 3D-scenes ?Approximation of
the plenoptic functions from a number of images
(Sample-based representation)
Avoiding complex 3D modeling and lighting
effects But it is not so easy
9
Related Works
McMillan, L., and Bishop, G. Plenoptic modeling
An image-based rendering system. Computer
Graphics (SIGGRAPH 95), 39-46.
Debevec, P.E., Taylor, C.J., and Malik, J.
Modeling and rendering architecture form
photographs A hybrid geometry- and image-based
approach. Computer Graphics (SIGGRAPH 96), 11-20.
Litwinowicz, P. Processing images and video for
an impressionist effect. Computer Graphics
(SIGGRAPH 97), 407-414.
10
Image-Based Representation
? Properties for interactive walk through -
Arbitrary viewpoints - Practical storage -
Efficient rendering ? Properties for
non-photorealistic imagery - Homeomorphic
reprojection - Predictable reprojection -
Filter flexibility
11
Image-Based Representation
Hybrid geometry- and image- based representation
with texture
Why texture? - compact form for the 5D
plenoptic function - implicit inter-object
occlusions - multi-scale prefiltering -
compression and paging - available rendering
hardware - homeomorphic and predictable mapping
to the view plane - simple and convenient
representation for NPR filtering
12
Image-Based Representation
(b) Capture photographs
(a) Build coarse 3D model
(c) Map photographs
(d) Compute coverage
Figure 3 Constructing textures from images
13
Camera Calibration
Def A process of determining the intrinsic
parameters and/or the extrinsic parameters of
camera.
1) Intrinsic parameters -f - kappa1 - Cx,
Cy - sx
2) Extrinsic parameters - Rx, Ry, Rz - Tx, Ty,
Tz
Ref Tsai, R.Y. A Versatile Camera Calibration
Technique for High Accuracy 3D Machine Vision
Metrology Using Off-the-Shelf TV Cameras and
Lenses. IEEE Journal of Robotics and Automation
3, 4 (Aug. 1987), 323-344
14
Beam Tracing
How to map the pixels of each photograph onto the
surfaces of a 3D model?
Figure 4 Part of GIBR system
Ref Funkhouser, T.A. A visibility algorithm for
hybrid geometry- and image- based modeling and
rendering. Computers and Graphics, Special Issue
on Visibility (1999).
15
Non Photorealistic Filtering
? Two major concerns (1) Avoiding visible
seams - how to apply NPR filters so as to
minimize resampling artifacts? (2) Controlling
the stroke size - how to allow stroke size to
vary appropriately for an image plane ?
16
Non Photorealistic Filtering
Figure 5 NPR photo textures
17
Non Photorealistic Filtering
Construct a single texture image for each
connected set of coplanar faces, and then we
execute the NPR filter on the whole texture as
one image
Figure 6 Group texture
Figure 7 NPR surface textures
18
Non Photorealistic Filtering
Apply strokes to each level of the mip-map,
knowing that it is suitable for projection to the
screen at a particular size. (Artmaps)
Figure 8 Art-maps
19
Non Photorealistic Filtering
(b)
(a)
Figure 9 (a) without art-maps (b) with art-maps
20
Non Photorealistic Filtering
Figure 10 Art-maps only
Figure 11 Rip-maps with art-maps
21
Non Photorealistic Filtering
Figure 12 (a) with rip-maps (b) varying strokes
(a)
(b)
22
Run time walkthrough
? Limitation is the capacity of graphics hardware
texture memory ? Use view frustum culling and
occlusion culling after breaking up large textures
Figure 9 Run time walkthrough with lines
Figure 13 Run time walkthrough with lines
23
Results
Figure 14 Museum, drybrush
24
Results
Figure 15 Museum, pastel
25
Results
Figure 16 Museum, van Gogh
26
Results
Figure 17 Building, watercolor
27
Conclusion

• ? Technical Contribution
• Constructing NPR textures that avoids seams
• Art-maps

• ? Future works
• Augmenting the scene with geometry-based elements
• View-dependent rendering
• Better stroke coherence