Animating Chinese Landscape Paintings and Panoramas

1
Animating Chinese Landscape Paintings and
Panoramas

• By
• Nelson Siu-Hang CHU
• Department of Computer Science
• The Hong Kong University of Science and Technology

2
Outline

• Introduction
• Motivations
• Two IBMR Approaches
• Multi-Perspective Modeling
• Baseline Extrusion
• An Improved Graphical User Interface
• Conclusions

3
Introduction

• Image Based Modeling Rendering (IBMR)
• Pre-acquired images -gt novel views

4
Introduction (cont.)

• Image Based Modeling Rendering (IBMR)
• Advantages
• Realism with less computation
• Computation independent of scene complexity
• Disadvantage
• Number of images needed is large, in general
• Success in film industry
• e.g. The Matrix SoftImage Magazine Aug. 99

5
Introduction (cont.)

• Tour Into the Picture (TIP) HAA97
• Visually 3D animation from a single image
• Easy-to-use graphical user interface (GUI)
• Good results in relatively short time
• Works especially well for inputs of paintings

6
Motivation

• Impressive results using the TIP technique

TIP
By Hoi Ng 1982
7
Motivation (cont.)

• Chinese landscape paintings
• Horizontal or vertical scrolls

from Mustard Seed Garden Painting Manual,
revised version by Chao Hsun, 1887
8
Multiple Perspectives in Chinese Landscape
Paintings
Eye locations
9
Motivation (cont.)

• Fixed-point perspective
• As in photographs or western paintings
• At most a spherical panorama
• Space depicted is limited

• Moving-point perspective
• As in Chinese landscape painting or
  multi-perspective images e.g. in WFH97
• Space NOT limited

• Goal Model and animate multi-perspective images

10
Motivation (cont.)

• Panorama
• Compact representation
• Easy to produce
• Cost effective for virtual reality (VR)

11
Motivation (cont.)

• Panorama Viewer
• Camera pan, zoom, tilt only
• Fixed viewpoint we cant move
• VR techniques giving better VR experience exist,
  but too costly
• Special hardware
• Large input data

• So, its valuable if can enhance immersive effect
  of panorama without much increase in production
  and computational costs

12
Outline

• Introduction
• Motivations
• Two IBMR Approaches
• Multi-Perspective Modeling
• Baseline Extrusion
• An Improved Graphical User Interface
• Conclusions

13
The TIP Technique
14
First Approach Multi-Perspective Modeling (MPM)

• TIP is not applicable for multi-perspective
  images
• horizons at different levels
• multiple cavities

• Solution introduce local and global models
  switch them smoothly

15
MPM Global Modeling

• Global model

16
MPM Sub-scene Modeling

• Local model for a sub-scene

17
MPM System Overview
For each sub-scene
18
MPM Model Switching
Scene entry points
19
Panorama as Input


Sub-scene 2
Sub-scene 1
20
MPM Model Switching
21
Animation Results

• Painting Fishing village in mild snow
• Real scene HKUST Piazza

22
Outline

• Introduction
• Motivations
• Two IBMR Approaches
• Multi-Perspective Modeling
• Baseline Extrusion
• An Improved Graphical User Interface
• Conclusions

23
Second Approach Baseline Extrusion

• Targets at panoramic inputs
• Builds a unified model
• Assumptions
• A fixed viewpoint (panorama)
• A flat ground is present in the scene

24
Second Approach Baseline Extrusion

• Main Idea
• user draws baselines on perspective views of
  the input image to indicate positions and
  orientations of billboards
• Extrude the baselines to form billboards

25
Second Approach Baseline Extrusion

• Basically two types
• Indoor man-made structures
• Outdoor usually a sky is captured

26
Longitude and latitude coordinates

• Input image represented as a spherical surface,
  with polar coordinates (?, ?)

Radius1
?
o
?
27
Calculation of billboard positions from baseline
28
Sample structure inferred from a baseline
29
Baseline Extrusion

• Billboard heights will be determined by
• Scanning mask image, or
• Ceiling height

30
Specific model component Ceiling Plane

• Define the ceiling height interactively

31
Animation Result

• Real scene Chapel Quad

Input panorama
32
Outline

• Introduction
• Motivations
• Two IBMR Approaches
• Multi-Perspective Modeling
• Baseline Extrusion
• An Improved Graphical User Interface
• Conclusions

33
Graphical User Interface(GUI) Improvement
Perspective Grids

• In TIP, a good GUI is the key!

Spidery mesh
34
Deriving model from a spidery mesh







35
Drawing Perspective Grids
36
Perspective Grids User operations
37
Advantages of perspective grids

• Visual modeling
• Show a better sense of depth
• Intuitively estimate the shape of the model
• Tool for non-technical users knowledge of
  field-of-view or focal length not required!
• Draw the grids using 2D geometry
• Real-time feedback guaranteed

38
Conclusions

• Proposed two approaches for producing animations
  
• From a single image with multiple perspectives
• From a panorama

• As extensions of TIP
• Tour into images of higher spatial complexity
• Produce longer animations to make it more
  entertaining

• As extensions of panorama viewing
• Produce animations of higher immersive effect
• Reasonable cost

39
Conclusions (cont.)

• Limitations of our IBMR approaches
• Navigation limited by geometry and texture
  information captured by the input image
• Multi-Perspective Modeling
• Some constrained nodes in animation path
• Not suitable for free navigation
• Baseline Extrusion
• Currently vertical billboards only
• Modeling inclined surfaces complex objects left
  as future work
• Proposed a minor GUI improvement

40
Questions?