Interaction between Real and Virtual Humans: Playing Checkers

1
Interaction between Real and Virtual
HumansPlaying Checkers
R. Torre, S. Balcisoy P. Fua, M. Ponder, and D.
Thalmann  Computer Graphics Lab (LIG) Swiss Fed
eral Institute of Technology
Atul Nayak
2
Virtual Humans

• Guided
• Represent the user
• Autonomous
• Act on their own

3
Virtual Worlds

• Virtual humans live in synthetic worlds

4
So whats new?

• Make the virtual human interact with the user
• 2-way communication
• Real-time
• Non-invasive
• Vision-based recognition system

5
Playing checkers.
6
Real time tracking of 3D objects

• To track a 3D convex object (a cube, a checkers
  board) we need
  
• A camera
• A target
• The wire frame 3D model of the target
• A powerful computer

7
Model Based Tracking

• Create a mathematical model of the target
• World coordinates are a function of the state
  variables that control the objects shape
  
• An objective function is minimized to find the
  best fit with image data
  

8
3-D tracking algorithm

• Get a video image
• Process the image
• X(t1) X(t) V(t).dT
• X position V speed dT time between t
  and t1
  
• Minimization of the objective function

9
Detecting the checkers

• If the sum of the intensities of each pixel in
  the real image of a white square is less than a
  certain threshold value , it means that a piece
  is present on that square.

10
Virtual Human Director (VHD)

• A system used to produce real-time multimedia
  products and TV shows
  
• Focuses on 2 issues
• Fully integrated virtual humans with facial and
  body animation, and speech
  
• A straightforward user interface for designers
  and directors

11
The VHD interface

• Virtual human animation and interaction
  capabilities integrated into one environment
  
• Control multiple actors
• Provide tools to create any scenery involving
  realistic virtual humans

12
The Augmented Reality System

• Synchronize the rendering of synthetic scenes
  with the real camera output
  
• VHD animates the virtual player and updates the
  scene input from tracker and checker game
  controller
  

13
Image composition

• Chroma-keying black is the key color
• All real objects represented in black
• Tables virtual representation masks the virtual
  human
  
• Blend the real filmed scene and the synthetic 3D
  scene to get the mixed image
  
• Real player sees the mixed image on a large
  monitor
  

14
Comments

• Very realistic virtual humans will be widely used
  in film, games and interactive television
  
• Integrate VR with the Web create 3D scenes with
  multimedia objects
  
• Replace checkers board with any other calibration
  object
  
• No extra devices you can see yourself in the
  virtual world
  

15
More cool demos.
16
More cool demos.
17
References

• VHD A System for Directing Real-Time Virtual
  Actors, G. Sannier, S. Balcisoy, N.
  Magnenat-Thalmann, D. Thalmann, The Visual
  Computer, Springer, Vol.15, No 7/8, 1999,
  pp.320-329
• All images and movies in this presentation have
  been taken from the LIGs website URL
  http//ligwww.epfl.ch/

18
Comments?