Controlling Gestures on Avatars

1
Controlling Gestures on Avatars

• Francesca Barrientos
• with
• John Canny

2
Goals

• Develop a simple interface for controlling
  nonverbal expression (gesture) on avatar bodies
• Application most likely a collaborative work
  tool
• Audio
• 3D interactive graphics on regular screen
• Desktop input devices
• Multi-user, networked

3
Outline

• Motivation
• State of the art
• Issues
• Research snapshot
• Solicitation

4
Motivation

• Explore capabilities of virtual humans for
  communication purposes
• Hard to control a fully articulated human
• Rendering
• Network bandwidth
• Ease of use

5
Commercial virtual worlds

• Blaxxun

ComicChat
Traveler
Oz
6
Typical gesture interface

• Selecting expressions in The Palace

7
Example interface
Video from VRAIS97 Capin Pandzic Thalman
Thalman
8
Communication interface

• Thalmanns

9
Virtual tennis
Thalmanns
10
Autonomous behavior

• Viljamsson

11
Scripted/reactive behaviors

• Perlin

Random noise adds life
12
How our goals differ

• Continuous manual control
• Non-obtrusive input interface
• Avatar movement is an extension of humans
  movements
• Lifelike motion quality

13
Issues

• Understanding nonverbal behavior
• Design of controls
• Domain
• Evaluation

14
Functions of nonverbal behavior

• Psychological/Expressive
• Expression of emotion
• Construction of utterances
• Social/Communicative/Interactional
• Conversational regulators
• Approval, persuasion, listening
• Artistic
• Dance, theater, puppetry

15
Nonverbal communication types

• Proxemics
• Kinesics
• Context
• Facial expression
• Gaze
• Paralinguistic
• ...

16
Controls Interface

• Gesture designs
• Kinematic mappings from input device motion to
  avatar movements
• Interface to select among mappings
• Dynamics behavior for lifelike motion

17
Gesture
18
Mappings
19
Early prototypes (video)

• Simple avatars implemented in Impulse
• Networked application
• Inverse kinematics mapping
• Physical simulation
• Mouse input

20
Slice of research
21
Domain provides context

• Design discussions
• Architecture
• Industrial design
• Theater
• Storyline and characters provide context
• Other applications?

22
Evaluation

• Expressive emphasis means evaluating ease of use
  and expressive satisfaction
• Communicative emphasis means evaluating
  interaction
• Motion quality Realism and Believability

23
Summary

• Goals of this project
• State of the art in avatar gesturing
• Study of non-verbal behavior
• Technical issues in gesture control
• Future plans

24
Solicitations

• Descriptive title for project
• Ideas for domain
• Thoughts on evaluation
• Anything else
• to fbarr_at_cs