Application of Motion Planning to HumanComputer Interaction

1
Application of Motion Planning to Human-Computer
Interaction

• Guest Speaker
• Tsai-Yen Li
• Computer Science Department
• National Chengchi University
• Taipei, Taiwan

CS26N Motion Planning for Robots, Digital
Actors, and other Moving Objects
2
Outline

• Introduction to 3D navigation
• Motion planning as assistance
• Extension to 3D full-body motion
• Third-person control
• Conclusions

3
Focus of HCI in This Talk

• Goal designing intelligent user interface for 3D
  navigation with available controls on a desktop
  computer
• Challenges in HCI for 3D display
• Realism (modeling, rendering, etc.)
• Interactivity (frame rate, smoothness)
• Naturalness (metaphor, easy-to-learn)
• Constraints (available devices, etc.)

4
Why 3D Navigation Is Difficult?

• Limited view
• typically 45-90
• Low frame rate
• needs gt 15 fps
• lacks precise control

5
Ideas

• Applying AI techniques such as motion planning to
  user interface design.
• Analogy real-time spelling checks in MS Word
• Incorporating planning in control loops
• Using spare cycles to assist navigation
• Characteristics
• Real-time
• Incremental
• Time-budgeted

6
Types of Navigation Control

• Out-of-Body

• First-Person

• Third-Person

1st
oob
3rd
MDK
Doom
Second Life
7
Outline

• Introduction to 3D navigation
• Motion planning as assistance
• Extension to 3D full-body motion
• Third-person control
• Conclusions

8
The Basic Path Planning Problem

• Workspace

9
Navigation Control with Mouse
2D Workspace
3D Scene
dp v dt dq w dt
q current viewpoint config q next viewpoint
config
10
Problem of Navigation Control
11
Defining the Planning Problem Predicting User
Intention
current viewpoint
obstacles
Possible Cases A No modification B Direct
modification C Indirect modification
Results 1. Trivial path A1, B1, C 2.
Non-trivial path 3. No path
12
Randomized Roadmap Method
Note projecting 3D paths onto 2D workspace.
13
Example of Navigation with the Help of Motion
Planning
14
Efficiency Comparison (with and without Planning)
average of ten experiments by different users
15
Outline

• Introduction to 3D navigation
• Motion planning as assistance
• Extension to 3D full-body motion
• Third-person control
• Conclusions

16
Out-of-Body Control

• Between first-person and third-person views
• Move the camera behind the body but remain
  attached

17
Challenge and Proposal

• Focus 3D upper body motion
• Challenge real-time motion planning for 3D
  avatar
• Approach
• Decoupled planning decomposing the upper body
  into manageable components.
• Budgeted planning planning for a window of a few
  steps only in every time frame.

18
Kinematics Model of Digital Actor

CAl (4)
CAr (4)
CW (2)

CR (3)
(1)
CLl (4)
CLr (4)
Total 139 DOF
19
Two-Level Planning
Upper-Body Motion Planning (CT-space)
CW, t CAl, t CAr, t
CAl (4)
CAr (4)
CW (2)
CR (3)
CR
Global Motion Planning (C-space) in 2D, can use
previous techniques
20
Objective Compliant Motion Planning
21
Upper-Body Motion Planning

• Given pelvis trajectory (function of time)
• Decoupled planning waist, then arms
• 3D collision detection is needed.
• Search space Configuration-Time space (CT-space)
  
• 3D for waist and 5D for each arm
• CT-Roadmap Roadmap in CT-space

22
Testing Scenarios
A
B
nlt7
nlt7
D
C
nlt5
nlt6
n size of keyframe queue with real-time
performance
23
Example of Real-time Control of Intelligent
Avatar (I)
19.6 FPS
24
Example of Real-time Control of Intelligent
Avatar (II)
14.2 FPS
25
Outline

• Introduction to 3D navigation
• Motion planning as assistance
• Extension to 3D full-body motion
• Third-person control
• Conclusions

26
Third-Person Avatar Control

• Planning full-body motions in real-time is still
  not feasible.
• Planning with a given library of versatile
  captured motions

27
Organizing Motion Clips in Motion Graph
Node Motion clip Arc Feasible transition
Picture from Sung et al. 2005
28
Planning Future Motions
Motion Graph
Future feasible motions
Motion Planner
Current state
29
Feasible Motion Tree (FMT)

• (Mi, Pj) motion i at position j.
• Root current, Others future.

30
Example of FMT
Nodes in FMT(Future Configurations)
Root of FMT (Current Configuration)
31
Example of Maintaining FMT
32
Maintaining Feasible Motion Tree

• Steps when within time budget
• Select the leaf node with the highest priority
• Explore children of this node
• Repeat
• Exploration strategies
• Breadth-first
• Command-matching
• Others

33
Experimental Results Navigation
34
Intelligent Character in a Shooting Game
35
Conclusions

• Motion planning is hard in general but
  appropriate assumptions and practical algorithms
  can render interesting results.
• Motion planning techniques can be used to design
  effective user interfaces for various types of
  navigation where interactivity is the key
  constraint.
• Some CPU cycles should be spend in planning to
  make UI more effective.

36
Thank You!

• http//imlab.cs.nccu.edu.tw
• Tsai-Yen Li
• li_at_nccu.edu.tw