VR for Complex Systems

1
VR for Complex Systems

• A Firemans Application

2
Training the chief fireman

• Chief of the firemen intervention squad
• Apply and supervise the protocol of intervention
• Coordinate his squads intervention
• Check for his crews safety
• Have a general view of the situation (collect
  information)
• Communicate within some minutes to the control
  centre

3
Training Used at the Moment

• PEN PAPER SIMULATION
• Low level of realism
• No interaction
• Easy and economic

• REAL-SCALE SIMULATION
• High level of realism
• Resource demanding

4
Hotel Ariane on Fire

• You the chief fireman
• You have to direct the rescue strategy
• Others hotel manager, guests, victims, other
  firemen.

5
Mission Rehearsal Exercise

• You US-army LT
• You have to provide reinforcements to another
  squad..but..
• Others SGT data, mother, injured child, medical
  attendent, crowd, soldiers
• You can interact only with SGT data

6
A.I. based approach

• Must implement a virtual brain and perceptive
  skill for every character
• Lack of a reliable cognitive model
• Poor conversational skills
• Slow reactions limited by computational power

7
Substituting agents with humans

• We dont need
• Natural language generation and interpretation
• High level implementation of human cognitive
  system

• We gain
• Contextual intelligence
• Quality of conversation

8
The Mixed Agent approach

• How to maximize avatars expressiveness and
  belivability.
• How to maximize the integration of the user in
  the system.

• Use more natural interfaces (dataglove image
  processing physiological captors)
• Create autonomous actions and fuse them with
  users command actions

9
Integrating gestures

• Which level of granularity and how to animate the
  characters?
• Which gestures to reproduce?
• How to identify interaction rules?

10
The importance of expressiveness

• Increasing emotional impact
• Increasing realism

11
How to perform gestures


12
Identifying the gesture

• Video analysis to
• Identify and categorize different gestures
• Identify different ways to perform the same
  gesture according to body posture.

13
Apply basic interaction rules

• Identification of interaction rules from videos

14

• Decomposing the interaction

A raises his hand to express the intention to
communicate with B
A looks somewhere else while keeping the hand
raised to remind B that he still wants to
communicate B looks at A waiting to
communicate but still walks in the other direction
B makes explicit with a gesture his interest to
go in the other direction. A looks in the
direction indicated by B
15
Implementing the rules

• If person in field of view gt rotate head towards
  him
• If intention to communicate gt rotate body towards
  him

16
The importance of the interpretation
A raises his hand to express the intention to
communicate with B
A looks somewhere else while keeping the hand
raised to remind B that he still wants to
communicate B looks at A waiting to
communicate but still walks in the other direction
B makes explicit with a gesture his interest to
go in the other direction. A looks in the
direction indicated by B
17
Final thoughts

• Classical representational approach has
  limitations. Mixed agent approach may lead to
  richer realistic simulations

• Problems
• Where do we delimit which aspects are under the
  control of the agent and which aspects are under
  the control of the user.
• How to have the user unintentionally influence
  the avatar.
• How to fuse the virtual with the real world so
  that it is believable useful.
• How to avoid the feeling of loosing control over
  the avatar
• How how how ??

18
? ? ?