Immersive Experiences for Team Training

1
Immersive Experiences for Team Training

• PROPOSAL FOR AN NSF SCIENCE AND TECHNOLOGY CENTER
• JUNE 2005-JUNE 2010-JUNE 2015
• 5.2 M/YEAR (including matching)
• UNC, UCF, Morehouse Medical College
• Will know March, 2005

2
Our Vision for Capability
3
The Goal

• Whole team in one space
• See, touch, talk to each other
• Personal synthetic stereo view of VE, with some
  real objects
• No encumbrances beyond sunglasses

4
Todays VEs for Team Training

• Superbly successful for vehicle drivers
• Everything reachable is real
• Everything virtual is out-the-window
• User is unencumbered
• Fine for groups sharing one viewpoint
• Pretty good for some solo training
• Quite poor for teams with individual viewpoints

5
Four Thrusts of the Center

• Training science
• University of Central Florida
• Radical immersive environments
• Collaboration with real trainers
• Morehouse College of Medicine
• UNC Trauma Surgery, Emerg Medicine
• U.S. Coast Guard Trg. Facil. Yorktown
• U-Drive-It Facility to change practice

6
Our Vision for Immersive Display

• Five-year
• CAVE-like
• but ceiling projectors
• 4 eyes, then 8 eyes with individual images
• Ten-year
• Full autostereoscopic for 8 eyes
• Lots of pixels on the walls
• Flys-eye lenses

7
Our Vision for Image Generation

• Five-year
• PC cluster with graphics cards
• Ten-year
• New light-field rendering architecture
• 4-D frame buffer all views from one rendering

8
Our Vision for Tracking

• Wide-area full-body tracking
• Tracking of tools as well as people
• Many approachesnet will be hybrid
• New theoretical approach for design
• Obviate head-tracking
• Acoustic spread-spectrum tracker
• Video tracking with sub-imposed skeletal models

9
Our Vision for Scenario Generation

• FAA System enhanced with branching
• Model acquisition by laser and video
• Scenario capture by computer vision
• First in UNC Emergency Room
• Next at USCG TF Yorktown
• Next in streets of Atlanta

10
Our Vision for Interaction

• Mixed real and virtual
• Objects
• People (autonomous agents)
• Collision detection
• Proper physical responses
• Novel interaction techniques
• Imperceptible redirectionwalking, hand
• Passive haptics (e.g., ledge floor in pit)

11
Redirected Walking Razzaque
Blue perceived path Red actual path
12
Basic VE Science Questions

• What is an effective VE?
• How can we measure effectiveness?
• What makes a VE effective?
• Can VEs effectively train small teams?

13
Our Vision for Basic Science of VEs

• Measure effectiveness of the illusions
• Physiological measures of stress
• Behavioral measures
• Subjective measures
• Training and training transfer measures

14
Physiological Measures Stress
15
Some ResultsHigh-Order Bits

• We can measure how scared you get
• Really walking really matters
• Latency really matters
• Frame-rate matters up to 30 fps
• Global illumination gt Local
• 3-D sound seems no better than 2-D
• Touching something gtgt touching nothing