Future Immersive Training Environment FITE JCTD

1
Future Immersive Training Environment (FITE) JCTD

• Pete Muller
• Deputy TM, FITE JCTDPotomac Training
  Corporationmuller_at_potomactrainingcorp.com

2
Overview of FITE JCTD Goals

• Demonstrates, assesses and transitions
  capabilities to enable a more effective small
  unit training in an immersive and realistic
  environment that sets conditions for honing
  complex decision making
• Increases sensory threshold.. 1st firefight no
  worse than last simulation
• Builds trust and confidence within small units
• Tactically assessed in operational demonstrations
  by Warfighters
• Transitions deliverables to Title 10 programs
  and Warfighters
• Executed in conjunction with, and informed by,
  the USJFCOM National Program for Small Unit
  Excellence

3
Infantry Immersion Trainer
4
What is an ImmersiveVirtual Environment?

• Movies and TV
• Holodeck from Star Trek
• X Men Danger Room
• The Matrix
• Books
• Swinton- The Defence of Duffers Drift
• Gibson- Neuromancer
• Stephenson- Snow Crash

5
Virtual Environment Continuum

Immersion/Presence/Realism

• Virtual Environments

• Gaming/Movies

• Live

FITE JCTD Sweet Spot

• Looks and feels like a real vehicle or
  environment
• Easy for vehicle
• Hard for Infantry

Sight and Sound are great, but youre sitting in
front of a screen- Movement is not possible or
unnatural
Resource intensive and limited by safety
constraints
Mixed Reality (MR) Augmented Reality (AR) Virtual
Reality (VR)
MOUT Sites
Web based Massively Multi-Player Online Game
(MMOG)
6
Definitions

• Immersion the objective level of sensory
  fidelity provided by a VR system. It is a measure
  of how close the stimuli produced by a VR system
  are to the equivalent stimuli that we would
  perceive in the real world. (Slater, 2003
  Bowman, 2007)
• Presence the users subjective psychological
  response to a VR system (Bowman, 2007)
• Virtual Reality (VR) The environment is totally
• computer-generated and surrounds the user
• Mixed Reality (MR) Real objects mixed with
  computer
• or video-generated
• Augmented Reality (AR) Computer-generated images
  mixed
• with a real scene through a HMD or HUD

7
The Infantry Challenge

• There has been much research in Virtual
• Environments for Training and Education
• There is a now a strong demand to apply
• Virtual Environments to the Infantry,
• but it is a very hard problem because
• The interface to the virtual world is your body
• Individual weapon interface is critical
• Increased emphasis on non-kinetic action
• Infantry is physically demanding

8
Virtual Reality (VR)

• Avatar depiction of a person's body in virtual
  reality
• Track and render users full-body and rifle
• Teammates see each others Avatar move in real
  time
• Avatar provides primary means of manipulating
  virtual objects

Physical Set-Up - Cameras track markers -
Head-Mounted Display
View of Users Avatar - Real Time Interaction -
Look - Move - Shoot
9
Augmented Reality
Incorporate virtual images into the real world -
Informational, like a pilots Heads Up Display
(HUD) - Virtual, such as characters
10
Configuration/Operation Views
Spiral 1
Spiral 2
Individual Worn Virtual Reality
Individual Worn Augmented Reality
Facility Based Mixed Reality
Shared Training Environment
Setting Conditions Human Dimension Focused,
Sensory IntegratedOverload, Suspension of
disbelief, Modular and reconfigurable and
Infinitely repeatable
11
Spiral 1Individually Worn Virtual Reality

• OBJECTIVE
• Demonstrate Individually worn Virtual Reality for
  training infantry skills
• Individual, team, and leader cognitive skills
  decision making training
• MILITARY RELEVANCE/OPERATIONAL IMPACT
• Realistic training environment
• Deployable and reconfigurable
• Enhanced AAR for improved cognitive skills
• Supports Army Dismounted Soldier

• TECHNICAL APPROACH
• Integrate additional technologies into
  ExpeditionDI
• Improve VBS2 to support scenarios
• Integrate RealWorld into ExpeditionDI
• Spiral testing at FITE JCTD Lab
• ExpeditionDI at JFCOM for additional testing

SCHEDULE
12
Spiral 2Facility-Based Mixed Reality

• OBJECTIVE
• Demonstrate advanced technologies for training
  infantry squad skills in a dedicated facility
  (e.g., Infantry Immersion Trainer (IIT))
• Individual, team, and leader cognitive skills
  decision making training
• MILITARY RELEVANCE/OPERATIONAL IMPACT
• Replicate battlefield conditions, cues, and
  attributes
• Enhanced Live Training and AAR
• Transition to USMC Squad Immersive Training
  Environment (SITE)

• TECHNICAL APPROACH
• Build upon existing IIT Infrastructure
• Experiment with 3D technologies
• Integrate emerging Tech including TRASYS Advanced
  Language Training System (ALTS) and Automated
  Performance Evaluation and Lessons Learned
  (APELL)
• Use both RealWorld and VBS2
• Spiral testing at FITE JCTD Lab

SCHEDULE
13
Spiral 2Individually Worn Augmented Reality

• OBJECTIVE
• Demonstrate advanced technologies for training
  infantry squad skills in a dedicated facility
• Individual, team, and leader cognitive skills
  decision making training
• MILITARY RELEVANCE/OPERATIONAL IMPACT
• Train anywhere and improve facility based
  experience
• Repeatable and reconfigurable scenarios for Home
  Station Training

SCHEDULE

• TECHNICAL APPROACH
• Build upon previous ONR efforts
• Improve COTS See through HMD
• Sys Integrator provides CGI input (RealWorld)
• Spiral testing at FITE JCTD Lab

14
Future Needs
Artificial Intelligence Haptics Tracking HMDs
(lower cost, larger FOV, more rugged,
etc) Virtual Locomotion