John E' Stewart, David M' Johnson,

1
John E. Stewart, David M. Johnson, William R.
Howse,U.S. Army Research InstituteFort Rucker
Research Unit, ALjohn.e.stewart_at_us.army.mil
What has been will be again, what has been done
will be done again there is nothing new under
the sun. Ecclesiastes, 1.9.
Fidelity Requirements for Army Aviation Training
Devices Issues and Answers
2

• Background
• The ARI Ft. Rucker Research Unit (FRRU), has been
  conducting aviation simulation and training
  research for over two decades. Our research has
  addressed many issues that from time to time that
  emerge at the Directorate of Simulation (DOS) and
  Directorate of Training and Doctrine (DOTD) at
  Ft. Rucker.
• Some typical research efforts have addressed
  simulator fidelity (i.e., cue requirements,
  low-cost simulators and training devices, motion
  (which will never go away!), and most recently,
  instructional strategies for collective aviation
  training.
• One thing I have learned in my 24 year career as
  a research psychologist, is that it is unwise to
  discard any old reports, data, or other archival
  materials. In applied research, it seems that
  time is circular, and the questions answered
  today will be asked again!

3

• ARIs research is rediscovered whenever the
  training system at Ft. Rucker undergoes radical
  change. One example of this is Flight School XXI
  (FS XXI), a simulation-focused Initial Entry
  Rotary Wing training system, consisting of
  high-fidelity simulators representing all of the
  Armys training and operational helicopters.
  Early evaluations of FSXXI revealed the
  following
• Student pilots learned to hover at 5 ft AGL in
  the simulator, but hovered at 10 ft AGL in the
  aircraft.
• Some student pilots, and experienced instructor
  pilots (IPs), became nauseated in the simulators.
• One hr of training in the simulator was not
  equivalent to one hr in the aircraft.
• ARI was not surprised to hear these issues in
  fact, these were to be expected whenever virtual
  simulation was employed for flight training. ARI
  had done research addressing all of the above
  issues. For example, the psychophysical
  distances in synthetic environments are not the
  same as those in the real world a certain
  percentage of pilots will become nauseated in a
  simulator, especially more experienced ones
  Simulator-aircraft transfer effectiveness ratios
  (TER) of 1.00 are extremely rare a more typical
  TER range is approximately .40-.50.

4

• After FS XXI Future Aviation Simulation
  Strategies (FASS) Study Group (SG). Nov.
  2006-August 2007. Convened by Directorate of
  Simulation to address functional requirements for
  next generation simulation training systems.
  Most of these were hardware/software engineering
  issues ARI was asked to provide input pertaining
  to training issues, based upon its research
  experience, on requirements (visual display
  fidelity, motion, aero model) as well as
  instructional strategies.
• Most of the issues raised by the SG dealt with
  functional requirements for effective training,
  rather than effective instructional strategies.
  This emphasis revealed a strong belief, that the
  greater the degree of realism in a virtual
  environment, the more effective the training.
• ARI responded to issues which were of concern to
  the SG, on the basis of
• 1. knowledge founded upon existing research
  data
• 2. knowledge based upon research in progress
• 3. knowledge requiring additional empirical
  research
• 4. knowledge that is not available, so the
  answer is not feasible at the present time.
• A few issues that seemed to stand out from the
  rest are presented A forthcoming ARI Research
  Report (Stewart, Johnson Howse) will address
  many more issues.

5

• Issues
• Relationship between desired capability and
  fidelity. Benchmarks for simulator fidelity are
  not known. The training developer usually has no
  objective knowledge of the fidelity necessary for
  training specific tasks. There is no
  comprehensive body of scientific research data
  specifying cue requirements for training specific
  flight tasks at particular levels, for specific
  populations of trainees (novice, advanced,
  recurrent). Frequently, training developers opt
  for as much fidelity as they can afford, since
  these requirements are not spelled out in any
  concrete, scientific way.
• Potential negative-habit transfer (NHT) risk
  areas associated with not incorporating a level
  of required fidelity to overcome a capability
  gap. NHT is part of aviation training jargon, and
  confounds polarity of training effect (diminution
  of performance) with desirability of learned
  behaviors (increases in unwanted behaviors). True
  NHT is rare. Flight simulators nearly always
  produce some net degree of positive transfer.
  There is even positive transfer between low-cost
  PC-based simulators and the aircraft. Point It
  is not worthwhile worrying about a validated
  flight simulator produced by a credible vendor
  causing overall negative ToT.

6

• Fidelity requirements and pilot experience. This
  is an important research issue. IPs seem to be
  less positive about training effectiveness of
  simulators than student pilots who train in them.
  Alessi (2000) Noble (2002), attack the intuitive
  belief that more fidelity is always better. What
  little evidence exists, suggests that this is not
  true for the novice pilot, who is still making
  mistakes. Little behavioral research exists to
  back up these contentions, though they are
  logical extrapolations from empirical research.
  Why? High cost of simulator-aircraft ToT
  research.
• Fidelity requirements for unit collective
  training. For both individual and collective,
  this depends on the tasks to be trained. Crews
  performing collective tasks should know how to
  fly their aircraft and operate avionics, nav.,
  comm., and weapons systems. They would not need
  the same level of visual display fidelity as they
  would for practicing emergency procedures, but
  would need high fidelity systems for both voice
  and data, as well as high-resolution
  semi-automated forces (SAF) database. We must
  first decide what to train, then what we need.
• Common fidelity requirements for individual and
  collective training. For both collective and
  individual training, it has been demonstrated
  that the main issue is not simulator complexity,
  but training strategies. Stewart, Dohme,
  Nullmeyer (2002), showed that relatively low-cost
  simulators can be effective, if the right
  training techniques are employed.

7

• 6. Cost-effective collective training devices.
  A collective training system need not be as
  costly and complex as the Aviation Combined Arms
  Tactical Trainer (AVCATT), a reconfigurable
  trainer for Army helicopters. In the same way
  that many individual skills can be trained
  part-task, collective skills can be trained at
  platoon or squad (as opposed to battalion) level,
  in low-cost, networked simulators, with a common
  virtual environment serving as the venue for
  validation. Nor must all collective simulation
  be virtual. Constructive desktop/laptop devices
  can serve as a supplement, imparting cognitive
  skills necessary to develop shared mental models
  required for performance of collective tasks.
• 7. Evidence for effectiveness of networked
  devices. There is some evidence that networked
  training devices have been employed with
  beneficial results (e.g., Bell Crane, 1993
  Crane, Robbins, Bennett, 2001). AFRL sought to
  determine functional requirements and
  instructional strategies for Distributed Mission
  Training systems for four-ship elements, at the
  operational unit level. One unpublished ARI field
  experiment (Howse, ) showed that a battalion
  pretrained in the Army Aviation CAV-Sim virtual
  network, outperformed a comparison battalion
  during a live exercise at the National Training
  Center. One would expect that collective skills
  would transfer to live exercises, but more
  research is needed in this area.

8

• Habits vs. Mental Models. With increasing
  complexity of aircraft systems, and the advent of
  digital cockpit technology, the majority of tasks
  trained must be cognitive. It is no longer
  habit transfer but mental models and
  knowledge structures. In virtual simulation
  environments procedural tasks are cognitive, and
  what is being acquired is a mental model of
  procedures underlying the task. Many of these
  skills can be trained and sustained part-task, in
  dedicated part-task trainers. Desktop/laptop
  devices are also candidates. Obviously, it is
  hard to determine the relative effectiveness of
  these milieux in the absence of more behavioral
  research.
• Simulator motion requirements for
  individual/crew and collective tasks. This is one
  issue that ARI has addressed extensively in a
  Technical Report (McCauley, 2006), who found
  that, while there was a substantial body of
  evidence supporting the effectiveness of flight
  simulation for training, there is virtually no
  evidence supporting the effectiveness of motion
  platforms. Motion seems to enhance in-simulator
  learning, but does not transfer to the aircraft.
  There is substantial evidence, however, that
  pilots prefer motion to non-motion when flying
  the simulator. For collective training, motion
  may be even less important, since pilots are
  practicing cognitive rather than psychomotor
  skills. However, there are cheaper ways of
  inducing motion than a full motion platform
  (e.g., force cuing).
• Yes, but are these systems training effective?
  Virtual training, individual and collective, is
  not just about hardware. This could be due in
  part to the acquisition process. AVCATT can
  create multi-ship tactical missions, using a
  variety of scenarios. But what are the best ways
  for exploiting its capabilities? AVCATT is
  expensive, but its effectiveness has not been
  empirically demonstrated.

9

• Conclusion There is nothing new under the sun!
• Again we find the paradox the technological
  base of simulation increases at a rapid pace,
  while training technology has shown little change
  over the past 20 years. ARI-Ft. Rucker has dealt
  with many of the questions and issues presented
  here over the past decade hardly any broached by
  FSXXI and FASS were new to the research staff.
  Investment in hardware far outstrips investment
  in training research.
• Institutional culture still supports fidelity
  over training, even though the scientific
  literature has shown that proficiency-based
  training usually trumps fidelity. This is partly
  due to the perception of the simulator as an
  attempt to replicate the experience of flight in
  the aircraft. Flight hours are simply offloaded
  to the simulator.
• The persistence of these institutional
  assumptions will not enhance the effectiveness or
  more importantly, the efficiency of training.
  Even if digital technology could replicate the
  aircraft, the returns in training outcomes would
  be disappointing when balanced against the cost
  of such an investment. Considering the capital
  cost of simulators, it is time to start using
  them properly as training devices.
• One positive outcome of ARIs participation The
  FASS-SG final report called for additional
  research on training effectiveness to address
  these and other issues pinpointed by ARI.

10
References
Alessi, S. (2000). Simulation design for training
and assessment. In H.F. ONeil D.H. Andrews,
(Eds.), Aircrew Training and Assessment (pp.
197-222). Mahwah, N.J. Erlbaum. Bell, H. H.,
Crane, P. (1993). Training utility of multiship
air combat simulation. In G.W. Evans M.
Mollaghasemi (Eds.), 1993 Winter Simulation
Conference Proceedings, Los Angeles. Crane, P.,
Robbins, R., Bennett, W. (2001). Using
distributed mission training to augment flight
lead upgrade training. (AFRL-HF-AZ-TR-2000-011).
Mesa, AZ Air Force Research Laboratory
Warfighter Training Research Division. Howse,
W. R. (2000). Virtual to live transfer for Army
attack helicopter units. ARI Newsletter, 10 (3),
8-10. McCauley, M. E. (2006). Do Army helicopter
training simulators need motion bases? (Tec. Rep.
1176). Arlington, VA U.S. Army Research
Institute for the Behavioral and Social Sciences.
Noble, C. (2002). The relationship between
fidelity and learning in aviation training and
assessment. Journal of Air Transportation, 7,
33-54. Stewart. J.E., Dohme, J.A., Nullmeyer,
R.T., (2002). U.S. Army initial entry rotary wing
transfer of training research. International
Journal of Aviation Psychology, 12,
359-375. Stewart, J.E., Johnson, D. M., Howse,
W. R. (in preparation). Fidelity requirements for
U.S. Army training devices issues and answers.
ARI Research Report. Arlington, VA U.S. Army
Research Institute for the Behavioral and Social
Sciences