Next Generation

1

• Next Generation
• By John OLooney, Ph.D.
• Carl Vinson Institute of Government
  System Information is
  available at http//iep.cviog.uga.edu
• Issue Child Protective Service (CPS) trainees
  and workers rarely get enough experience in
  practicing their role prior to confronting the
  challenges of a CPS case. New training technology
  that simulates the challenges of a case
  investigation and provides detailed feedback to
  trainees may help to improve case workers case
  analysis, judgment, and decision-making skills.
  
• Current Approaches The current approach to CPS
  training involves classroom work and a limited
  amount of role playing. Role playing exercises
  provide one of the best training challenges for
  CPS candidates (or other front-line staff whose
  every day decisions are difficult, important, and
  involve substantial discretion, judgment and
  interpretation of policy). Role playing
  represents a good training tool because it
  demands that the trainee engage in a holistic
  situation that mirrors multiple aspects of a CPS
  workers real-world role. It is much more
  demanding than your average multiple-choice,
  paper and pencil tests because role playing
  requires that the trainees think on their feet
  and be producers of questions and directors of an
  inquiry--rather than simply reacting to a series
  of book knowledge questions, the answers to which
  can often be guessed at from the question
  context. While traditional role playing
  exercises provide one of the best training
  challenges for CPS candidates, such simulations
  have not been used to their full potential
  because they are
• prohibitively expensive to conduct in a way that
  would provide trainees with role-playing-exercises
  -on-demand (i.e., there would need to be a large
  cadre of actors available to play the various
  roles). While classroom exercises of this type
  are good low-cost substitutes, these exercises
  rarely afford all the trainees sufficient
  opportunities to play the role of CPS
  investigator in multiple scenarios presenting a
  large variety of challenges and case nuances.
• difficult to conduct in a way that provides all
  trainees with a uniform experience or challenge.
• hard for trainers to thoroughly and consistently
  assess
• A New Approach Computer-based simulation holds
  some promise for overcoming many of the barriers
  to training new employees via effective
  role-playing-based exercises. While computers
  have long been used for text-based decision
  support, their use as means to provide trainees
  with a simulated experience has been limited by a
  number of technical and animation-production
  problems. While these barriers have by no means
  been entirely overcome, advances in computer
  science, artificial intelligence, speech
  recognition, and character animation make it
  possible to begin to build computer-based
  simulations that can, we believe, be effective
  training tools.
• We are working on a Child Protective Services
  Investigation Simulator as a prototype system for
  training all kind of workers who need to refine
  their judgment-making skills in contexts that
  more clearly match on-the-job challenges--particul
  arly when making the wrong decision on the job
  would have high cost or dangerous consequences.
  The simulator we are building will allow trainees
  to engage multiple computer characters in
  role-play situations. The trainee would play the
  role of the CPS investigator and would interview
  other characters such as teachers, parents,
  neighbors, relatives, psychologists, doctors,
  etc. Different characters would have different
  stories to tell, facts to convey, or lies to
  telljust as occurs in actual CPS investigations.

2

• Training Technology
• Features of the System
• The system enables a trainee to
• Pose questions and make statements to animated
  computer characters in natural language or
  minimally structured language and have customized
  answers returned based on the role, knowledge,
  truth-telling behavior, and mood of that
  character as well as on prior events.
• View action scenes involving multiple characters
  in a family interaction situation or scenes such
  as
• The parents getting the child to stop grapping at
  items in the grocery store
• The child refusing to go to school or do homework
• The parents fighting with each other
• Receive notifications of events (e.g., a new
  report of abuse or a sighting of a child
  maltreatment incident) either randomly, at a
  scheduled time, or as the consequence of a prior
  event, decision or action.
• React (by gathering more information, making
  decisions, asking for help, authority, or
  permission, or notifying others) to a variety of
  simulated events and scenarios.
• Have particular responses dependent on prior
  events (e.g., getting a truthful answer from the
  parent about whether she had instructed the child
  to visit the doctor only after the CPS worker has
  interviewed the doctor on that issue).

3

• Technologies Employed
• The system is designed to be entirely
  Internet-based. The core of the intelligence of
  the system consists of
• a set of relational databases (e.g., Access or
  SQL Server)
• connected to the Internet Information Server (Web
  Server)
• via a series of Active Server pages that contain
  the code for the system
• connecting with the databases through an ODBC
  connection.

Development Issues Text Retrieval versus
Question-Response Snippets Early efforts to
adapt existing text retrieval software for
possible use in the system did not prove
fruitful--in part because these technologies
would not provide the degree of control needed
for the proposed training system. Typically,
text retrieval systems assume that knowledge or
facts are one thing, drawn from one text, and
that it does not change. In contrast, in our
system we need to be able to have facts vary by
which character is speaking, by the mood of the
character, by what prior facts have been
revealed, by the subject matter being talked
about, and by the degree to which the character
has been prepared to reveal the knowledge they
possess. In short, our system needs to
incorporate a very detailed level of control over
small bits of information, and to match a
trainees query with different responses under
different conditions. Brute Force Matching versus
Ontology-based Much of the developmental work
involved testing alternative technological
approaches to the natural language processing
challenge. While there are many approaches in
this regard, most can be grouped into two broad
categories of 1) a syntactical/statistical or
brute force approach that typically uses word
positions and parts of speech to identify the
input and designate appropriate responses and 2)
a semantic/ontological approach, which attempts
to provide the appropriate responses based on a
deeper understanding of the meaning of the query.
Each approach has certain strengths and
weaknesses. Speed of Response In order to
attempt to recognize the thousands of ways that
people can ask what is essentially the same
question it can be useful to incorporate
components such as a sentence parser that would
determine the parts of speech of the users
input, and WordNet, a synonym producer. However
as more components are added to the system,
response time can be hampered. In ordinary text
retrieval operations this is not as much of an
issue, but in simulated interactions with
computer characters, the semblance of realism is
quickly undermined when response times exceed 2-3
seconds. Question Recognition Templates and
Control Characters A major difficulty with
natural language systems lies in getting the
system to recognize specific sentences as
essentially being an instance of an underlying
question. Our approach to this problem has been
to create a number of templates that enable a
non-programmer to rapidly develop a large set of
question-recognition structures that will account
for a large number of the ways that people might
possibly ask the same question. These templates
exist as programmed web forms that generate the
set of question-recognition structures from the
input of an author. The templates place the
words and appropriate synonyms into the various
structures according to the role that particular
are words play in the sentence. The language
recognition code is also programmed to recognize
a number of control characters that allow one to
broaden the match conditions or provide
alternative wordings on which to base a match.
For example the / character is used in between
synonyms, words between brackets must match
exactly in word order (e.g., exact word
ordering ), the represents one or more words,
etc.
Case Input (Flexible versus Core Case Facts)
Allowing control over the specification of each
response to a question and the conditions under
which a response is provided facilitates making
the simulation more life-like. At the same time,
however, this large measure of customize-ability
has a major disadvantage it requires a great
deal of time to specify the individual responses
for each character for each possible question
that the character might be able to answer. Even
with web-based authoring tools to simplify this
process to a point and click sequence, the
authoring of a case with 250 underlying case
facts and 5 characters could potentially require
5 or more days. Consequently, we have developed
an alternative way (using checkboxes) in which to
specify a set of core case facts (specifically,
whether the fact is true and whether it is
required that the trainee discover the fact in
order to make decisions). Testing and Adding
New Questions and Question Structures Natural
language-oriented simulations often need to be
grown from the ground up (i.e., feeding in new
matching routines that are based on the language
that end-users actually employ) as well as from
the top-down (by specifying the relationship
between the actor, events, facts, etc. in the
simulated world). We allow potential end-users
to test their language against the systems
language recognition capability and to choose
from a list of the candidate matches to their
input if a perfect match is not found. Once
chosen and submitted, the next time a user
provides the same input, the system will now
recognize it as being the same as the identified
best candidate question. Providing the Most
Appropriate and Customized Responses Language
recognition systems that employ control and
wildcard characters make it possible for any
given input by a user to potentially trigger more
than one matching response. Since the system
should logically only provide one response, we
have developed methods of scoring the possible
matches (to the underlying question recognition
templates) such that the match that is most
unique (e.g., that relies on fewer wildcard
matches) is the one that is used. Similarly,
because the system will allow one to specify
different (more or less customized responses) to
the same match (to an underlying question
recognition template), we have developed a
sorting algorithm that identifies and uses the
response associated with the highest number of
specified factors (e.g., a specific character
mood versus any mood or a specific prior event
versus any event, etc.). Hence, a response
that required and received a match on 3 of these
factors would be provided ahead of one that
required and received a match of 2 of these
factors. Question Asking Templates While the
question recognition templates for rapidly
creating sets of recognition structures can go a
long way toward creating a system that can
potentially recognize a much larger amount of
potential user input, it should be recognized
that the natural variation in the way human
beings express their thoughts is likely to at
some point to challenge the capabilities of the
language processing engine. As such, we are also
working on developing guidelines and templates
for user input that can be used whenever a user
is having trouble being understood by the system.