Target Selection

1
Target Selection

• CIS 488/588
• Bruce R. Maxim
• UM-Dearborn

2
Case Study - 1

• Important problem space attributes affecting
  target selection
• Distance form origin to target
• Distance between enemy position and target
• Distance between target and estimated enemy
  position
• Relative angle between projectile trajectory and
  velocity of target

3
(No Transcript)
4
Case Study - 2

• Important animat attributes affecting target
  selection
• Angular divergence from animat heading to target
  as well as the angular velocities involved
• Speed of travel and direction of movement affect
  ability to hit targets
• Should be possible to create a perceptron that
  can guess how much damage a projectile will
  inflict on the enemy and use this to assist in
  target detection

5
Design Rationale - 1

• A fully connected, feed-forward MLP module should
  do the job
• It would be nice for the animat to learn target
  selection on-line
• A good compromise would be to gather game data
  on-line, but actually is done off-line
• This allows the designer some control over
  features to emphasize and provide some guidance
  to the learning process

6
Design Rationale - 2

• Having lots of data is good, since it allows
  assessment of the impact of noise on a game
• Since no two fights are ever the same in a good
  game, noise can be expected to affect learning a
  great deal
• Might need to create statistical profile for the
  MLP to learn from

7
Module Design

• Initialization in XML
• ltlayer inputs4 units8/gt
• ltlayer units1/gt
• Interfaces (both incremental and batch)
• void Run(const vectorltfloatgt input,
• const vectorltfloatgt output)
• float Sample(const vectorltfloatgt input,
• const vectorltfloatgt output)
• void Randomize( )
• float Batch(const vectorltPatterngt inputs,
• const vectorltPatterngt outputs)

8
Data Structures

• Array of layers
• Each layer contains a set of neurons plus output
  array for layer
• Training data stored separately
• Activation function derivative also stored along
  with the gradient error for each neuron
• Layers also store weight deltas used to allow
  momentum in steepest descent

9
Simulation and Learning

• Simulation is set of nested loops processing the
  layers in turn
• Each loop is located in its own function (except
  for innermost loop)
• Learning algorithms perform forward simulation to
  determine derivatives and gradients
• Backpropagation of error is then used to adjust
  the neuron gradients

10
Algorithm Outline

• // Selects best target by predicting damage
• // Obstacles are checked before calling function
• function select_target
• repeat
• // propose target spot near enemy
• target position randvec( )
• // perceptron predicts hit probability
• value estimate_damage(target)
• until value gt threshold // stop if big enough
• return target

11
Tracking Rockets - 1

• AI must be able to answer the following
• Did projectile explode nearby?
• Was it another players projectile?
• Was there any damage from projectile?
• What conditions present when shot fired?
• To prevent misinterpretations each animat is only
  allowed to have one rocket in the air at a time

12
Tracking Rockets - 2

• Information gathering
• When rocket fired start tracking and remember
  initial conditions
• When sensor detects noise, see if it was caused
  by a rocket
• If rocket explosion detected set target to point
  of collision and expected time of collision to
  current time
• When collision time is past use data to train NN

13
Dealing with Noise

• Simulation data contains lots of noise
• Game situations have been simplified
• Features gathered from situations are limited
• Environment is unpredictable since it contains
  autonomous agents
• Gathering on-line data is real tough
• Better to gather lots of data from several
  animats and log it for later analysis and
  off-line training

14
Inputs and Outputs

• Chosen from distances and dot products using four
  points in space
• Player origin
• Enemy position
• Estimated position
• Chosen target
• Output is Boolean value indicating whether damage
  was inflicted or not

15
(No Transcript)
16
Training

• Bots are not given unlimited rockets
• They are expected to pick up a rocket launcher
  and forage for ammunition
• During testing rockets are freely available
• Animats without ammo serve as good targets any
  way
• Learning can be sped up by running quake in
  server mode without graphics

17
Splash

• Predicts enemy position and generates a random
  target around it
• A perceptron is used to evaluate the likelihood
  of success
• Rocket is fired if the chances of success are
  acceptable
• Training data is gathered when expected collision
  time is past

18
Evaluation - 1

• Noisy environment, average error rate 25
• Takes batch algorithms hundreds of training
  periods (epochs) to become competent
• The NN visibly improves the animats shooting
  abilities
• Perceptron often aims at floor near enemy tends
  to favor spots
• Close to estimate
• Close to enemy current position
• Close to rocket origin

19
Evaluation - 2

• Perceptron good at finding flaws in suggestions
  made by target generation mechanism
• Prevents kamikaze shots, by enforcing minimal
  distance to target
• Perceptron only improves target selection
  according to its experience
• Does not seem to learn stupid behaviors