Kai Virtanen, Janne Karelahti,

1
A Multistage Influence Diagram Game for
Maneuvering Decisions in Air Combat

• Kai Virtanen, Janne Karelahti,
• Tuomas Raivio, and Raimo P. Hämäläinen
• Systems Analysis Laboratory
• Helsinki University of Technology

2
Maneuvering decisions in one-on-one air combat
¼
Outcome depends on all the maneuvers of both
players Þ Dynamic game problem
Objective Find the best maneuvering sequences
with respect to the overall goals of a pilot! -
Preference model - Uncertainties - Behavior of
the adversary - Dynamic decision environment
3
Influence diagram (Howard, Matheson 1984)

• Directed acyclic graphs
• Describes the major factors of a decision problem
• Offers several possibilities for quantitative
  analysis

Time precedence
Informational arc
Alternatives available to DM
Decision
Random variables
Conditional arc
Chance
Probabilistic or functional dependence
Deterministic variables
Conditional arc
Deterministic
A utility function
Conditional arc
Utility
4
Influence diagram (continued)

• State of the world is described by attributes
• States are associated with
• Utility
• Probability
• Utility is a commensurable measure for goodness
  of attributes
• Results include probability distributions over
  utility
• Decisions based on utility distributions
• Information gathering and updating using Bayesian
  reasoning

5
Decision theoretical maneuvering models

• Single stage influence diagram (Virtanen et al.
  1999)
• Short-sighted decision making
• Multistage influence diagram (Virtanen et al.
  2004)
• Long-sighted decision making
• Preference optimal flight path against a given
  trajectory
• Single stage influence diagram game (Virtanen et
  al. 2003)
• Short-sighted decision making
• Components representing the behavior of the
  adversary
• New multistage influence diagram game model
• Long-sighted decision making
• Components representing the behavior of the
  adversary
• Solution with a moving horizon control approach

6
Influence diagram for a single maneuvering
decision
Adversary's Present State
Adversary's Maneuver
Adversarys State
Measurement
Combat State
Present Measurement
Present Combat State
Situation Evaluation
Present State
State
Maneuver
Present Threat Situation Assessment
Threat Situation Assessment
7
Multistage influence diagram air combat game
White
Black

• Goals of the players
• 1. Avoid being captured by the adversary
• 2. Capture the adversary
• Four possible outcomes
• Evolution of the players states described by a
  set of differential equations, a point mass model
• Evolution of the probabilities described by
  Bayes theorem

8
Graphical representation of the game
Blacks viewpoint
Combat state
White's viewpoint
stage t-1
stage t
9
Threat situation assessment

• Infers the threat situation from the viewpoint of
  a single player
• Discrete random variable, four outcomes
• Neutral
• Advantage
• Disadvantage
• Mutual disadvantage
• Probabilities are updated with Bayes theorem
• Pposterior( outcome combat state) 8
• Pprior( outcome ) X Plikelihood( combat state
  outcome )
• Each outcome leads to a specific goal described
  with a utility function

10
Moving horizon control approach
Players states at stage t
Truncated influence diagram game lasting stages
t, tDt,, tKDt
Dynamic programming
KDt length of the planning horizon
Game optimal control sequences over stages t,
tDt, , tKDt
ttDt

• Resulting game optimal controls
• the cumulative expected
• utility is maximized
• approximative feedback
• Nash equilibrium

Implement the controls of stage t
Players states at stage tDt
11
Numerical example

• Black initially pursuing White
• Whites aircraft more agile
• White wins
• Look-ahead strategies
• one-step, solid lines, payoffs White/Black
  1.21
• two-step, dashed lines, payoffs White/Black
  1.25

altitude, km
White
Black
y-range, km
x-range, km
12
Threat probability distributions
Black
White
Probability
Probability
time, sec.
time, sec.
13
Effects of the likelihood functions

• Threat probability rate of change defined by the
  likelihood functions
• Steep likelihood functions
• Evolution of threat probabilities is sensitive to
  certain changes in combat state gt Outcomes are
  distinguished sharply

Black, flat likelihoods
White, steep likelihoods
14
Conclusions

• The multistage influence diagram game
• Models preferences under uncertainty and multiple
  competing objectives in one-on-one air combat
• Takes into account
• Rational behavior of the adversary
• Dynamics of flight and decision making
• The moving horizon control approach
• Game optimal control sequences w.r.t. the
  preference model of the players
• Utilization
• Air combat simulators, a good computer guided
  aircraft
• Contributions to the existing air combat game
  formulations
• New way to treat uncertainties in air combat
  modeling
• Roles of the players are varied dynamically

15
References

• Howard, R.A., and Matheson, J.E., Influence
  Diagrams, The Principles and Applications of
  Decision Analysis, Vol. 2, edited by R.A. Howard
  and J.E. Matheson, Strategic Decision Group, Palo
  Alto, CA, 1984.
• Virtanen, K., Raivio, T., and Hämäläinen, R.P.,
  Decision Theoretical Approach to Pilot
  Simulation, Journal of Aircraft, Vol. 36, No. 4,
  1999.
• Virtanen, K., Raivio, T., and Hämäläinen, R.P.,
  Influence Diagram Modeling of Decision Making in
  a Dynamic Game Setting, Proceedings of the 1st
  Bayesian Modeling Applications Workshop of the
  19th Conference on Uncertainty in Artificial
  Intelligence, 2003.
• Virtanen, K., Raivio, T., and Hämäläinen, R.P.,
  Modeling Pilot's Sequential Maneuvering
  Decisions by a Multistage Influence Diagram,
  Journal of Guidance, Control, and Dynamics, Vol.
  27, No. 4, 2004.
• Kais dissertation available at
  www.sal.hut.fi/Personnel/Homepages/KaiV/thesis/