Hierarchical Planning in Game AI

1
Hierarchical Planning in Game AI

• Ke Xu
• 9/22/2004

2
Outline

• Hierarchical Planning in Dynamic Worlds (chapter
  3.5)
• Applying Hierarchical Planning
• Strategic Planning for UT Bots
• Multi-Tiered AI Framework (chapter 7.9)
• Explanation for Hierarchical Planning

3
Problems with the Reactive Approach

• Relies on the God
• Difficult to perform a sequence of actions
• Difficult to coordinate

4
Classical Planning

• Current States ? a plan ? Goal States
• Operator
• A precondition list
• An add list
• A delete list
• Limitation
• Assumes that the world does not change during the
  time gap between planning and execution

5
Hierarchical Planning

• Idea decompose high-level, abstract tasks into
  low-level, concrete tasks (actions)
• Strictly more expressive than operator
  representation
• More effective
• Hierarchical Task Networks (HTN)
• Tasks primitive (action) / non-primitive
  (compound tasks)
• Methods (task reduction schemas) expand or
  reduce non-primitive tasks
• Operators (no preconditions, only effects)
  reduce methods
• Critics remove conflicts to reduce backtracking

6
HTN Example

Explode the target
Plant the bomb
Get a better weapon
Secure the plant site
Approach to the plant site
Guard the bomb
Buy
Select a route
Plant
Pick
Kill enemies
Kill Pick
Cover teammates
7
The HTN Planning Procedure
Problem P

Primitive Only?
Resolvable c?
Y
Y
Resolve c
N
N
Return results
Return failure
Choose t
Choose m
Replace t
Resolve c
8
Re-planning in HTN

• Partial Re-planning
• Anytime
• Anywhere
• Repair rather than re-plan from sketch
• Propagate the effect of re-planning

• Example

Top-Level Task Kill Agent
T2 Attack Agent
T1 Pick-Up Weapon X
9
Planning Agent Cooperation

• Why Interaction and dependency of agent actions
• How
• Eliminating conflicts to localize plan effects
  to individual agents
• Coordinating plans to use synchronization actions

• Coordinating HP

crisper coordination
lower cost / more flexibility
Coordination levels
10
Outline

• Hierarchical Planning in Dynamic Worlds (chapter
  3.5)
• Applying Hierarchical Planning
• Strategic Planning for UT Bots
• Multi-Tiered AI Framework (chapter 7.9)
• Explanation for Hierarchical Planning

11
Strategic Planning for UT Bots

• UT Bots
• Event-driven
• Client-Server architecture
• Behavior control
• Java Bots
• Soar Bots
• Operator preconditions and effects
• Rules select/apply/compare/interrupt/terminate

• The Issues
• Individual reaction to dynamic environment
• Contribution to the winning strategy
• Single bot

12
HTN Representation of Strategies

• Method

• Operator

13
Built-In Preconditions and Effects

• Problems
• Expressions of precondition are difficult and
  time-consuming
• Effects require complex executions
• Solution
• Hard-coded evaluations and executions
• Each operator affects a single bot the
  coordination is reflected in the hierarchy but
  not in the specific actions

14
Strategy Change vs. Strategy Modification

• Conditions may change when the current strategy
  is been pursued
• Pre-define a threshold for strategies
• Re-planning
• No need to change all tasks in the plan

15
Outline

• Hierarchical Planning in Dynamic Worlds (chapter
  3.5)
• Applying Hierarchical Planning
• Strategic Planning for UT Bots
• Multi-Tiered AI Framework (chapter 7.9)
• Explanation for Hierarchical Planning

16
Multi-Tiered AI Framework

• The Intelligence Structure
• Strategic Intelligence (SI)
• Operational Intelligence (OI)
• Tactical Intelligence (TI)
• Individual Unit (IU)

SI makes general goals and plans
OI is concerned with implementing the general
orders from SI
TI Prepares the data for IU
17
Situational Projects

• SPs the basic messages for communications
  between different levels of AIs

OI1
18
The MTAIF Class Architecture
SI
package of functions and strategies

Unit Object
OI
TI
IU
19
Multiple SP Containers

• A concept of RTS load balancing
• Allow the processing of a SP and the subsequent
  action to occur over multiple frames
• A series of linked SP containers
• - pathfinding

20
Threads

• Some SPs require longer execution time (e.g.,
  massing troop or marching)
• Use threads for each linked SP container
• Need to update the data and SPs

21
Other Applications of MTAIF

• Classical turn-based and RTS games
• Sports games

22
Outline

• Hierarchical Planning in Dynamic Worlds (chapter
  3.5)
• Applying Hierarchical Planning
• Strategic Planning for UT Bots
• Multi-Tiered AI Framework (chapter 7.9)
• Explanation for Hierarchical Planning

23
Explanations for HP

• Role of Explanation to improve the game model
  which is implemented in HTN planning
• Allowing users to interrogate the behavior of a
  computer player
• Explaining what caused/lead to the current state
• Explaining the motivation for knowledge/reasoning
  refinement
• Preventing the learning of unrealistic/non-doctrin
  al behaviors

24
Representing Explanations
An explanation is a collection of methods
preconditions and preferences, tasks (compound
and primitive), and actions.
25
Representing Explanations (Cont.)
26
Explanation Types (for RTS)

• Strategy selection
• Course of action outcome
• Game model update
• Prediction

27
Stratagus/Magnant
28
References

• AI Game Programming Wisdom 2
• H. Munoz-Avila T. Fisher, Strategic Planning
  for Unreal Tournament Bots.
• H. Munoz-Avila D. Aha, On the Role of
  Explanation for Hierarchical Case-Based Planning
  in Real-Time Strategy Games.