Controlling and Configuring Large UAV Teams

1
Controlling and Configuring Large UAV Teams

• Paul Scerri, Yang Xu, Jumpol Polvichai, Katia
  Sycara and Mike Lewis
• Carnegie Mellon University and University of
  Pittsburgh

2
Context

• Aim to build large heterogeneous teams for
  complex tasks
• Robots, agents, people
• 10,000s of actors
• Multiagent version of Belief Desire Intention
  approach to autonomous behavior
• Builds on key abstraction of a Team Oriented Plan
• Defines the activities that must take place and
  interactions between those activities
• Supported by extensive theoretical (logical) work
• Key algorithms are NP-complete or worse
• Heuristics required for scalability

3
Specific Target Application Wide Area Search
Munitions (WASMs)

• Part munition, part unmanned aerial vehicle
• Single use
• Variety of sensors
• Limited fuel supply, approximately 30 minutes
• Communicate with each other, manned aircraft
• Concept of Operations (under development)
• Small number of manned aircraft
• Potentially other ground forces
• 100s of WASMs, performing a variety of missions
• Attack
• Search
• Battle damage assessment
• Decoys
• Communication relays
• Flight test planned September, 2005
• 1 real and 3 simulated WASMs

4
Token-Based Coordination

• Token self contained packet capable of being
  sent around team
• Information content
• Control content
• Local models
• Team members use receipt of tokens to create
  local models of other team members
• What sorts of things are they/are they not
  working on?
• What sorts of things might they need to know?
• Local models are used to improve the routing of
  future tokens
• Token flows implement coordination
• No brittle, non-scalable message protocols

5
Token Based Algorithms

• Plan instantiation
• Removal of duplicate plans
• Role allocation
• Information sharing
• Resource allocation
• Sensor fusion
• Recovering from faulty sensor readings

6
Token Based Algorithms

• Plan instantiation
• Removal of duplicate plans
• Role allocation
• Information sharing
• Resource allocation
• Sensor fusion
• Recovering from faulty sensor readings

Information about environment passed around in
tokens, when agent receives tokens matching plan
pre-conditions plan is initiated Very high
probability all applicable plans are
initiated Liao et al, 2004
7
Token Based Algorithms

• Plan instantiation
• Removal of duplicate plans
• Role allocation
• Information sharing
• Resource allocation
• Sensor fusion
• Recovering from faulty sensor readings

Information about initiated plans shared in
tokens Very high probability some agent gets to
find out about any duplicate plans Liao
et al, 2004
8
Token Based Algorithms

• Plan instantiation
• Removal of duplicate plans
• Role allocation
• Information sharing
• Resource allocation
• Sensor fusion
• Recovering from faulty sensor readings

Responsibility to perform a role encapsulated in
token Only team member with token can perform
role Team member must have capability gt
threshold to perform role Threshold calculated
from estimates of likely role allocation
outcome Okamoto, 2004
9
Token Based Algorithms

• Plan instantiation
• Removal of duplicate plans
• Role allocation
• Information sharing
• Resource allocation
• Sensor fusion
• Recovering from faulty sensor readings

Locally sensed information shared via token to
get information to team members performing role
effected by information Does not require sensing
agent to know who needs information Xu
et al, 2004
10
Token Based Algorithms

• Plan instantiation
• Removal of duplicate plans
• Role allocation
• Information sharing
• Resource allocation
• Sensor fusion
• Recovering from faulty sensor readings

Access to resource represented by token Only
team member with token can use resource Team
member must have need gt threshold to keep
resource Threshold changes dynamically as it
moves around the team, seeing resource
need
11
Token Based Algorithms

• Plan instantiation
• Removal of duplicate plans
• Role allocation
• Information sharing
• Resource allocation
• Sensor fusion
• Recovering from faulty sensor readings

Uncertain sensor readings encapsulated in tokens
and sent around the team Very high probability
that at least one team member gets related sensor
readings and fuses for higher confidence

12
Token Based Algorithms

• Plan instantiation
• Removal of duplicate plans
• Role allocation
• Information sharing
• Resource allocation
• Sensor fusion
• Recovering from faulty sensor readings

Assumptions used to make decisions are attached
to tokens resulting from that decision Very high
probability agents with contradictory information
see the assumptions and initiate checking
process
13
Layered Team Member Architecture

• Bottom Local Reasoning Team members local
  actions are restricted by the tokens they have
• E.g., without resource token, cannot use resource
• Middle Coordination Reasoning Movement of tokens
  around the team implements the coordination
• Flows of tokens
• Local models inform token routing
• Top Meta-reasoning Ensures token flows work
  effectively

14
Synergies Between Token Algorithms

• Overall performance depends on how well tokens
  are routed
• Team members use local models to improve routing
• Observation Execution of algorithms shares
  information that other algorithms can exploit
• E.g., if role for strike near Pittsburgh was
  allocated to WASM X, then air space resources
  around Pittsburgh likely of interest to WASM X
• Implementation Use all tokens to improve local
  models
• E.g., role tokens change local models, resource
  tokens move according to local models
• Result When all algorithms are working together
  the system actually performs better

15
Implementing Synergies - Example

• When receiving an role token from neighbor i
• Probability of sending information token to i
  changed proportionally to similarity between
  plan initiated token and information token
• Probability of sending resource token to i
  changed inverse proportionally to similarity
  between role and resource token

16
Meta-Reasoning for Token Flows

• Specific tokens are meta-reasoned about
• Identify tokens that are not behaving as expected
• Bring to human attention
• Examples
• Tokens that live too long
• Tokens that travel too much
• See Scerri et al, AIAA 2004
• Overall flows can be controlled to optimize for
  specific criteria
• By controlling the flows, we control how the
  coordination works

17
Neural Networks for Modeling and Controlling
Token Flows

• Use a simple input/output feed-forward neural
  networks to represent a team performance model
• Three-layer FFNN is capable of representing any
  arbitrary functions
• Extend to Dynamic Networks concept to cope with
  dynamic behaviors
• This kind of network enlarges the capacity to
  deal with non-deterministic problem
• Learn the model with genetic algorithms
  methodology
• Excellent for dealing with very huge and noisy
  training data set
• Based on around 1,000,000 simulation runs

18
Configuring Algorithms with NN

• Offline
• Change environment and algorithm parameters,
  observe expected performance
• Online
• Use NN in reverse to find parameter settings
  for specific optimization criteria
• As environment changes
• As requirements change
• Allows tradeoff between performance of all
  algorithms

19
Configuration Interface
20
Results Token Based Coordination
Messages
Total Reward
Fully Integrated
Random
21
Results Configuring Algorithms
22
Results Online Control
23
Machinetta Bringing it all Together

• Encapsulate token-based coordination approach
  into reusable software module
• Called a proxy
• Proxies provide domain independent coordination
  algorithms
• Do not provide domain specific communication and
  interface code
• Available in the public domain
• Machinetta used to demonstrate coordination of up
  to 500 distributed, heterogeneous team members in
  several distinct domains
• Demonstrates that token based coordination is
  feasible
• Biggest teams developed to date?

24
Conclusions and Future Work

• Token-based coordination as a feasible
  alternative paradigm for large teams
• Additional layer over token flows gives high
  levels of control
• Future Work
• Can we make more precise mathematical models?
• Markov chains?