Cooperative Control of Distributed Autonomous Vehicles in Adversarial Environments

1
Cooperative Control of Distributed Autonomous
Vehicles in Adversarial Environments

• AFOSR 2001 MURI Kickoff
• Caltech/Cornell/MIT/UCLA
• May 14, 2001

2
Agenda
3
Vision Networks of (semi) Autonomous Vehicles

• Large scale operations
• Fault tolerance through redundant deployment
• Cost effectiveness through simple/specialized
  components
• Complex collective behavior through simple local
  behavior

4
Challenges

• Local information/decision making
• Constrained communications
• Large scale of operations
• Uncertain dynamic environment
• Hostile adversarial presence

5
Approach

• Multidisciplinary
• Multiscale Modeling Hierarchical Planning
  Logical Programming Environments Complexity
  Management Distributed Protocols Language
  Adaptation Biological Modeling
• Analytical Constructive
• Experimental
• Case Study Simulations Hybrid Hardware
  Realization

6
Hierarchical Formulation

• Fundamental challenge lack of centralized
  decisions
• Analytical difficulties
• Computational complexity
• Meaningful tractable?
• Relevance of hierarchy
• Aggregation, distributed computation, reduced
  communication, etc.
• Supporting social/biological evidence
• Self similarity

7
Research Focus

• Scalability, modeling reduction
• Representation of distributed low level
  components in a manner amenable to high level
  planning with reduced complexity.
• High level planning
• Development of analytical methods and
  computational algorithms for coordinated team
  strategies.
• Low level planning
• Realization of team strategies through low level
  strategies and optimization.
• Communications
• Investigation of communications issues within
  and among levels.

8
Case Studies

• Motivate Illustrate Research
• Multi-vehicle tasking with obstacle and mutual
  avoidance (one-sided)
• Autonomous suppression of enemy defenses
  (two-sided)

9
Experimental Testbeds

• Two hybrid facilities
• HiFi virtual vehicles with hardware
  communications
• Simple hardware vehicles with virtual
  communications/distribution
• Internal open-platform investigations

10
Expected Outcomes

• Theory Analytical understanding of achievable
  performance of distributed cooperative control
  systems.
• Computation Algorithms software tools for
  control design, testing, evaluation, and rapid
  prototyping.
• Experimentation Application to simulated and
  hardware testbeds.
• Education Multidisciplinary program with
  increased DoD visibility.

11
Expected Insights

• How to address scalability through modeling
  decomposition.
• How to address computational complexity in
  hierarchical designs.
• How to develop reliable multi-layered cooperative
  strategies.
• How to counter adversarial actions with
  constrained communications.
• How to integrate local optimizations for
  collective performance.
• How to synchronize cooperating elements through
  modeling and ID.
• How to exploit neurological models to design
  cooperating elements.
• How to achieve reliable communications in
  hierarchical structures.
• How to derive adaptive languages for autonomous
  operations.

12
Team Strengths People
Caltech Jason Hickey (CS) Richard Murray (CDS/ME) Cornell Raffaello DAndrea (MAE) Bart Selman (CS) Carla Gomes (CS)
MIT Munther Dahleh (EE) Eric Feron (AE) Steve Massaquoi (EE/Neuro) Brian Williams (AE) UCLA David Chichka (AE) Greg Pottie (EE) Jeff Shamma (MAE) Jason Speyer (MAE) Charles Taylor (Bio)
13
Team Strengths Experience
Caltech DARPA SEC DURIP Cornell RoboCup AFRL/Cornell Intelligent Information Systems Institute
MIT DARPA JFACC NSF Natural motor control ONR Distributed cooperative languages UCLA DARPA JFACC ONR Minuteman NSF Learning in natural artificial systems
14
Agenda
15
Project Management

• Monthly seminar series
• Twice annual research meetings
• Graduate student exchanges

16
Technical Approach

• Scalability, Modeling Reduction
• Representation of distributed low level
  components in a manner amenable to high level
  planning with reduced complexity.
• High level planning
• Low level planning
• Communications

17
Scalability, Modeling Reduction

• Objective Reduction of complexity for design.
• Example Coordinated flight
• Each vehicle represents control problem
• Want real time coordinated flight planning
• Answer Quantization?

18
Quantization

• Naïve suggestion Discretize state space?
• Issue Reduction not related to control problem.
• Alternative
• Presume individual aircraft controllers.
• Discretize via primitives of achievable
  maneuvers.

19
Robust Hybrid Automata

• Recent work at MIT for single vehicle
• Automata Nodes (state subset) connected by
  trajectories
• Hybrid Trajectories executed by dynamical system
• Robust Tolerance set for node transitions
• Reduction directly related to control problem.

20
Quantization Consequences

• Complexity reduction.
• Low level limitations captured in admissible high
  level node transitions.
• Node transitions can be human inspired.
• Can define primitive strings to continue
  hierarchy.
• Discrete state !

21
Higher Order Primitives

• Simple example String of primitives
• Need to derive coordinated primatives
• Will involve multiple RHA
• Needed for complementary vehicle packages, e.g.,
  maneuverability vs vulnerability
• Draw upon existing expertise
• Construct play sequences
• Control interpretation Feedforward

22
Encoding of Primitives

• Objective Capture high level intent
• Previous example Node sequences/Play sequences
• Alternative Parameters in the low level
  optimization

23
Encoding via Objective Functions

• Recent work at Caltech for real-time trajectory
  generation
• uto,toT arg min ? L(x,u) dt F(x(T),u(T))
• dx/dt f(x,u), g(x,u) lt 0
• Exploit structure of dynamics and warm restart
  to solve real-time.

24
Encoded Objectives, cont

• Can use penalty functions as encoded objectives
• Example Cooperative trajectory planning
• Node sequence terminal penalty sequence
• Objective function mutual avoidance
• Different penalty functions reflect different
  cooperative roles, e.g., lead vs middle vs trailer

25
Encoded Objectives, cont

• Recent work at MIT (DARPA/JFACC) for hierarchical
  encoding

Reduced order model AND Reduced order objective
26
Uncertain Adversarial Conditions

• Uncertain environment motivated robust hybrid
  automaton
• Hostile adversary implies non-deterministic
  outcomes
• Approach non-deterministic robust hybrid
  automata
• Examples
• Random outcome of battle engagement
• Set-valued enemy actions

27
Model Reduction with Adversaries

• Deterministic case Hierarchical consistency
• Random case Averaged consistency (e.g.,
  manufacturing systems)
• Adversarial case Cannot average
• Approach Encoding of low level adversarial
  encounters

28
Adversarial Encoding, cont

• Recent work at UCLA (DARPA/JFACC) for high-level
  adversarial modeling

• Actual engagment Sector-by-sector
• Model engagement Single sector
• Allowed game-theoretic constructions

29
Technical Approach

• Scalability, Modeling Reduction
• High level planning
• Development of analytical methods and
  computational algorithms for coordinated team
  strategies.
• Low level planning
• Communications

30
High Level Planning

• Hierarchical reduction likely leads to
  quantization.
• Increasing relevance of CS
• Combinatorial complexity management
• Logical programming environment
• Control question Find coarseness formulations
  that allow analytical insight.

31
High Level Planning, cont

• Conceptual approach is top-down or 2-way
• Opted for designed behavior vs emergent
  behavior
• Motivated by desire to meet specifications
  understand performance trade-offs.

32
High Level Complexity Management

• Dimensionality reduction via higher order
  primitives
• Recent work at MIT with randomized algorithms
• Random waypoint selection in configuration space
• Valuable in real-time obstacle avoidance
• Two-time scale complexity management
• Slow scale reinforcement learning/NDP
• Fast scale randomized algorithms

33
Fast/Slow Scale Issues

• Computational obstacles persist despite
  dimensionality reduction
• Recent work at Cornell for complexity
• Critical problem formulation
• Algorithm portfolios

34
Phase Transitions and Randomization

• Identify critical variable/constraint ratio
• Abrupt transition of practical complexity
• Understanding phase transition guides problem
  formulations
• Can approach sub-critical problem envelope
• Can bring in randomized algorithm portfolio

35
Logical Programming Environments

• Design questions become logical issues in
  discrete high level models
• Utility of logical programming environment
• Automated checking of design
• Assistance in exploratory design
• Verifiable re-use of existing design

36
LPE, cont

• High level planning can decompose into
• Scheduling tasks
• Executing tasks
• Example
• Formation
• Avoidance
• Regroup
• Abort

37
LPE, cont

• Design question Under what scenarios will
  vehicle collection successfully avoid obstacle?
• Formulate as logical question based on subtask
  and subgoals.
• Leads to multi-layered questions (within high
  level).
• Will benefit from complexity management research.

38
Team Strategies in Adversarial Environments

• Main team issue Available information
• Major analytical difficulties with simple
  deviation from centralized info
• Example Linear decentralized control
• Limited results for one two sided team problems

39
Team Strategies, cont

• Approach Exploit quantization computation
• Impose coarseness until analytical solution
  structure emerges
• Use new insight to guide heuristic designs
• Example UCLA DARPA/JFACC efforts

40
Low Level Planning

• Scalability, Modeling Reduction
• High level planning
• Low level planning
• Realization of team strategies through low level
  strategies and optimization.
• Communications

41
Local Optimization vs Global Objectives

• Low level optimization need not be consistent
  with high level intent
• Subtle issue Same team becomes competitors
• Examples Stadium viewing, collective feeding,
  voting mechanisms, prisoners dilemma

42
Conscientious Local Optimization

• Recent work at UCLA considers self-restraining
  local optimization
• Form model of interactions with team
• Optimize based on interactive model
• Self-impose conforming to model
• Leads to handshaking of model achievable
  performance
• Interactive models both stochastic deterministic

43
Uncertain Adversarial Conditions

• Traditional receding horizon dilemma
• Short horizon for uncertainty (poor prediction)
• Long horizon for stability
• Aggravated by existence of adversary
• Approach Encode uncertainty/adversary in penalty
  functions

44
Cooperative Identification

• Local execution requires synchronized mode of
  operation
• Example High order primitive of trajectory
  sequence
• Approach Recent work at MIT (DARPA/JFACC) on
  intent ID applied to team members
• Communications viewpoint Constrained consensus

45
Natural Cooperative Architectures

• Recent work at MIT on control architecture models
  of low level brain features
• Direct How higher order primitives are
  assembled, recalled, executed.
• Indirect How internal models are resolved for
  prediction of future environmental interactions.

46
Communications

• What vs how to communicate?
• Priorities
• Quantity
• QoS requirements
• Approach Combine distributed mobile networks
  with controls communication requirements for new
  reliable protocols.

47
Adaptive Languages

• Controls/Communication intersection
• Pre-programmed vehicles with local information
• Collective behavior determined by communicated
  information
• Control decision communications decision
• Approach Adaptive languages/protocols as new
  perspective in highly autonomous operations

48
Agenda

• Scalability, modeling reduction
• Representation of distributed low level
  components in a manner amenable to high level
  planning with reduced complexity.
• High level planning
• Development of analytical methods and
  computational algorithms for coordinated team
  strategies.
• Low level planning
• Realization of team strategies through low level
  strategies and optimization.
• Communications
• Investigation of communications issues within
  and among levels.

49
Agenda