June 10, 2002

1
June 10, 2002
2
Adaptive Coordinated Control of Intelligent
Multi-Agent Teams

• Shankar Sastry, Ruzena Bajcsy, Laurent El Ghaoui,
  Mike Jordan, Jitendra Malik, Stuart Russell,
  Pravin Varaiya (Berkeley)
• Vijay Kumar, Kostas Danillidis, James Ostrowski,
  George Pappas, C. J. Taylor (Penn)
• Howie Choset, Alfred Rizzi, Charles Thorpe (CMU)

3
Background

• ARO-MURI Integrated Approach to Intelligent
  Systems 1996-2001. Partners Berkeley, Stanford,
  Cornell. Highlights
• Creation of Field of Hybrid Systems
  Foundations, Methods, Analysis, Control
• Vision Based Navigation and Control
• Major Force Driving Bayesian Networks, Graphical
  Models, Dynamical Probabilisitic Networks,
  Learning, Rapproachment of AI and Control

4
Integrated Approach to Intelligent Systems

• Disciplinary Evolution
• Control Theory
• Optimal control, linear control, nonlinear
  control, adaptive control, stochastic control
• Mathematics differential equations
• Artificial Intelligence
• Reasoning, adaptation, neural networks, natural
  language, expert systems
• Mathematics formal logic
• Computational Neuroscience Cognitive Science
• Sensing, vision, taction, olfaction neural
  networks
• Mathematics (recently developed)

5
Why Hierarchical Hybrid Control?

• Central Control Paradigm. that is sensors and
  actuators interacting locally, breaks down when
  dealing with distributed systems due to
• Complexity scale
• Necessity of tight or optimal operations

• Key Characteristics of Distributed Intelligent
  Systems
• Hierarchical or modular to control complexity
• Globally organized emergent behavior
• Robust, adaptive and fault tolerant, and degraded
  modes of operation
• Architectural organization involving the use of
  compositionality

6
Why Hybrid Hierarchical Control?

• Intelligence Augmentation for Human-Centered
  Systems
• Autonomous Intelligence
• Why integrative? Due to
• - the need to merge sensor fusion and
  hierarchies of sensing with actuation across many
  agents, with desired emergent behavior
• - the need to merge logical decision making and
  continuous action
• - the need reconcile the need for safety of
  individual agents with collective optimality
• Control, artificial intelligence and cognitive
  neuroscience deal with continuous action, logical
  reasoning and human/machine understanding,
  respectively

7
Technology Drivers Semi-Autonomous Multi-Agent
Systems

• The need for a theoretical framework for an
  integrative approach arises from advances in
  computation, communication, intelligent
  materials, visualization and other technologies
  which make it possible to expect more from a
  multi-agent system than from a centralized
  control framework.
• Distributed Command and Control
• Distributed Communication Systems
• Distributed Power Systems
• Intelligent Vehicle Highway Systems
• Air Traffic Management Systems
• Intelligent Telemedical Systems
• Intelligent Manufacturing Systems
• Unmanned Aerial Vehicle Networks
• Mobile Offshore Bases

8
Theoretical Underpinnings

• Architectural Design for Multi-Agent Systems
• Hybrid Systems
• Centralization for optimality
• Decentralization for safety, reliability and
  speed of response
• Perception Systems Sharing Many Representations
• Hierarchical aggregation
• Wide-area surveillance
• Low-level perception
• Frameworks for Representing and Reasoning with
  Uncertainty
• Incorporation of Learning, Adaptation and Fault
  Tolerance
• Parametric uncertainty with update and adaptation
  at the continuous levels, learning of new
  logical entities --reinforcement learning at
  the logical levels and metal-learning for
  redefining architecture

9
What Are Hybrid Systems?

• Dynamical systems with interacting continuous and
  discrete dynamics

10
Why Hybrid Systems?

• Modeling abstraction of
• Continuous systems with phased operation (e.g.
  walking robots, mechanical systems with
  collisions, circuits with diodes)
• Continuous systems controlled by discrete inputs
  (e.g. switches, valves, digital computers)
• Coordinating processes (multi-agent systems)
• Important in applications
• Hardware verification/CAD, real time software
• Manufacturing, communication networks, multimedia
• Large scale, multi-agent systems
• Automated Highway Systems (AHS)
• Air Traffic Management Systems (ATM)
• Uninhabited Aerial Vehicles (UAV), Power Networks

11
Framework
12
Control Challenges

• Large number of semiautonomous agents
• Coordinate to
• Make efficient use of common resource
• Achieve a common goal
• Individual agents have various modes of operation
  
• Agents optimize locally, coordinate to resolve
  conflicts
• System architecture is hierarchical and
  distributed
• Safety critical systems
• Challenge Develop models, analysis, and
  synthesis tools for designing and verifying the
  safety of multi-agent systems

13
Taking Stock in Hybrid Systems

• Hybrid Systems and Control established as a
  discipline, taught to undergrads, grads.
  Monographs, textbooks being written by all
  co-PIs Lee and Varaiya, Henzinger and Alur,
  Lygeros, Tomlin and Sastry. Workshop on Hybrid
  Systems established (first was in Berkeley in
  1998). Special Issues in IEEE Proceedings,
  Systems and Control Letters, Automatica, IEEE
  Transactions on Automatic Control,
• Software Programming languages, tools and
  frameworks for Simulation and Control Ptolemy
  II, Giotto, Massaccio all developed. Ongoing work
  on verification tools.
• Hardware in the loop demonstrations on the local
  UAVs, formation flying to follow.
• Embedded software EMSOFT established, new IEEE
  Proceedings Special Issue 2003.

14
UCB/UCSF Laparoscopic Telesurgical Workstation
15
ANIMAL LAB TRIALS 1998
16
Suturing with Unimanual System, 1998
17
Berkeley BEAR Fleet Ursa Maxima 1
Based on Yamaha RMAX industrial helicopter
Integrated Nav/Comm Module
Length 3.63m Width0.72m Height 1.08m Dry
Weight 58 kg Payload 30kg Engine Output 21
hp Rotor Diameter 3.115m Flight system
operation time 60 min
18
Flight Control System Experiments
Landing scenario with SAS (Dec 1999)
PositionHeading Lock (Dec 1999)
PositionHeading Lock (May 2000)
Attitude control with mu-syn (July 2000)
19
Pursuit-Evasion Game Experiment Setup
Waypoint Command
Pursuer UAV
Current Position, Vehicle Stats
Evader location detected by Vision system
Ground Command Post
Current Position, Vehicle Stats
Evader UGV
20
Pursuit-Evasion Game Experiment

• PEG with four UGVs
• Global-Max pursuit policy
• Simulated camera view
• (radius 7.5m with 50degree conic view)
• Pursuer0.3m/s Evader0.5m/s MAX

21
Experimental Results Pursuit Evasion Games with
4UGVs (Spring 01)
22
Pursuit-Evasion Game Experiment

• PEG with four UGVs and a UAV
• Global-Max pursuit policy
• Simulated camera view
• (radius 7.5m with 50degree conic view)
• Pursuer0.3m/s Evader0.5m/s MAX

23
Experimental Results Pursuit Evasion Games with
4UGVs and 1 UAV (Spring01)
24
What is Different Today?

1. The world and national security threats are
   different mobile operations in urban terrain,
   hostage rescue, anti terrorism operations,
   homeland protection.
2. Use of robotic and mixed initiative forces, the
   need for coordination of manned and unmanned
   forces
3. The need for dynamic strategies and tactics for
   dealing with a determined and flexible adversary.
4. Exploitation of the 3rd dimension by organic UAVs

25
New Technical Innovations

• Control of the 3 D Digital battlefield need to
  use 3rd dimension, aerial forces, robotic and
  mixed initiative forces, untethered
  communications
• Adaptive Coordinated Control of Multiple Agents
  reconfiguration of teams dynamically in response
  to adversarial action
• Intelligent coordination of multiple agents
  ability to discover intent and reconfigure
  strategies adaptively

26
Intellectual OrganizationThrust Areas

• Architecture Design for Adaptive, Dynamic
  Planning
• Integration of Rich Multi-Sensor Information into
  Virtual Environments incorporating human
  intervention
• Handling Uncertainty and Adversarial Intent in
  Adaptive Planning

27
Challenge Scenarios

• Reconaissance and Intelligence robotic ranger
  force for scouting fixed area for time critical
  targets
• Mixed Initiative Engagement in urban environments
  using micro-UAVs, UGVs. Emphasis on immersive
  environments for deploying
• Recognition and Tracking of Unfriendlies
  emphasis on networked vision for tracking.

28
Hierarchical Architectures for Dynamic Adaptive
Planning

• Progess to date in hierarchical architectures for
  decision making in normal modes of operation.
  Main emphasis here will be on replanning in
  fault or degraded modes of operation
  including deviations from hierarchical operation.
• Key technical issues
• Abstractions of Hybrid Systems for Architecture
  Design
• Hierarchical abstractions
• Assume-guarantee reasoning for abstractions

29
Thrust I continued

• Control of Hybrid Systems
• Numerical Solutions for Controller Synthesis
• Hierarchical Solutions of Synthesis Procedures
• Liveness and other acceptance conditions
• Controller Libraries
• Many world semantics and hierarchy semantics
• Modal decomposition
• Exceptions
• Team and Task Allocation

30
Integration of Multi-Sensor Information Into
Virtual Environments

• Adaptive Hierarchial Networks for Acquiring and
  providing information
• Networked sensors
• Bandwidth utilitzation
• Extraction of 3 D Models from Distributed Sensors
• 3 D models from video data
• Integration of real and virtual environments
• Environments for Human Intervention Decision
  Making
• Situational awareness
• Display of uncertain data
• Triaging of data for decision making

31
Smart Dust, Dot Motes, MICA Motes

• Dot motes, MICA motes and smart dust

32
Tiny OS (TOS)

• Jason Hill, Robert Szewczyk, Alec Woo, David
  Culler
• TinyOS
• Ad hoc networking

33
MAVs for Delivery
www.spyplanes.com

• 60 mph
• 18 min
• 1 mi comm

34
Last 2 of 6 motes are dropped from MAV
35
Field of wireless sensor nodes

• Ad hoc, rather than engineered placement
• At least two potential modes of observation
• Acoustic, magnetic, RF

36
Subset of more powerful assets

• Gateway nodes with pan-tilt camera
• Limited instantaneous field of view

37
Set of objects moving through
38
Track a distinguished object
39
Many interesting problems

• Targeting of the cameras so as to have objects of
  interest in the field of view
• Collaborate between field of nodes and platform
  to perform ranging and localization to create
  coordinate system
• Building of a routing structures between field
  nodes and higher-level resources
• Targeting of high-level assets
• Sensors guide video assets in real time
• Video assets refine sensor-based estimate
• Network resources focused on region of importance

40
Abstraction of Sensorwebs

• Properties of general sensor nodes are described
  by
• sensing range, confidence on the sensed data
• memory, computation capability, clock skew
• Communication range, bandwidth, time delay,
  transmission loss
• broadcasting methods (periodic or event-based)
• To apply sensor nodes for the experiments with
  UAV/UGVs introduce super-nodes (or gateways),
  which can
• gather information from sub-nodes ( filtering or
  fusion of the data from sub-nodes for partial map
  building)
• communicate with UAV/UGVs

41
Uncertainty and Adversarial Intent

• Models of Uncertainty
• Environmental non deterministic and
  probabilistic
• Adversarial
• Guarantees of Success in the face of uncertainty
• Decision making in the presence of uncertainty
• Learning of Adversarial Strategy
• Probing strategies
• Games, partial information solution concepts
• Adaptation to changing utility functions of
  adversary