Autonomous Robot Teams in Dynamic and Uncertain Environments

1
Autonomous Robot Teams in Dynamic and Uncertain
Environments

• Manuela Veloso
• Tucker Balch
• Mike Bowling, James Bruce, Rosemary Emery, Scott
  Lenser,
• Ashley Stroupe, John Sweeney, Will Uther, Elly
  Winner
• The MultiRobot Lab, CMU

2
Challenges in Multirobot Domains

• Localization
• Manipulation and navigation
• Communication
• Fusion of distributed sensing
• Graceful degradation in the face of reduced
  knowledge
• Learning

3
Sensor Resetting Localization

• Visual landmark-based probabilistic technique.
• Initially applied to Sony Aibo, where robot
  movement is noisy and not well modeled.
• Addresses situations where sensor information
  does not match estimated position.
• Useful in any application where landmarks are
  available.

4
SRL Approach

• SRL adds an additional step to the sensor update
  phase of localization
• If the probability of the estimated position is
  low given the sensor readings,
• Then SRL replaces some samples with samples drawn
  from the pdf given by the sensors P(ls).

5
SRL Recent Progress

• Ported to TeamBots environment.
• Extended to address ambiguous landmarks.

6
SRL Multiple Robots in TeamBots
7
CMVision Tools for Real-Time Color-Based
Tracking
30Hz
Original image
Classified regions with bounding boxes and
centroids

• YUV or RGB
• 32 colors classified
• Regions, even disjoint, labeled as objects
• Thresholds set manually or learned
• Released to community on web
• www.cs.cmu.edu/multirobotlab

8
CMVision Visualization Tool

• Online tool shows image, classified image and
  location of any pixel in color space.
• Thresholds can be set manually or learned.

9
Robot Colonization

• General problem
• Search for various types of objects in an
  unmapped environment, transport them to specific
  locations according to type.
• Some objects may be of greater value than others
  (e.g. severely injured people).
• The environment may change dynamically, thus
  impacting the feasibility of some plans at
  execution time.
• Applications
• search and rescue,
• de-mining, and
• construction.

10
Robot Colonization Recent Progress

• Two Cye-based robots assembled, three more under
  construction.
• Integration with TeamBots
• simulation,
• robot execution,
• visual sensing of obstacles and other objects,
  and
• communication
• Behaviors for navigation and manipulation have
  been developed.
• Shared sensing developed.

11
Navigation and Manipulation

• Must navigate to a position from which the object
  can be pushed.
• While pushing, must maintain frictional contact
  with box -- no sharp turns.
• Simultaneously avoid obstacles.
• Accomplished using motor schemas in TeamBots.

12
Robot Colonization Communication and Distributed
Sensing

• Enable all robots to perceive objects any one
  of them senses directly.
• Leverage multiple distributed sensors using
  probabilistic models of sensor noise.

13
Robot Colonization Example Scenario
14
Multi-Fidelity Behaviors

• General modes of behavior.
• Different levels of behaviors as a function of
  the accuracy of the processed sensory data
  visual, localization.
• Recover ball - low fidelity.
• Search for ball - low fidelity.
• Approach ball - low, high fidelity.
• Score - low, medium, high fidelity.

15
Multi-Fidelity Behaviors Example
16
Multi-Fidelity Behaviors Example

• Search, low-fidelity
• (random search)
• Until the robot sees the ball,
• walk forward a random distance,
• turn a random angle
• Approach, low-fidelity
• Run straight towards the ball.
• Approach, high-fidelity
• Skew approach to ball to get behind it,
• when closer to its goal position.

17
Multi-Fidelity Behaviors Example

• Score, low-fidelity
• Until the robot sees the goal,
• walk sideways around the ball.
• Walk forward pushing the ball.
• Score, high-fidelity
• Circle ball using shortest distance.
• If facing goal, push ball forward.
• Recover, low-fidelity
• Walk backwards for preset time.
• If do not see ball,
• then turn in the direction last seen.

18
Adjusted Policy Hill Climbing

• Motivation agents must learn strategies to adapt
  to other agents.
• Approach stochastic game theory and multiagent
  reinforcement learning.
• New algorithm
• adjusted policy hill climbing,
• rational and (empirically) convergent.

19
Adjusted Policy Hill Climbing Algorithm
20
Adjusted Policy Hill Climbing Results
Adjusted policy hill climbing
Policy hill climbing
Two-agent competitive zero-sum scenario
21
Adjusted Policy Hill Climbing Results
22
Recent Relevant Publications

• Software released to robotics community
• TeamBots 2.0 released via web
• CMVision 1.0 released via web
• Sensor Resetting Localization for poorly modeled
  mobile robots, Lenser Veloso, ICRA-00
• Social potentials for scalable multi-robot
  formations, Balch Hybinette, ICRA-00 (short
  version also at ICMAS-00)
• Real-time color image segmentation using
  commodity hardware, Bruce, Balch Veloso,
  WIRE-00
• Multi-Fidelity Behaviors for Robots, Winner
  Veloso, AAAI-00
• On behavior classification in adversarial
  environments, Riley Veloso, AAAI-00 student
  abstract
• Vision-servoed localization and behavior-based
  planning for a quadruped legged robot, Veloso,
  Winner, Lenser, Bruce Balch, AIPS-00

23
Whats next...

• Continue scale up to larger teams of autonomous
  robots.
• Continue development of control of small robots
  for soccer and colonization.
• Continue to pursue research issues
• cooperative localization,
• communication for strategy and shared sensing,
• real-time modeling of adversaries,
• real-time multirobot continuous planning,
• multirobot control learning.

24

• www.cs.cmu.edu/multirobotlab