Multi Robot OITL Scaling Experiments

1
Multi Robot OITL Scaling Experiments

• Research Team
• U of Pittsburgh M. Lewis
• Huadong Wang, Shih Yi Chien, Zheng Ma,
• Peiju Lee, Dhruba Baishya
• CMU K. Sycara, P. Scerri
• Prasanna Velagapudi, Breelyn Kane

2
Cornell
GMU
CMU Psychology
MIT
Pitt
CMU Robotics
Level 1,3
Level 2
Level 1,2
Level 1
Scaling of cognitive performance and workload
Level 1-2.5
Level 1-3
Level 1,3
Level 1
Task allocation among humans/agents
Probabilistic models of human decision-making in
network situations
Level 1,2
Level 1-2.5
Level 3 ?
Level 2
Level 1-3
Decentralized control search and planning
Level 1,2
Level 2
Information fusion
Level 1,2
Level 1,3, 4
Network performance as a function of topology
Level 4
Level 2
Communication, evolution, language
Level 3
Level 2, 3
Adaptive automation
Level 1,2
Level 1
3
Architectural Framework for Human Control of
Multirobot Teams

• For controlling large(r) teams we need to
  consider how difficulty for the operator grows in
  N robots
• Borrowing from computational complexity we think
  there are 3 basic classes of commands
• O(1) difficulty independent of N robots
• O(n) difficulty proportional to N robots
• O(gtn) increase much greater than additive
• Vision Collaborating teams of humans robots
  using commands from all classes

4
Setting Goals O(1)
Operator draws regions to be searched on
screen The complexity of the plan is independent
of N of robots Robots may be either
independently autonomous or cooperating
autonomously
5
Individual Control O(n)
Teleoperation or Waypoint control Each
additional robot adds the same incremental effort
6
Coordinating robots O(gtn)
7
As size grows, complexity of Coordination should
dominate
O(gtn)
O(n)
Cognitive limit
O(1)
N of Robots
8
Plan of Attack

• O(1)- many problems such as opacity but not
  scaling
• O(gtn)- Automate coordination
• because difficulty is combinatorial and
  interdependence of action precludes human control
• O(n) because of independence among robots
  amenable to scheduling, automation, and teamwork
  interventions
• Neglect tolerance model
• Call center other team centered approaches
• individual automation

9
Plan of Attack

• O(1)- many problems such as opacity but not
  scaling
• O(gtn)- Automate coordination
• because difficulty is combinatorial and
  interdependence of action precludes human control
• O(n) because of independence among robots
  amenable to scheduling, automation, and teamwork
  interventions
• Neglect tolerance model
• Call center other team centered approaches
• individual automation

10
Problem Human Control of Large(r) Robot Teams

• IDEA
• Examine difficulty of control tasks as N robots
  increases
• Identify which tasks must be automated or
  allocated differently to allow control of larger
  teams
• Automate identified tasks retest

11
Methodology Experiment Design

• Fulltask condition
• Participants both dictated the robots paths
  and controlled their cameras to search for
  victims and to mark them on the map.
• Exploration subtask
• Participants directed the team of robots in
  order to explore as wide an area as possible.
• Perceptual search condition
• Participants searched for victims by
  controlling cameras mounted on robots following
  predetermined paths selected to match
  characteristics of paths generated under the
  other two conditions.

12
USARSim Validation Studies

• Synthetic video
• Carpin, S., Stoyanov, T., Nevatia, Y., Lewis, M.
  and Wang, J. (2006a). Quantitative assessments of
  USARSim accuracy". Proceedings of PerMIS 2006
• Hokuyo laser range finder
• Carpin, S., Wang, J., Lewis, M., Birk, A., and
  Jacoff, A. (2005). High fidelity tools for rescue
  robotics Results and perspectives, Robocup 2005
  Symposium.
• Platform physics behavior
• Carpin, S., Lewis, M., Wang, J., Balakirsky, S.
  and Scrapper, C. (2006b). Bridging the gap
  between simulation and reality in urban search
  and rescue. Robocup 2006 Robot Soccer World
  Cup X, Springer, Lecture Notes in Artificial
  Intelligence
• Lewis, M., Hughes, S., Wang, J., Koes, M. and
  Carpin, S., Validating USARsim for use in HRI
  research, Proceedings of the 49th Annual Meeting
  of the Human Factors and Ergonomics Society,
  Orlando, FL, 457-461, 2005.
• Pepper, C., Balakirsky, S. and Scrapper, C.,
  Robot Simulation Physics Validation, Proceedings
  of PerMIS07, 2007.
• Taylor, B., Balakirsky, S., Messina, E. and
  Quinn, R., Design and Validation of a Whegs Robot
  in USARSim, Proceedings of PerMIS07.
• Zaratti, M., Fratarcangeli, M., and Iocchi, L., A
  3D Simulator of Multiple Legged Robots based on
  USARSim. Robocup 2006 Robot Soccer World Cup X,
  Springer, LNAI, 2006.

www.sourceforge.net/project/usarsim
13
Introduction GUI for Multi-robot Control
14
Robots and Maps
Office-like environments
P2AT Robot
15
Methodology Experiment Design

• Between Groups repeated measure design
• 315 Participants from University of Pittsburgh
• Standard Instruction, 20min Training, 315min
  Testing Session, NASA-TLX workload survey after
  each testing Session

4 Robots 8 Robots 12 Robots
FULLTASK
EXPORATION
PERCEPTUAL SEARCH
Number of Robots
Conditions
16
Methodology Experiment Design

• Independent Variables
• Conditions of Task
• Numbers of Robots
• Dependent Variables
• NASA-TLX Workload
• Victims found
• Area Explored
• Switches in focus among robots
• Number of assigned missions
• Average path length
• Robots neglected or operated only once

17
Results

• Victim Found as a function of N robots

F1,28 27.4 p lt .0001
18
Results

• Area explored as a function of N robots

F1,28 21.17 p lt .002
19
Results

• Workload as a function of N robots

F1,27 21.17 p lt .0001 Fulltask x search
20
Conclusions Experiment 1

• Exploration was the limiting component of Full
  task performance
• ?2 for improvement in performance with team size
  much higher for finding victims than area
  explored
• Workload much lower for perceptual search
• Full task performance fell apart at 12
• Half the victims found as in perceptual search

21
Call Center Control

• Restricted to O(n) commands
• Responsive to variation in demands
• Performance depends on match between model task
• automation vs. monitoring

22
How a Call Center Works

• As requests arrive operators service them using
  FIFO or similar discipline
• Does not require monitoring or extended SA
• Can benefit from scheduling results for assigning
  tasks based on expected duration
• As operators task approaches that of a server
  control benefits will become more pronounced
• Requires automating navigation through path
  planning, monitoring through self
  diagnosis/reflection, etc.

23
Team Experiment-1 (control)

• Assigned robots- each operator assigned 12 robots
  to control
• VS.
• Call Center- both operators given opportunity to
  control any of the 24 robots
• Task foraging same as earlier fulltask condition
• Operator role poor approximation of server

24
12 Assigned Robot Condition
25
24 Robot Control Interface
26
Performance better for assigned robots
27
Call Center operators neglect more robots
28
Workload is higher for assigned robots
29
Team Experiment 2

• Path planning automated using max entropy
  algorithms (previously used for UAVs)
• Operators monitor to find victims and free stuck
  robots
• Algorithm performance shown comparable to human
  operators (data from experiment-1)
• New algorithms designed tested for high traffic
  control
• Assigned robots- each operator assigned 12 robots
  to control
• VS.
• Call Center- both operators given opportunity to
  control any of the 24 robots
• Spatial Orientation test added to protocol to
  provide data for cognitive modeling of workload
  attention switching

30
Experiment 2 uses data from 1 as control
Assigned Robots Call Center
Manual path planning Experiment 1 Experiment 1
Automated path planning Experiment 2 Experiment 2
31
Victims Found
F1,56 13.436 p.001
difference
32
Region explored both main effects interaction
Autonomy F1,566.982 p.011
Interaction F1,567.878 p.007
Team F1,563.701 p.059
difference
difference
33
Victims/Area effect for autonomy, interaction,
autonomy for assigned robots
Autonomy F1,567.138 p.01
Interaction F1,567.138 p.054
34
RMS Error
Teams F1,5618.031 plt.001
Autonomy F1,565.434 plt.023
35
NASA-TLX
Team t(118)1.933 p.056
36
Collaborations

• Working with Christian Lebiere David Reitter
  (CMU) to develop cognitive models of operators
• Reitter, D., Lebiere, C., Lewis, M., Wang, H.,
  and Ma, Z. (2009). A cognitive model of visual
  path planning in a multi-robot control system,
  Proceedings of the 2009 IEEE International
  Conference on Systems, Man, and Cybernetics,
  October.
• Supplying data for team supervisory control model
  being developed by Brian Mekdeci in Missy
  Cummings lab

37
Future Research

• Extend to operator teams for O(n)
• Self diagnosis
• Self reflection
• Asynchronous/merged camera views
• Replays for SA

38
New Areas

• O(1)- many problems such as opacity but not
  scaling
• O(gtn)- Automate coordination
• because difficulty is combinatorial and
  interdependence of action precludes human control
• O(n) because of independence among robots
  amenable to scheduling, automation, and teamwork
  interventions
• Neglect tolerance model
• Call center other team centered approaches
• individual automation

39
Effective coordination for interdependent UVs

• Translucent
• Plan libraries explicit human roles
• Machinetta MAS implemented in current testbed
• OITL experiments to begin after Call Center
  completes
• Opaque
• Centralized controller typically optimizing
• Will investigate divide conquer design approach
• Biologically inspired control laws emergent
  coordination
• Will investigate amorphous algorithm approach

40
Things that make Algorithms Opaque

• Centralized optimization requires defining a
  figure of merit (FM)/goal to guide execution
• Human goals involve domain objects but algorithm
  only optimizes to its FM
• Operator lacks expressivity e.g. hit X
• Operator lacks mental model
• Biologically inspired local control laws lack
  levers for human control
• Investigate human intelligible propagating
  control approaches

41
Cornell
GMU
CMU Psychology
MIT
Pitt
CMU Robotics
Level 1,3
Level 2
Level 1,2
Level 1
Scaling of cognitive performance and workload
Level 1-2.5
Level 1-3
Level 1,3
Level 1 Level 2
Task allocation among humans/agents
Probabilistic models of human decision-making in
network situations
Level 1,2
Level 1-2.5
Level 3 ?
Level 2
Level 1-3
Decentralized control search and planning
Level 1,2
Level 2
Information fusion
Level 1,2
Level 1,3, 4
Level 1,2
Network performance as a function of topology
Level 4
Level 2
Communication, evolution, language
Level 3
Level 2, 3
Adaptive automation
Level 1,2
Level 1
Level 1,2
42
Publications

• Wang, H., Lewis, M., Velagapudi, P., Scerri, P.,
  and Sycara K. (2009). How Search and its Subtasks
  Scale in N Robots, Proceedings of the Forth
  ACM/IEEE International Conference on Human-Robot
  Interaction (HRI'09), March 9-13.
• Lewis, M., Wang, H., Velagapudi, P., Scerri, P.
  Sycara, K. (2009). Using humans as sensors in
  robotic search, Proceedings of the 12th
  International Conference on Information Fusion,
  July 6-9.
• Baishya, D. Lewis, M. (2009). Algorithm
  steering for mixed-initiative robot teams, 8th
  International Conference on Autonomous Agents and
  Multiagent Systems (AAMAS09) Workshop on
  Mixed-Initiative MAS, May 10-15.
• Lewis, M., Balakirsky, S. Carpin, S. (2009).
  Contributions of the virtual robot RoboCup Rescue
  competition to research in robotics, 21st
  International Joint Conference on Artificial
  Intelligence (IJCAI09) Workshop on Competitions
  in Artificial Intelligence and Robotics, July 12.
• Lewis, M., Sycara, K., Scerri, P. (2009).
  Scaling up wide-area-search-munition teams, IEEE
  Intelligent Systems, 24(3), 10-13.
• Velagapudi, P., Owens, S., Scerri, P., Sycara,
  K., Lewis, M. (2009). Environmental factors
  affecting situation awareness in unmanned aerial
  vehicles, AIAA Unmanned..Unlimited Conference,
  April 6-9.
• Lewis, M. and Wang, J. (2009). Measuring
  coordination demand in multirobot teams,
  Proceedings of the 53rd Annual Meeting of the
  Human Factors and Ergonomics Society, October.
• Reitter, D., Lebiere, C., Lewis, M., Wang, H.,
  and Ma, Z. (2009). A cognitive model of visual
  path planning in a multi-robot control system,
  Proceedings of the 2009 IEEE International
  Conference on Systems, Man, and Cybernetics,
  October.
• Velagapudi, P., Wang, H., Lewis, M., Scerri, P.,
  and Sycara, K. (2009). Scaling Effects for
  Streaming Video vs. Static Panorama in Multirobot
  Search, The 2009 IEEE/RSJ International
  Conference on Intelligent RObots and Systems,
  October.
• Wang, H., Lewis, M., Velagapudi, P., Scerri, P.,
  and Sycara, K. (2009). Scaling effects for
  synchronous vs. asynchronous video in multi-robot
  search, Proceedings of the 53rd Annual Meeting of
  the Human Factors and Ergonomics Society,
  October.
• Wang, H., Chien, S., Lewis, M., Velagapudi, P.,
  Scerri, P., and Sycara, K. (2009). Human teams
  for large scale multirobot control, Proceedings
  of the 2009 IEEE International Conference on
  Systems, Man, and Cybernetics, October.