Software Architectures for Agents and Mobile Robots

1
Software Architectures for Agents and Mobile
Robots
Hans-Dieter Burkhard Humboldt University
Berlin Institute of Informatics
www.ki.informatik.hu-berlin.de
2
Topics of the talk

• Software Architectures for Agents and Mobile
  Robots

• AI at Humboldt University
• Agents Robots
• Architectures
• Mental states
• Control, Planning
• Double Pass Architecture

3
Artificial Intelligence at Humboldt University

Understanding emerges by doing. Applied to the
study of mental processes, this means modeling
of intelligent behavior by machines. Artificial
Intelligence has two aspects First modeling
with the goal of better understanding, and
second engineering of useful machines.
4
Artificial Intelligence at Humboldt University
English version
Case Based Reasoning Knowledge Management Agent
Oriented Techniques Distributed AI Socionics
Applications in Medicine Intelligent Robotics
www.ki.informatik.hu-berlin.de

5
Example Online Travel Agency

6
Travel Agent How does it work

• Stimulus-Response

Customer
Agent
Prepare answer (select and present best matching
offers)
Specify wish (fill in form)
7
Travel Agent How does it work

• Stimulus-Response Agent needs
• Knowledge about
• offers (data base)
• similarity (acceptable alternative offers)
• Capabilities to
• Update offers
• Interaction with customer
• Search of best matching offers
• ( ? Case Retrieval Nets)

8
Travel Agent How does it work
CRN CASE RETRIEVAL NET
9
Travel Agent How could it work

• Advisory agent

Customer
Agent
I would like to go for holidays.
10
Travel Agent How could it work

• Advisory agent

Customer
Agent
Fine. Do you like swimming?
I would like to go for holidays.
11
Travel Agent How could it work

• Advisory agent

Customer
Agent
Fine. Do you like swimming?
Yes, I like to be with my friend on a white
strand, no other tourists. And I enjoy sports.
12
Travel Agent How could it work

• Advisory agent

Customer
Agent
Wonderful. And in the evening?
Yes, I like to be with my friend on a white
strand, no other tourists. And I enjoy sports.
13
Travel Agent How could it work

• Advisory agent

Customer
Agent
Wonderful. And in the evening?
Good entertainment, exclusive bars, etc.
14
Travel Agent How could it work

• Advisory agent

Customer
Agent
Sounds fantastic, is this like what you
want? (presents an offer)
Good entertainment, exclusive bars, etc.
15
Travel Agent How could it work

• Advisory agent

Customer
Agent
Sounds fantastic, is this like what you
want? (presents an offer)
Looks fantastic. But it is far behind of my
financial limits, may be less exclusive.
16
Travel Agent How could it work

• Advisory agent

Customer
Agent
So, lets see. Whats about that? (presents
another offer)
Looks fantastic. But it is far behind of my
financial limits, may be less exclusive.
17
Travel Agent How could it work

• Advisory Agent needs
• Needs of Stimulus Response Agent
• (offers, capabilities, ...) as before

Dialog
18
Travel Agent How could it work

• Advisory Agent needs
• Dynamic knowledge about dialog with customer
• History of dialog
• (Hypothetical) Model of current customer
• Wishes, intentions
• Capabilities
• Beliefs
• (Flexible) Plan for
• Discovering customers wishes, intentions, ...
• Selling most valuable products

19
Agent Oriented Techniques

• Information agents
• Autonomous systems
• Cooperative systems
• Socionics humans autonomous machines
• Cooperation
• Sociological requirements
• Organizational aspects

Agents work autonomously on behalf of their
users.

• Autonomy Following own rules
• (example chess program)
• Autonomy w.r.t. somebody
• Complexity of decisions

20
Control of Autonomous Mobile Robots

• Problem Dynamically changing environments

Autonomous agents in real environments
Problems Localization, Movements, Control
21
Classical distinction of agents (robots)

• Reactive
• Simple stimulus response behavior
• No planning
• No persistent states
• Deliberative
• Complicated deliberation
• Planning
• Persistent states

22
Sense-Think-Act-Cycle, Persistency

Environment
23
Reactive Systems

• Obstacle avoidance by keeping distance
• Chess program ( ? - not simple)

Sensor-Actor-Coupling
24
Deliberative Systems

• With 3 persistent states for worldmodel, goals,
  plans

25
Travel agent

Agent
Worldmodel Discriminating customer
update
Goal Sell pricey
select
means-ends
Plan Show attractive offers etc.
output
execute
26
Unfolding the cycle
27
Synchronization Problem
Simple Synchronization
update
select
means-ends

• Problems for
• dynamical environments
• complex processes

update
select
Conflict
means-ends
28
Question

• ROBOT AGENT INSIDE A BODY ?

29
Simple architectures for physical agents

• Stimulus-Response
• Immediate reactions to inputs from the real
  world.
• The best model of the world is the world
  itself.
• Braitenberg
• Vehicle

No need for a complex agent inside the robot
30
Soccer Playing Robots

By the year 2050, develop a team of fully
autonomous humanoid robots that can win against
the human world soccer champion team.
ENIAC 1946
Deep Blue 1997
Test field for Goal driven research
31
Annual World Championships and Conferences

Simulation
Humanoid
Middle size
Rescue
www.robocup.org
Sony legged
Small size
32
Simple Stimulus-Response Behavior

• Run to the ball

33
Simple Stimulus-Response Behavior

• Run to the ball

34
Simple Stimulus-Response Behavior

• Run to the ball

35
Simple Stimulus-Response Behavior

• Run to the ball

LOOP worldmodel perceive (input)
commitment deliberate (worldmodel)
output execute(commitment)
select
sense
execute
A xxx B yyy C zzz
Agent
think
Sensor-Actor-Coupling
36
Why are they acting Triggering events

• Stimulus-Response
• recent events in the environment
• Goal-directed
• recent events in the environment
• internal goals

37
Goal-directed Behavior

• Improvement
• Anticipate future situations Goal

x
38
Goal-directed Behavior

• Improvement
• Anticipate future situations Goal

x
39
Goal-directed Behavior

• Improvement
• Anticipate future situations Goal

40
Mental States

• Concerning past
• Worldmodel
• Concerning future
• Commitment (goal, intention,
  plan, ...)
• Mental states are persistent states
• Keep information for more than one cycle

41
Stimulus-Response with Worldmodel

• Simulate unobservable events worldmodel

42
Stimulus-Response with Worldmodel

• Simulate unobservable events worldmodel

43
Stimulus-Response with Worldmodel

• Simulate unobservable events worldmodel

LOOP worldmodel_new update (input,
worldmodel_old) commitment
deliberate (worldmodel) output
execute(commitment)
44
Worldmodel

• persistent state concerning the past
• Worldmodel (Belief)

Preprocessing of input from sensory signals
worldmodel_new update (input,
worldmodel_old)
45
Plan for Cooperation

• Cooperation using joint intention (double pass)
• Remark Simulation of recent situation
  (world model )
• needs knowledge about
  teammates intention

46
Plan for Cooperation

• Cooperation using joint intention (double pass)
• Remark Simulation of recent situation
  (world model )
• needs knowledge about
  teammates intention

47
Plan for Cooperation

• Cooperation using joint intention (double pass)
• Remark Simulation of recent situation
  (world model )
• needs knowledge about
  teammates intention

48
Plan for Cooperation

• Cooperation using joint intention (double pass)
• Remark Simulation of recent situation
  (world model )
• needs knowledge about
  teammates intention

49
Plan for Cooperation

• Cooperation using joint intention (double pass)
• Remark Simulation of recent situation
  (world model )
• needs knowledge about
  teammates intention

50
Plan for Cooperation

• Cooperation using joint intention (double pass)
• Remark Simulation of recent situation
  (world model )
• needs knowledge about
  teammates intention

51
Commitments Goal-Directed Architecture
LOOP worldmodel_new update (input,
worldmodel_old) commitment_new deliberate
(worldmodel_new,commitment_old) output
execute (commitment_new)

• Difference to Stimulus Response
• Persistent state concerning the future
• (commitment goal, plan ...)

new alternatives
Commitment_new
Commitment_old
52
AT Humboldt 98 (Simulation league)

• worldmodel
• intentions
• plans
• utilities

53
Utility
Time to reach the ball (simulation of future)
54
Utility

• Fastest player
• to reach the ball
• (simulation
• of future)

55
Utility
Appropriate kick direction (simulation of future)
56
Problems Time Trade-Off

• Fast decision
• newest data
• rough criteria
• Complex deliberation
• detailed analysis, long term plans
• synchronization problem

think
Sensor-Actor-Coupling
worldmodel
plan
means-ends
goal
Agent
select
57
Problems Time Trade-Off

• Fast decision
• vs.
• Complex deliberation
• ? Architectures with different levels (layers)

• Need for balance between
• low level (Stimulus-Response)
• high level (Goal-directed)

58
Option Hierarchy
59
Choice-Option (OR-Branching)
State (Place)
condition
Transition with condition
Current State (marked Place)
Offensive
MaxUtility
MaxUtility
MaxUtility
...
Score
DoublePass/2
DoublePass/1
finished or canceled
finished or canceled
ball out of kickrange
...
60
Sequence-Option (AND-Branching)
State (Place)
condition
Transition with condition
Current State (marked Place)
Pass
Intercept
Run
Reposition
Dribble
Teammate passes
Pass finished
Teammate free
Teammate finished Pass
61
Extension for unexpected situation

• Additional transitions (with simple
  conditions)

Offensive
MaxUtility
MaxUtility
MaxUtility
...
ball control goal free
DoublePass/2
DoublePass/1
Score
ball out of kickrange
finished or canceled
finished or canceled
...
problem with team mate
62
Problems Stability Trade-Off
?

• Stabile behavior
• achieve goals
• reliability in cooperation
• ? fanatism
• Adaptation to new situation
• flexibility
• ? oscillation
• ? re-planning

?
commitment_old
new alternatives
commitment_new
63
Oscillation (Noisy Sensory Data)
64
Adaptation (Changing Plan)
65
Adaptation (Changing Plan)
66
Adaptation (Changing Plan)
67
Adaptation (Changing Plan)
68
Adaptation (Changing Plan)
69
Problems Stability Trade-Off

• Stabile behavior
• vs.
• Adaptation to new situation
• persistent state concerning future
• bias for old behavior (preventing from
  oscillation)

Need for balanced re-deliberation
70
Problems Context Problem
PlaySoccer
Offensive
Defensive
. . .

• Example
• (Opponent behaves in unexpected way)
• Active Behavior inside Dribbling
• Invalid Condition for Double Pass

Score
OffsideTrap
Attack
ChangeWings/1
DoublePass/2
DoublePass/1
...
...
Dribble
Kick
. . .
...
...
Pass
Run
Reposition
Need for re-consideration on all levels
? Problem for stack oriented runtime systems
Intercept
71
Stack oriented architectures

• Classical architectures are stack oriented
• Only the procedure on top of stack is active
• i.e., only low level behavior
• Higher level behavior can become active only when
• lower levels are finished/interrupted

Intentions may change on any level - caused by
external events
72
Travel agent
Intentions may change on any level - caused by
external events

Ooops no chance to sell pricey ...
Customer
Agent
Looks fantastic. But it is far behind of my
financial limits, may be less exclusive.
73
Problems Least Commitment

• Start Partial Plan
• Later Exact Parameters

Needs consideration on all levels
74
Double Pass Architecture

• Predefined Option Hierarchy
• Choosen Part of it
• Intention subtree
• (choosen by Deliberator)
• Active Part of it
• Activity path
• (updated by Executor)

75
Intention Subtree (chosen by Deliberator)
76
Activity Path Active Options
PlaySoccer
Offensive
Defensive
. . .
Score
OffsideTrap
Attack
ChangeWings/1
DoublePass/2
DoublePass/1
...
...
Dribble
Kick
. . .
...
...
...
...
...
...
Pass
...
...
...
...
Run
. . .
. . .
. . .
Reposition
...
...
Intercept
...
...
77
Doubled One-Pass-Architecture

• Deliberator-Pass (goal-oriented)
• builds Intention Subtree
• one deliberator pass may work over several
  cycles
• Executor-Pass (stimulus-response)
• traverses and adjusts Activity Path
• limited search space by Intention subtree
• one executor pass per cycle
• Differences to classical programming
• Control flow by deliberation (agent oriented)
• Double Pass Runtime Organization (not by stacks)

78
Deliberator Constructs Intention Subtree
PlaySoccer
Offensive
Defensive
. . .
Score
OffsideTrap
Attack
ChangeWings/1
DoublePass/2
DoublePass/1
...
...
Dribble
Kick
. . .
...
...
Pass
...
...
Run
Reposition
Construction may need longer time
...
...
Intercept
...
...
79
Executor-Pass through all levels
PlaySoccer
Offensive
Defensive
. . .
Score
OffsideTrap
Attack
ChangeWings/1
DoublePass/2
DoublePass/1
...
...
Dribble
Kick
. . .
...
...
...
...
...
...
Pass
...
...
...
...
Run
. . .
. . .
. . .
Reposition
...
...
in each cycle through all levels
Intercept
...
...
80
Executor-Pass through all levels
PlaySoccer
Offensive
Defensive
. . .
Score
OffsideTrap
Attack
ChangeWings/1
DoublePass/2
DoublePass/1
...
...
Dribble
Kick
. . .
...
...
...
...
...
...
Pass
...
...
...
...
Run
. . .
. . .
. . .
Reposition
...
...
in each cycle through all levels
Intercept
...
...
81
Executor-Pass through all levels
PlaySoccer
Offensive
Defensive
. . .
Score
OffsideTrap
Attack
ChangeWings/1
DoublePass/2
DoublePass/1
...
...
Dribble
Kick
. . .
...
...
...
...
...
...
Pass
...
...
...
...
Run
. . .
. . .
. . .
Reposition
...
...
in each cycle through all levels
Intercept
...
...
82
Executor-Pass through all levels
PlaySoccer
Offensive
Defensive
. . .
Score
OffsideTrap
Attack
ChangeWings/1
DoublePass/2
DoublePass/1
...
...
Dribble
Kick
. . .
...
...
...
...
...
...
Pass
...
...
...
...
Run
. . .
. . .
. . .
Reposition
...
...
in each cycle through all levels
Intercept
...
...
83
Executor-Pass through all levels
PlaySoccer
Offensive
Defensive
. . .
Score
OffsideTrap
Attack
ChangeWings/1
DoublePass/2
DoublePass/1
...
...
Dribble
Kick
. . .
...
...
...
...
...
...
Pass
...
...
...
...
Run
. . .
. . .
. . .
Reposition
...
...
in each cycle through all levels
Intercept
...
...
84
Double Pass Architecture

• Predefined Option Hierarchy
• Deliberator
• long term deliberation (not time critical)
• commitment for intentions intention subtree
• Executor
• short term reconsideration (time critical)
• performs intentions on the activity path

Both working top-down from root to leaves
85
Synchronization (parallel work)
Deliberator
Sensors
Perception
Activity path
Actions
Executor
86
Synchronization (sequential work)
Sensors
Perception
Deliberation
Plan
Deliberator
Sensors
Perception
Activity path
Actions
Executor
87
Double Pass Architecture Objectives

• Balance between low level/high level
• - Time Trade-off
• Balanced Re-deliberation
• - Stability Trade-Off
• Re-consideration on all levels
• - Context Problem
• - Least Commitment Problem

Long Term Research Goal Learning of complex
behavior (Case Based Reasoning)
88
In Progress

• Double Pass Architecture
• Formal specification
• Implementation
• Skills Behaviors

THANK YOU !