Cooperating Intelligent Systems

1
Cooperating Intelligent Systems

• Intelligent Agents
• Chapter 2, AIMA

2
An agent

• An agent perceives its environment through
  sensors and acts upon that environment through
  actuators.
• Percepts x
• Actions a
• Agent function f

Image borrowed from W. H. Hsu, KSU
3
An agent

• An agent perceives its environment through
  sensors and acts upon that environment through
  actuators.
• Percepts x
• Actions a
• Agent function f

Image borrowed from W. H. Hsu, KSU
Percept sequence
4
Example Vacuum cleaner world
Image borrowed from V. Pavlovic, Rutgers
Percepts x1(t) ? A, B, x2(t) ? clean,
dirty Actions a1(t) suck, a2(t) right,
a3(t) left
5
Example Vacuum cleaner world
Image borrowed from V. Pavlovic, Rutgers
Percepts x1(t) ? A, B, x2(t) ? clean,
dirty Actions a1(t) suck, a2(t) right,
a3(t) left
This is an example of a reflex agent
6
A rational agent
A rational agent does the right thing For
each possible percept sequence,
x(t)...x(0),should a rational agent select the
action thatis expected to maximize its
performance measure, given the evidence provided
by thepercept sequence and whatever
built-inknowledge the agent has.
We design the performance measure, S
7
Rationality

• Rationality ? omniscience
• Rational decision depends on the agents
  experiences in the past (up to now), not expected
  experiences in the future or others experiences
  (unknown to the agent).
• Rationality means optimizing expected
  performance, omniscience is perfect knowledge.

8
Vacuum cleaner world performance measure
Image borrowed from V. Pavlovic, Rutgers
State definedperf. measure
Does notreally leadto goodbehavior
Action definedperf. measure
9
Task environment

• Task environment problem to which the agent is
  a solution.

10
Some basic agents

• Random agent
• Reflex agent
• Model-based agent
• Goal-based agent
• Utility-based agent
• Learning agents

11
The reflex agent The action a(t) is selected
based on only the most recent percept x(t) No
consideration of percept history. Can end up in
infinite loops.

• The random agent
• The action a(t) is selected purely at random,
  without any consideration of the percept x(t)
• Not very intelligent.

12
function SIMPLE-REFLEX-AGENT(percept) returns
action static rules, a set of condition-action
rules state ? INTERPRET-INPUT (percept) rule
? RULE-MATCH (state,rules) action ?
RULE-ACTION rule return action
First match. No further matches sought. Only one
level of deduction.
A simple reflex agent works by finding a rule
whose condition matches the current situation (as
defined by the percept) and then doing the action
associated with that rule.
Slide borrowed from Sandholm _at_ CMU
13
Simple reflex agent

• Table lookup of condition-action pairs defining
  all possible condition-action rules necessary to
  interact in an environment
• e.g. if car-in-front-is-breaking then initiate
  breaking
• Problems
• Table is still too big to generate and to store
  (e.g. taxi)
• Takes long time to build the table
• No knowledge of non-perceptual parts of the
  current state
• Not adaptive to changes in the environment
  requires entire table to be updated if changes
  occur
• Looping Cant make actions conditional

Slide borrowed from Sandholm _at_ CMU
14
The goal based agent The action a(t) is
selected based on the percept x(t), the current
state q(t), and the future expected set of
states. One or more of the states is the goal
state.

• The model based agent
• The action a(t) is selected based on the percept
  x(t) and the current state q(t).
• The state q(t) keeps track of the past actions
  and the percept history.

15
Reflex agent with internal state
Model based agent
Slide borrowed from Sandholm _at_ CMU
16
Agent with explicit goals
Goal based agent
Slide borrowed from Sandholm _at_ CMU
17
The learning agent The learning agent is
similar to the utility based agent. The
difference is that the knowledge parts can now
adapt (i.e. The prediction of future states, the
utility, ...etc.)

• The utility based agent
• The action a(t) is selected based on the percept
  x(t), and the utility of future, current, and
  past states q(t).
• The utility function U(q(t)) expresses the
  benefit the agent has from being in state q(t).

18
Utility-based agent
Slide borrowed from Sandholm _at_ CMU
19
Discussion

• Exercise 2.2
• Both the performance measure and the utility
  function measure how well an agent is doing.
  Explain the difference between the two.
• They can be the same but do not have to be. The
  performance function is used externally to
  measure the agents performance. The utility
  function is used internally to measure (or
  estimate) the performance. There is always a
  performance function but not always an utility
  function.

20
Discussion

• Exercise 2.2
• Both the performance measure and the utility
  function measure how well an agent is doing.
  Explain the difference between the two.
• They can be the same but do not have to be. The
  performance function is used externally to
  measure the agents performance. The utility
  function is used internally (by the agent) to
  measure (or estimate) its performance. There is
  always a performance function but not always an
  utility function (cf. random agent).

21
Exercise

• Exercise 2.4
• Lets examine the rationality of various
  vacuum-cleaner agent functions.
• Show that the simple vacuum-cleaner agent
  function described in figure 2.3 is indeed
  rational under the assumptions listed on page 36.
• Describe a rational agent function for the
  modified performance measure that deducts one
  point for each movement. Does the corresponding
  agent program require internal state?
• Discuss possible agent designs for the cases in
  which clean squares can become dirty and the
  geography of the environment is unknown. Does it
  make sense for the agent to learn from its
  experience in these cases? If so, what should it
  learn?

22
Exercise

• Exercise 2.4
• Lets examine the rationality of various
  vacuum-cleaner agent functions.
• Show that the simple vacuum-cleaner agent
  function described in figure 2.3 is indeed
  rational under the assumptions listed on page 36.
• Describe a rational agent function for the
  modified performance measure that deducts one
  point for each movement. Does the corresponding
  agent program require internal state?
• Discuss possible agent designs for the cases in
  which clean squares can become dirty and the
  geography of the environment is unknown. Does it
  make sense for the agent to learn from its
  experience in these cases? If so, what should it
  learn?

23
What should bethe performancemeasure?
24
Table-driven agent
function TABLE-DRIVEN-AGENT (percept) returns
action static percepts, a sequence, initially
empty table, a table, indexed by
percept sequences, initially fully specified
append percept to the end of percepts action ?
LOOKUP(percepts, table) return action
An agent based on a pre-specified lookup table.
It keeps track of percept sequence and just looks
up the best action

• Problems
• Huge number of possible percepts (consider an
  automated taxi with a camera as the sensor) gt
  lookup table would be huge
• Takes long time to build the table
• Not adaptive to changes in the environment
  requires entire table to be updated if changes
  occur

Slide borrowed from Sandholm _at_ CMU
25
What should bethe performancemeasure?
26
Possible states of the world A, Clean B,
Clean A, Clean B, Dirty A, Dirty B,
Dirty A, Dirty B, Clean How long will it
take for the agent to clean the world?
Possible states of the world A, Clean B,
Clean ? world is clean after 0 steps A, Clean
B, Dirty ? world is clean after 2 steps if
agent is in A and... A, Dirty B, Dirty ?
world is clean after 3 steps A, Dirty B,
Clean ? world is clean after 1 step if agent is
in A and... Can any agent do it faster (in fewer
steps)?
27
Exercise 2.4

• If (square A dirty square B clean) then the
  world is clean after one step. No agent can do
  this quicker.If (square A clean square B
  dirty) then the world is clean after two steps.
  No agent can do this quicker.If (square A dirty
  square B dirty) then the world is clean after
  three steps. No agent can do this quicker.The
  agent is rational (elapsed time is our
  performance measure).

Image borrowed from V. Pavlovic, Rutgers
28
Exercise

• Exercise 2.4
• Lets examine the rationality of various
  vacuum-cleaner agent functions.
• Show that the simple vacuum-cleaner agent
  function described in figure 2.3 is indeed
  rational under the assumptions listed on page 36.
• Describe a rational agent function for the
  modified performance measure that deducts one
  point for each movement. Does the corresponding
  agent program require internal state?
• Discuss possible agent designs for the cases in
  which clean squares can become dirty and the
  geography of the environment is unknown. Does it
  make sense for the agent to learn from its
  experience in these cases? If so, what should it
  learn?

29
Exercise 2.4

• The reflex agent will continue moving even after
  the world is clean. An agent that has memory
  would do better than the reflex agent if there is
  a penalty for each move. Memory prevents the
  agent from visiting squares where it has already
  cleaned.(The environment has no production of
  dirt a dirty square that has been cleaned
  remains clean.)

Image borrowed from V. Pavlovic, Rutgers
30
Exercise

• Exercise 2.4
• Lets examine the rationality of various
  vacuum-cleaner agent functions.
• Show that the simple vacuum-cleaner agent
  function described in figure 2.3 is indeed
  rational under the assumptions listed on page 36.
• Describe a rational agent function for the
  modified performance measure that deducts one
  point for each movement. Does the corresponding
  agent program require internal state?
• Discuss possible agent designs for the cases in
  which clean squares can become dirty and the
  geography of the environment is unknown. Does it
  make sense for the agent to learn from its
  experience in these cases? If so, what should it
  learn?

31
Exercise 2.4

• If the agent has a very long lifetime (infinite)
  then it is better to learn a map. The map can
  tell where the probability is high for dirt to
  accumulate. The map can carry information about
  how much time has passed since the vacuum cleaner
  agent visited a certain square, and thus also the
  probability that the square has become dirty.If
  the agent has a short lifetime, then it may just
  as well wander around randomly (there is no time
  to build a map).

Image borrowed from V. Pavlovic, Rutgers