Designing Intelligent Agents

1
Designing Intelligent Agents
A wholistic approach of AI
CPSC 420 Artificial Intelligence (Summer 2004)

2
What is an Intelligent Agent?

• Definition An agent perceives its environment
  via sensors and acts in that environment with its
  effectors.
• Hence, an agent gets percepts one at a time, and
  maps this percept sequence to actions (one action
  at a time)
• Examples
• A robot
• A web shopping program
• A factory
• A traffic control system

CPSC 420 Artificial Intelligence (Summer 2004)

3
An Agent and its Environment
An example of the agent and environment
dichotomy. This figure illustrates a robot taking
actions that affect the state of the environment
then receiving percepts with new information on
the environment. (Image courtesy of Beryl Simon.)
CPSC 420 Artificial Intelligence (Summer 2004)

4
How do we design an intelligent agent?
CPSC 420 Artificial Intelligence (Summer 2004)

5
World Model

• A the action space
• P the percept space
• E the environment A ? P
• S the internal state of E
• Perception function S ? P
• World dynamics S ? A ? P

CPSC 420 Artificial Intelligence (Summer 2004)

6
The Concept of Rationality
What is rational at any given time depends on
four things

• Performance measurement the criterion for
  success of an agents behavior
• Agents prior knowledge of the environment
• The actions that the agent can perform
• Agents percept sequence to date

Ideal rational agent
For each possible percept sequence, a rational
agent should select an action that is expected to
maximize its performance measure, given the
evidence provided by the percept sequence and
whatever built-in knowledge the agent has.
CPSC 420 Artificial Intelligence (Summer 2004)

7

The Concept of Rationality

• Rational ? Omniscience
• Rational ? Success
• Information gathering modifying future percepts
• Learning modifying and augmenting prior
  knowledge
• Autonomy an agent relies on the prior knowledge
  of its designer rather than its own percepts, we
  say that it lacks autonomy

CPSC 420 Artificial Intelligence (Summer 2004)

8
Agent Design Problem

• U utility function S ? ? (or S ? ?)
• The agent design problem find P ? A (agent
  function)
• Mapping of sequences (history) of percepts to
  actions
• Maximizes the utility of the resulting sequence
  of states (each action maps from one state to
  next state)

Three periods of the task of computing agent
function

• when the agent is being designed, some of the
  computation is done by its designer
• when it is deliberating on its next action, the
  agent does more
• the agent learns from experience.

CPSC 420 Artificial Intelligence (Summer 2004)

9
Limited Rationality

• The agent might not be able to compute the best
  action (subject to its beliefs and goals)
• Limited rationality acting in the best way you
  can subject to the computational constraints that
  you have
• The (limited rational) agent design problem
  finding P ? A (agent function)
• Mapping of sequences (history) of percepts to
  actions
• Maximizes the utility of the resulting sequence
  of states (each action maps from one state to
  next state)
• subject to the computational constraints

CPSC 420 Artificial Intelligence (Summer 2004)

10
Agent Architecture Program
Agent program is a concrete implementation of
agent function, running on the agent architecture.
11
Issues of Agent Program

• How to specify completely the domain where the
  agent is going to work in?
• If you expect a problem to be solved, you have
  to say what is the problem
• Specification is usually iterative build agent,
  test, modify specification
• Why isnt this just software engineering?
• There is a huge gap between specification and
  the program
• Is automatic programming possible?
• It could be, but it is so hard and most people
  have give up
• We do not construct programs automatically
• We need to map classes of environments and
  utilities to structures of agent programs that
  solve that class of problem.

CPSC 420 Artificial Intelligence (Summer 2004)

12
Encoding Thinking

• Arent agents supposed to think?
• If you can be endowed with an optimal table of
  reactions/reflexes (P A ), why do you need
  think?
• How big is the table? How many world states and
  sequences of percepts?
• Embodied AI approach in some domains, the
  required reaction table can be specified
  compactly in a program (written by a human)
• In some domain, we can take advantage of the
  fact that most things that could happen but dont.

CPSC 420 Artificial Intelligence (Summer 2004)

13
How the agent function can be implemented ?
CPSC 420 Artificial Intelligence (Summer 2004)

14
Structures of Agents
Reflex (reactive) agent without memorizing the
historical perceptions
p
a
Policy
What can you solve this way?

• Accessible environments
• Playing chess
• Navigating down a hallway

CPSC 420 Artificial Intelligence (Summer 2004)

15
Structures of Agents
Agent with memorizing the historical perceptions
Mental state
p
a
State Estimator
Policy
State estimator/Memory

• What the agent has chosen to remember from the
  history of percepts
• Maps what it knew before, what it just perceived
  and what it just did, into what it knows now.

CPSC 420 Artificial Intelligence (Summer 2004)

16
Planning Agent Policy
Planning is explicitly considering future
consequences of actions in order to choose the
best one.
s1
U
Current State
a1
U
ai that leads to max U
s
s2
a
a2
U
a3
U
s3
U
U
CPSC 420 Artificial Intelligence (Summer 2004)

17
Different Agent Programs

• Table-driven agent
• Simple reflex agent
• Reflex agent with internal state
• Agent with explicit goals
• Utility-based agent
• Learning agent

More sophisticated
CPSC 420 Artificial Intelligence (Summer 2004)

18
Table-Driven Agent
function TABLE-DRIVEN-AGENT (percept) returns
action static percepts, a sequence, initially
empty table, a table, indexed by
percept sequences, initially fully specified
append percept to the end of percepts action ?
LOOKUP(percepts, table) return action
An agent based on a prespecified lookup table.
It keeps track of percept sequence and just looks
up the best action

• Problems
• Huge number of possible percepts (consider an
  automated taxi with a camera as the sensor) gt
  lookup table would be huge
• Takes long time to build the table
• Not adaptive to changes in the environment
  requires entire table to be updated if changes
  occur

CPSC 420 Artificial Intelligence (Summer 2004)

19
Simple Reflex Agent
function SIMPLE-REFLEX-AGENT(percept) returns
action static rules, a set of condition-action
rules state ? INTERPRET-INPUT (percept) rule
? RULE-MATCH (state,rules) (one level
deduction) action ? RULE-ACTION rule return
action
A simple reflex agent works by finding a rule
whose condition matches the current situation (as
defined by the percept) and then doing the action
associated with that rule.
CPSC 420 Artificial Intelligence (Summer 2004)

20
Reflex Agent with Internal State
CPSC 420 Artificial Intelligence (Summer 2004)

21
Reflex Agent with Internal State
function REFLEX-AGENT-WITH-STATE (percept)
returns action static state, a description
of the current world state rules, a
set of condition-action rules state ?
UPDATE-STATE (state, percept) rule ?
RULE-MATCH (state, rules) action ?
RULE-ACTION rule state ? UPDATE-STATE
(state, action) return action
A reflex agent with internal state works by
finding a rule whose condition matches the
current situation (as defined by the percept and
the stored internal state) and then doing the
action associated with that rule.
CPSC 420 Artificial Intelligence (Summer 2004)

22
Reflex Agent with Internal State

• Encode internal state of the world to remember
  the past as contained in earlier percepts
• Needed because sensors do no usually give the
  entire state of the world at each input, so
  perception of the environment is captured over
  time. State used to encode different world
  states that generate the same immediate percept
• Requires ability to represent change in the
  world with/without the agent
• One possibility is to represent just the latest
  state, but then cannot reason about hypothetical
  courses of action

23
Agent with Explicit Goals
CPSC 420 Artificial Intelligence (Summer 2004)

24
Agent with Explicit Goals

• Choose actions so as to achieve a (given or
  computed) goal a description of desirable
  situations. e.g. where the taxi wants to go
• Keeping track of the current state is often not
  enough need to add goals to decide which
  situations are good
• Deliberative instead of reactive
• May have to consider long sequences of possible
  actions before deciding if goal is achieved
  involves considerations of the future, what will
  happen if I do? (search and planning)
• More flexible than reflex agent. In the reflex
  agent, the entire database of rules would have to
  be rewritten

CPSC 420 Artificial Intelligence (Summer 2004)

25
Utility-Based Agent
CPSC 420 Artificial Intelligence (Summer 2004)

26
Utility-Based Agent

• When there are multiple possible alternatives,
  how to decide which one is best?
• A goal specifies a crude destination between a
  happy and unhappy state, but often need a more
  general performance measure that describes
  degree of happiness
• Utility function U S ? ? indicating a measure
  of success when at a given state
• Allows decisions comparing choice between
  conflicting goals, and choice between likelihood
  of success and importance of goal (if achievement
  is uncertain)

CPSC 420 Artificial Intelligence (Summer 2004)

27
Learning

• What if the agent dont know much about the
  environment when you start or if the environment
  changes?
• Learning!
• Part of the agents job is to use sequences of
  percepts to estimate the missing details in the
  world dynamics
• Learning is not very different from perception,
  they both find out about the world based on
  experience
• Perception short time scale
• Learning long time scale

CPSC 420 Artificial Intelligence (Summer 2004)

28
Learning Agent
CPSC 420 Artificial Intelligence (Summer 2004)

29
Classes of Environments
CPSC 420 Artificial Intelligence (Summer 2004)

30
Classes of Environments
Accessible (observable) The agents sensory
apparatus gives it access to the complete state
of the environment Deterministic The next
state of the environment is completely determined
by the current state and the actions selected by
the agent Subjective non-determinism -
Limited memory (poker) - Too complex
environment to model directly (weather, dice)
- Inaccessibility Episodic The agents
experience is divided into independent
episodes, each episode consisting of agent
perceiving and then acting. Quality of action
depends just on the episode itself, because
subsequent episodes do not depend on what actions
occur in previous episodes. ? Do not need to
think ahead
CPSC 420 Artificial Intelligence (Summer 2004)

31
Classes of Environments
Static If the environment can change while the
agent is deliberating, then the environment is
dynamic otherwise its static. Discrete
There are a limited number of distinct, clearly
defined percepts and actions we say that
environment is discrete. Multi-agent
Need to worry about time
Dynamic
Need to observe while deliberating
The hardest case is partially observable,
stochastic, sequential, dynamic, continuous, and
multiagent.
CPSC 420 Artificial Intelligence (Summer 2004)

32
CPSC 420 Artificial Intelligence (Summer 2004)

33
Running the agents and the environment
procedure RUN-ENVIRONMENT (state, UPDATE-FN,
agents, termination) inputs state, the
initial state of the environment
UPDATE-FN, function to modify the environment
agents, a set of agents
termination, a predicate to test when we are
done repeat for each agent in agents
do PERCEPTagent ?
GET-PERCEPT(agent,state) end
for each agent in agents do
ACTIONagent ? PROGRAMagent (PERCEPTagent)
end state ? UPDATE-FN(actions,
agents, state) until termination (state)
CPSC 420 Artificial Intelligence (Summer 2004)