Introduction to Rational Agents

1
Introduction to (Rational) Agents

• Kee-Eung Kim
• KAIST Computer Science

2
Agents Environment

• An agent is anything that can be viewed as
  perceiving its environment through sensors and
  acting upon the environment through actuators
• Human agents
• Robotic agents
• Software agents
• The world model
• A the Action space
• P the Percept space
• E the Environment A ? P
• S internal state may not be visible to the
  agent
• Perception function S ? P
• World dynamics S ? A ? S
• Agent function P ? A (mapping of sequences of
  percepts to actions)

PerceptionFunction
p
WorldDynamics
a
s
3
Example Vacuum-Cleaner World

• Percept space location and status, e.g., A,
  dirty
• Action space move left (Left), move right
  (Right), suck up the dirt (Suck), or do nothing
  (NoOp)

4
Example A Vacuum-Cleaner Agent

• An example of agent function
• What is the right way to fill out the table?
• What is the correct answer for the right-hand
  column?
• What makes an agent good or bad, intelligent or
  stupid?

5
Rational Agents

• Rationality to be most successful
• Performance measure that defines the degree of
  successOften specified by the utility function
  U S ? ? (or S ? ?)
• Everything that the agent has perceived so far
  (percept sequence)
• Depends on the agents knowledge of the
  environment
• Depends on the actions that the agent can perform
• Designing a rational agent
• Find agent function P ? A (mapping of sequences
  of percepts to actions),
• Which maximizes the utility of the resulting
  sequence of states (each action results in
  transition from one state to the next state),
• Constrained by the agents knowledge of the
  environment and the actions that the agent is
  able to execute.
• rationality ? omniscience, rationality ?
  success
• Exploration, learning, autonomy

6
Specification of Task Environments

• Task environments problems, rational agents
  solutions
• Specification by PEAS Performance measure,
  Environment, Actuators, Sensors
• Automated taxi driver
• P Safe, fast, legal, comfortable trip, maximize
  profits
• E Roads, other traffic, pedestrians, customers
• A Steering wheel, accelerator, break, signal,
  horn, display
• S Cameras, sonar, speedometer, GPS, odometer,
  engine sensors, keyboard
• Satellite image analysis system
• P Correct image categorization
• E Downlink from orbiting satellite
• A Display categorization of scene
• S Color pixel arrays

7
Environment Types

• Fully observable (vs. partially observable) An
  agent's sensors give it access to the complete
  state of the environment at each point in time.
• Deterministic (vs. stochastic) The next state of
  the environment is completely determined by the
  current state and the action executed by the
  agent. If the environment is deterministic except
  for the actions of other agents, environment is
  called strategic.
• Episodic (vs. sequential) The agent's experience
  is divided into atomic episodes (each episode
  consists of the agent perceiving and then
  performing a single action), and the choice of
  action in each episode depends only on the
  episode itself.

8
Environment Types

• Static (vs. dynamic) The environment is
  unchanged while an agent is deliberating. (The
  environment is semi-dynamic if the environment
  itself does not change with the passage of time
  but the agent's performance score does)
• Discrete (vs. continuous) A limited number of
  distinct, clearly defined percepts and actions.
• Single agent (vs. multiagent) An agent operating
  by itself in an environment.
• Competitive multiagent environment
• Cooperative multiagent environment

9
Environment Type Examples
10
Agent Program Naïve Example
Function TABLE-DRIVEN-AGENT(percept) returns an
action static percepts, a sequence initially
empty table, a table of actions, indexed by
percept sequence append percept to the end of
percepts action ? LOOKUP(percepts,
table) return action

• Problems? Can you see why it is naïve?

11
Simple Reflex Agents

• Action selection based on the current percept,
  ignoring the rest of the percept history
• Correct decision can be made only when?
• Randomized simple reflex agents?

Function SIMPLE-REFLEX-AGENT(percept) returns an
action static rules, a set of condition-action
rules state ? INTERPRET-INPUT(percept) rule ?
RULE-MATCH(state, rule) action ?
RULE-ACTIONrule return action
12
Model-based Reflex Agents

• Maintain some sort of internal state that depends
  on the percept history
• Keep track of the part of the world it cant see
  now

Function REFLEX-AGENT-WITH-STATE(percept) returns
an action static rules, a set of
condition-action rules state, a description of
the current world state action, the most recent
action, initially none state ?
UPDATE-STATE(state, action, percept) rule ?
RULE-MATCH(state, rule) action ?
RULE-ACTIONrule return action
13
Goal-based Agents

• The agent needs a goal to know which situations
  are desirable
• Major difference future is taken into account
• Typically investigated in search and planning
  research
• Flexible in the sense that the knowledge that
  supports its decisions is represented explicitly
  and can be modified

14
Utility-based Agents

• Among many ways to achieve the goal, find that
  are quicker, safer, more reliable, or cheaper
  than others
• Take into account utility function S ? ?
• degree of happiness

15
Learning Agents

• All previous agent-programs describe methods for
  selecting actions.
• Yet it does not explain the origin of these
  programs
• Learning mechanisms can be used to perform this
  task
• Teach them instead of instructing them
• Advantage is the robustness of the program toward
  initially unknown environments

16
Summary

• Agents and agent functions
• Performance measure and rational agents
• Specification of task environment through PEAS
• Agent programs that implement agent functions
• Simple reflex agents
• Model-based reflex agents
• Goal-based agents
• Utility-based agents
• Learning agents