Intelligent Agents

1
Intelligent Agents state of the art
Aleksander Pivk

• Materials collected at
• America School on Agents and Multi-agent
  Systems(University of Southern California)

2
What is an agent?

• The main point about agents is that they are
  autonomous capable of acting independently,
  exhibiting control over their internal state.
• Thus an agent is a computer system capable of
  autonomous action in some environment.

3
What is an agent?

• Trivial (non-interesting) agents
• thermostat
• An intelligent agent is a computer system capable
  of flexible autonomous action in some
  environment.By flexible, we mean
• reactive
• pro-active
• social.

4
Intelligent Agents and AI

• A little intelligence goes a long way!
• Oren Etzioni, speaking of commercial experience
  of NETBOT, Inc.
• Microsoft Office Assistant

We made our agents dumber and dumberuntil
finally they made some money.
5
Purely Reactive Agents

• Some agents decide what to do without reference
  to their history they base their making
  decision entirely on the present, with no
  reference at all to the past.
• We call such agents purely reactive action S?
  A
• A thermostat is a purely reactive
  agent action(s)

6
Perception

• Introduce the perception system
• The see function is the agents ability to
  observe its env., whereas the action function
  represents the agents decision making process.
• New functions see S ?P maps environment
  states to percepts action P ?A maps
  sequences of percepts to actions

7
Agents with State

• Lets consider agents that maintain state
• Have some internal data structure, used to record
  inf. about the env. state and history.
• Let I be the set of all internal states of the
  agent.
• Functions see S ?P maps environment
  states to percepts action I ?A maps from
  internal states to actions next I ? P ? I
  maps an internal states and percept to IS

8
Application Research Domains

• Learning Agents
• Embodied Agents
• Logics for Agents
• Coordination, Cooperation, Collaboration
• Market-based Multi-agent Systems

9
Learning Agents

• Why should agents learn?
• Learning user and world models, action-to-utility
  mappings, problem solving
• Learning modalities
• from users (observation, interaction, being told)
  
• from other agents (collaborative filtering, from
  experts)
• from experience (supervised, reinforcement,
  probabilistic models)
• Learning techniques
• neural/decision networks, decision
  trees,reinforcement learning, instance based
  learning
• Assistant agents (work effort, productivity)

10
Learning for Information Agents

• Information agents
• Access information from a variety of data sources
• Integrate the data from these sources
• Monitor and provide notifications
• Technical challenges
• Turning semi-structured data into structured data
• Ensuring continued access to the data
• Resolving naming inconsistencies across sources
• Building agents that efficiently execute their
  tasks

11
Country Information Agent

World Governments
Agent
NATO Members
CIA World Factbook
1995
1996
1997
12
Flight Delay Prediction Agent
Yahoo Weather
Agent
13
Real Estate Notification Agent
New Listing 3br 2bath200K
Send EmailNotification
14
Travel Planning Agent
15
Wrapper Induction

• Problem description
• Web sources present data in human-readable format
• take user query
• apply it to data base
• present results in template HTML page
• To integrate data from multiple sources, one must
  first extract relevant information from Web pages
• Task learn extraction rules based on labeled
  examples
• Hand-writing rules is tedious, error prone, and
  time consuming

16
Example of Extraction Task
NAME Casablanca Restaurant STREET 220
Lincoln Boulevard CITY Venice PHONE
[phone]
17
WIEN Kushmerick et al 97, 00

• Assumes items are always in fixed, known order
• Name J. Doe 1 Main 111-1111. ltpgt Name E.
  Poe
• Introduces several types of wrappers
• LR
• Advantages
• Fast to learn extract
• Drawbacks
• Cannot handle permutations and missing items
• Must label entire page

Name

.

18
SoftMealy Hsu Dung, 98

• Learns a transducer
• Advantages
• Also learns order of items
• Allows item permutations missing items
• Uses wildcards (eg, Number, AllCaps, etc)
• Drawback
• Must see all possible permutations

Addr


Name

Phone
Phone
.
19
WHIRL Wrappers Cohen 99

• Learns underlying HTML template
• WHIRL soft logic to measure document similarity
• Name html_table_tr_td
• Address html_table_tr_td_td
• Advantages
• Learns from unlabeled data
• Explicitly exploits HTML structure
• Disadvantage
• Not as expressive as previous ones
• Works only at the level of HTML nodes

20
STALKER Muslea et al, 98 99 01

• Hierarchical wrapper induction
• Decomposes a hard problem in several easier ones
• Extracts items independently of each other
• Each rule is a finite automaton
• Advantages
• Powerful extraction language (eg, embedded list)
• One hard-to-extract item does not affect others
• Disadvantage
• Does not exploit item order (sometimes may help)

21
Extraction Rules
Extraction rule sequence of landmarks
SkipTo(Phone) SkipTo(ltigt)
SkipTo(lt/igt)
Name Joels ltpgt Phone ltigt [phone]
lt/igtltpgt Review
22
More about Extraction Rules
Name Joels ltpgt Phone ltigt [phone]
lt/igtltpgt Review
Name Kims ltpgt Phone (toll free) ltbgt (800)
757-1111 lt/bgt
Name Kims ltpgt Phoneltbgt [phone]
lt/bgtltpgtReview
Start EITHER SkipTo( Phone ltigt )
OR SkipTo( Phone ) SkipTo( ltbgt)
23
Learning the Extraction Rules
GUI
Inductive Learning System
Extraction Rules
Labeled Pages
24
Example of Rule Induction
Training Examples

• Name Del Taco ltpgt Phone (toll free) ltbgt ( 800
  ) 123-4567 lt/bgtltpgtCuisine ...
• Name Burger King ltpgt Phone ( 310 ) 987-9876
  ltpgt Cuisine

Initial candidate SkipTo( ( )
25
Example of Rule Induction
Training Examples

• Name Del Taco ltpgt Phone (toll free) ltbgt ( 800
  ) 123-4567 lt/bgtltpgtCuisine ...
• Name Burger King ltpgt Phone ( 310 ) 987-9876
  ltpgt Cuisine

Initial candidate SkipTo( ( )

SkipTo(Phone) SkipTo() SkipTo( ( )
...
SkipTo( ltbgt ( ) ... SkipTo(Phone)
SkipTo( ( ) ... SkipTo() SkipTo(()
26
Multi-view Learning
Two ways to find start of the phone number
SkipTo( Phone )
BackTo( ( Number ) )
Name KFC ltpgt Phone [phone] ltpgt
Review Fried chicken
27
Co-Testing

-
Labeled data
Unlabeled data
28
Co-Testing for Wrapper Induction
BackTo( (Number) )
SkipTo( Phone )
Name Joels ltpgt Phone [phone]
ltpgtReview ...
Name Kims ltpgt Phone [phone]
ltpgtReview ...
29
Embodied Agents
30
Embodied Agents

• Animated agent research integrates
• Artistic animation
• Computer graphics
• Intelligent agents
• Why build animated agents?
• For more effective communication
• For artistic effect
• As models for robotic or human agents
• When behavior cannot be scripted
• e.g., due to interactions with people and other
  agents
• In agents, we begin to see dynamic models of
  thought and action

31
Animated Pedagogical Agents

• Animated characters that
• Interact with students in learning envs.
• Help keep learning on track
• Act as guides, tutors, teammates
• Engage in instructional dialog
• Enhance motivation and interest
• APAs require
• Realistic, lifelike behavior
• A rich set of cognitive and social abilities for
  effective learning

32
Steve An Embodied Intelligent Agent for Virtual
Environments

• J. Rickel, L. Johnson, M. Thiebaux, et al.
• 3D agent that interacts with students in virtual
  environments
• Can work together with multiple students and
  multiple users

33
Steves Architecture (detailed)
34
Mission Rehearsal Exercise
35
Co. Co. Co.
36
Example Hidden Pictures

• Simple (visual) search task
• How would YOU work as a part of a team to solve
  it ?

37
Market-based MAS

• Marketspace class of agent interaction env.
• What you need to know
• Economic foundations and principles
• Game theory
• Price system (general equlibrium)
• Auction theory
• Design issues and experience
• Market models and mechanisms
• Trading Agents

38
Business games

• http//www.cmu.edu/comlabgames-------------------
  ----------------------------------------At the
  comlabgames website, www.cmu.edu/comlabgames,
  there are three modules for designing, playing
  and analyzing the experimental outcomes of games
  two person strategic form games, multi-person
  extensive form games, and auctions and markets.
  The original comlabgames website is visited on
  average 50,000 of time each week, and is linked
  to hundreds of other sites. UCLA mirrors the
  original site. Comlabgames is very easy to use,
  and the students just bring their laptops to
  class, design the games, run them and then
  analyze the data.
• Vesna Prasnikar, Marko Grobelnik

39
Logics for Agents

• Symbolic Reasoning
• an agent contains an explicitly represented,
  symbolic model of the world
• makes decisions (action to take) via symbolic
  reasoning
• problems
• transduction how to translate the real world in
  accurate, adequate symbolic description (speech
  understanding)
• representation/reasoning how to symbolically
  represent inf., and how to get agents to reason
  with this inf. (planning)
• Theorem Proving Agents agent decides what to do
  by using logic to encode the theory stating the
  best action in any situation (predicates rules)
  
• Agent oriented programming AGENT0 and PLACA

40
Logics for Agents

• Practical Reasoning (BDI Logic)
• is a matter of weighting conflicting
  considerations for and against competing options,
  where the relevant considerations are provided by
  the agent desires/values about and what the agent
  believes (Bratman).
• directed towards actions (theoretical towards
  belief)
• consists of two activities
• deliberation deciding what state of affairs we
  want to achieve
• means-end reasoning deciding how to achieve them
• implemented BDI agents IRMA, PRS, Desiderata,
  LORA