Internet Resources Discovery IRD

1
Internet Resources Discovery (IRD)

• Shopping Agents

2
ShopBot

• ShopBot is a softbot that carries out comparison
  shopping at Web vendors on a persons behalf.
• It autonomously learns to extract product
  information from the Web vendors.
• ShopBot learns how to query a stores searchable
  product catalog.

3
Example Search Form
4
Shopbot Assumptions

• Navigation simplicity
• virtual stores have simple - user-friendly
  interfaces.
• Uniformity
• product lists have similar structure.
• Separation tenure
• description fields in a list of products will be
  separated
• every product will have a separate line.

5
ShopBot Implementation

• ShopBot consists of two major components
• Learner
• Gets as input the domain description and site
  URL.
• Creates the vendor description for use by
  Shopper.
• Moderately time-consuming works off-line.
• Shopper
• On-line agent, shops for requested products using
  vendors descriptions created by Learner.

6
ShopBot Learner
for each vendor search for indices for each
potential index for each sample product
query on attributes accumulate
responses analyze
Domain Description
URLs of possible vendors
ShopBot Learner
Online vendors
Vendor Description
7
Learner Domain Description

• Example products P1, P2, , Pn
• Attributes of the productsmanufacturer(P1)
  Digital Village, name(P1) Starship
  Titanic, etc
• Shopbot learner uses domain-specific heuristics.

8
Learner Vendor Description

• URL of a searchable index form.
• A function that maps product attributes to the
  form fields.
• Functions for extracting data from pages
  returned
• A function that recognizes failure pages
  (product not found).
• A function that strips header and trailer from
  successful pages.
• A function that extracts a set of product
  descriptions from the remaining text on a
  successful page.

9
Learner Strategy

• Identify an appropriate search form.
• Finds all forms, discards inappropriate forms
  using simple heuristics.
• Determine how to fill in the forms.
• Identify product description formats
• Fills in the form with dummy product to identify
  failure page.
• Identifies header, trailer and the body for
  success page.
• Parse body to a number of product descriptions
  using domain-specific heuristics.

10
ShopBot Buyer
get user request for each vendor go to index
fill in form parse results sort display
Domain Description
VendorDescription
ShopBot Buyer
Online Vendors
GUI
11
The Jango Architecture
http//www.jango.com
2. Jango Server --interprets query
Netbot Server
3. Multiple Sources Queried Simultaneously--Amazo
n.com, --American Library Association...
Client
12
Jango - Example
13
Jango -- example
14
Jango -- example
15
Jango Shopping Agent
16
Shopping Results
17
Another Shopping Example
18
Softbots - Summary

• 1. A meta-service that leverages existing
  services and collates their results.
• 2. It enables a human user to state what he or
  she wants to accomplish.
• 3. It attempts to disambiguate the request and to
  dynamically determine how and where to satisfy
  it.
• 4. It utilizes automatic planning technology to
  dynamically generate the results