Using Machine Learning to Discover and Understand Structured Data

1
Using Machine Learning to Discover and Understand
Structured Data

• William W. Cohen
• Machine Learning Dept. and Language Technologies
  Inst.
• School of Computer Science
• Carnegie Mellon University

2
Outline

• Information integration
• Some history
• The problem, the economics, and the economic
  problem
• Soft information integration
• Concrete uses of soft integration
• Classification
• Collaborative filtering
• Set expansion

3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
WHIRL project (1997-2000)

• WHIRL initiated when at ATT Bell Labs

ATT Research
ATT Labs - Research
ATT Research
ATT Labs
ATT Research Shannon Laboratory
ATT Shannon Labs
13
When are two entities the same?

• Bell Labs
• Bell Telephone Labs
• ATT Bell Labs
• AT Labs
• ATT LabsResearch
• ATT Labs Research, Shannon Laboratory
• Shannon Labs
• Bell Labs Innovations
• Lucent Technologies/Bell Labs Innovations

1925
History of Innovation From 1925 to today, ATT
has attracted some of the world's greatest
scientists, engineers and developers.
www.research.att.com
Bell Labs Facts Bell Laboratories, the research
and development arm of Lucent Technologies, has
been operating continuously since 1925
bell-labs.com
14
In the once upon a time days of the First Age of
Magic, the prudent sorcerer regarded his own true
name as his most valued possession but also the
greatest threat to his continued good health,
for--the stories go--once an enemy, even a weak
unskilled enemy, learned the sorcerer's true
name, then routine and widely known spells could
destroy or enslave even the most powerful. As
times passed, and we graduated to the Age of
Reason and thence to the first and second
industrial revolutions, such notions were
discredited. Now it seems that the Wheel has
turned full circle (even if there never really
was a First Age) and we are back to worrying
about true names again The first hint Mr.
Slippery had that his own True Name might be
known--and, for that matter, known to the Great
Enemy--came with the appearance of two black
Lincolns humming up the long dirt driveway ...
Roger Pollack was in his garden weeding, had been
there nearly the whole morning.... Four heavy-set
men and a hard-looking female piled out, started
purposefully across his well-tended cabbage
patch. This had been, of course, Roger Pollack's
great fear. They had discovered Mr. Slippery's
True Name and it was Roger Andrew Pollack
TIN/SSAN 0959-34-2861.
15
When are two entities are the same?
Buddhism rejects the key element in folk
psychology the idea of a self (a unified
personal identity that is continuous through
time) King Milinda and Nagasena (the Buddhist
sage) discuss personal identity Milinda
gradually realizes that "Nagasena" (the word)
does not stand for anything he can point to
not the hairs on Nagasena's head, nor the hairs
of the body, nor the "nails, teeth, skin,
muscles, sinews, bones, marrow, kidneys, ..."
etc Milinda concludes that "Nagasena" doesn't
stand for anything If we can't say what a person
is, then how do we know a person is the same
person through time? There's really no you,
and if there's no you, there are no beliefs or
desires for you to have The folk psychology
picture is profoundly misleading and believing it
will make you miserable. -S. LaFave
16
(No Transcript)
17
Deduction via co-operation
User

• Economic issues
• Who pays for integration? Who tracks errors
  inconsistencies? Who fixes bugs? Who pushes for
  clarity in underlying concepts and object
  identifiers?
• Standards approach ? publishers are responsible
  ? publishers pay
• Mediator approach 3rd party does the work,
  agnostic as to cost

Integrated KB
Site1
Site3
Site2
KB1
KB3
KB2
Standard Terminology
18
Traditional approach
Linkage
Queries
Uncertainty about what to link must be decided by
the integration system, not the end user
19
WHIRL approach
Strongest links those agreeable to most users
Weaker links those agreeable to some users
even weaker links
20
WHIRL approach
Link items as needed by Q
Incrementally produce a ranked list of possible
links, with best matches first. User (or
downstream process) decides how much of the list
to generate and examine.
21
WHIRL queries

• Assume two relations
• review(movieTitle,reviewText) archive of reviews
• listing(theatre, movieTitle, showTimes, ) now
  showing

22
WHIRL queries

• Find reviews of sci-fi comedies movie domain
• FROM review SELECT WHERE r.textsci fi comedy
• (like standard ranked retrieval of sci-fi
  comedy)
• Where is that sci-fi comedy playing?
• FROM review as r, LISTING as s, SELECT
• WHERE r.titles.title and r.textsci fi comedy
• (best answers titles are similar to each other
  e.g., Hitchhikers Guide to the Galaxy and The
  Hitchhikers Guide to the Galaxy, 2005 and the
  review text is similar to sci-fi comedy)

23
WHIRL queries

• Similarity is based on TFIDF? rare words are most
  important.
• Search for high-ranking answers uses inverted
  indices.

- It is easy to find the (few) items that match
on important terms - Search for strong matches
can prune unimportant terms
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
Outline

• Information integration
• Some history
• The problem, the economics, and the economic
  problem
• Soft information integration
• Concrete uses of soft integration
• Classification
• Collaborative filtering
• Set expansion

29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
Outline

• Information integration
• Some history
• The problem, the economics, and the economic
  problem
• Soft information integration
• Concrete uses of soft integration
• Classification
• Collaborative filtering
• Set expansion

40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
Outline

• Information integration
• Some history
• The problem, the economics, and the economic
  problem
• Soft information integration
• Concrete uses of soft integration
• Classification
• Collaborative filtering
• Set expansion using generalized notion of
  similarity

45
Recent work non-textual similarity
Christos Faloutsos, CMU
William W. Cohen, CMU
cohen
cmu
william
w
dr
Dr. W. W. Cohen
George H. W. Bush
George W. Bush
46
Recent work

• Personalized PageRank aka Random Walk with
  Restart
• Similarity measure for nodes in a graph,
  analogous to TFIDF for text in a WHIRL database
• natural extension to PageRank
• amenable to learning parameters of the walk
  (gradient search, w/ various optimization
  metrics)
• Toutanova, Manning NG, ICML2004 Nie et al,
  WWW2005 Xi et al, SIGIR 2005
• various speedup techniques exist
• queries
• Given type t and node x, find yT(y)t and yx

47
Learning to Search Email
Einat Minkov, CMU Andrew Ng, Stanford
SIGIR 2006, CEAS 2006, WebKDD/SNA 2007
CALO
Term In Subject
Sent To
William
graph
proposal
CMU
6/17/07
6/18/07
einat_at_cs.cmu.edu
48
Tasks that are like similarity queries
Person namedisambiguation
term andy file msgId
person
Threading
file msgId

• What are the adjacent messages in this thread?
• A proxy for finding more messages like this one

file
Alias finding
What are the email-addresses of Jason ?...
term Jason
email-address
Meeting attendees finder
Which email-addresses (persons) should I notify
about this meeting?
meeting mtgId
email-address
49
Learning to search better
Task T (query class)
Query q

Query a
Query b
Rel. answers a
Rel. answers b
Rel. answers q
GRAPH WALK

• node rank 1
• node rank 2
• node rank 3
• node rank 4
• node rank 10
• node rank 11
• node rank 12
• node rank 50

• node rank 1
• node rank 2
• node rank 3
• node rank 4
• node rank 10
• node rank 11
• node rank 12
• node rank 50

• node rank 1
• node rank 2
• node rank 3
• node rank 4
• node rank 10
• node rank 11
• node rank 12
• node rank 50

50
Learning
Node re-ordering
Feature generation
Learnre-ranker
Re-rankingfunction
train task
Graph walk
51
Learning Approach
train task
test task
Voted Perceptron RankSVM PerceptronCommittees
Joacchim KDD 2002, Elsas et al WSDM 2008
Collins Koo, CL 2005 Collins, ACL 2002
52
Learning approaches
Edge weight tuning
Graph walk
Weightupdate
Theta
53
Learning approaches
Edge weight tuning
Diligenti et al, IJCAI 2005 Toutanova Ng,
ICML 2005
Graph walk
Weightupdate
Theta
Graph walk
task
Question which is better?
54
Results on one task
Mgmt. game
PERSON NAME DISAMBIGUATION
55
Results on several tasks (MAP)

Namedisambiguation






Threading










Alias finding
56
Set Expansion using the Web

• Canon
• Nikon
• Olympus

Richard Wang, CMU

• Pentax
• Sony
• Kodak
• Minolta
• Panasonic
• Casio
• Leica
• Fuji
• Samsung

• Fetcher download web pages from the Web
• Extractor learn wrappers from web pages
• Ranker rank entities extracted by wrappers

57
The Extractor

• Learn wrappers from web documents and seeds on
  the fly
• Utilize semi-structured documents
• Wrappers defined at character level
• No tokenization required thus language
  independent
• However, very specific thus page-dependent
• Wrappers derived from document d is applied to d
  only

58
(No Transcript)
59
Building a Graph
ford, nissan, toyota
Wrapper 2
find
northpointcars.com
extract
curryauto.com
derive
chevrolet 22.5
volvo chicago 8.4
Wrapper 1
honda 26.1
Wrapper 3
Wrapper 4
acura 34.6
bmw pittsburgh 8.4

• A graph consists of a fixed set of
• Node Types seeds, document, wrapper, mention
• Labeled Directed Edges find, derive, extract
• Each edge asserts that a binary relation r holds
• Each edge has an inverse relation r-1 (graph is
  cyclic)

Minkov et al. Contextual Search and Name
Disambiguation in Email using Graphs. SIGIR 2006
60
(No Transcript)
61
Top three mentions are the seeds
Try it out at http//rcwang.com/seal
62
Relational Set Expansion
Seeds
63
Additional relevant research

• Alon Halevey and friends
• Pay as you go, on the fly, data integration
  (e.g., SIGMOD 98) integrate partially, then
  allow user to perform search to make up for
  inaccuracy of result
• Anhai Doan and friends
• Best effort information extraction (SIGMOD 98)
  write an approximate program for extraction from
  web pages, then allow user to perform search to
  make up for inaccuracy of result
• Semi-structured extensions
• Kushmerics ELIXIR (SIGIR 2001) Bernsteins
  iSPARQL (eg ESWC 2008)
• Soft joins
• Gravano et al WWW2003 Text Joints in an RDMS for
  Web Data Integration
• Bayardo et al, WWW2007 Scaling up all-pairs
  similarity search.
• Koudas et al, SIGMOD 2006 Record linkage
  similarity measures and algorithms (survey)

64
Outline

• Information integration
• Some history
• The problem, the economics, and the economic
  problem
• Soft information integration
• Concrete uses of soft integration
• Classification
• Collaborative filtering
• Set expansion
• Questions?