Bridging the StructuredUnstructured Gap

1
Bridging the Structured-Unstructured Gap
entity
abstraction
is-a
person
region
city
scientist
time
district
physicist
year
state
astronomer
hasDigit
isDDDD
Born in New York in 1934 , Sagan wasa noted
astronomer whose lifelong passionwas searching
for intelligent life in the cosmos.
2
Graph Proximity Search

• Graphs with typed nodes and edges ubiquitous
• Score candidates by proximity to match nodes
• Short path from match nodes
• Many parallel paths from match nodes
• PageRank, commute time, escape probability,

3
Problems and Some Solutions

• Annotation and disambiguation
• Add links from token segments to entity catalog
• Learning to rank (KDD2006, ICML2007)
• Learn relative importance of edge types from
  relevance feedback
• Entity search (WWW2006)
• Typical battery life of Lenovo X300 laptop
• Collective ranking of snippets and entities
• Indexing for proximity search (WWW2007)
• Constant query time independent of graph size

4
Scaling Up

• Aggressive open-domain Web annotation
• Entity catalog from WordNet, Wikipedia,
• Search API 2.0 text structure, indexing
• Mining semistructured fact/relation views
• Cloud in our basement!
• 320 cores, 320GB RAM, 120TB disk
• Terabytes of crawled Web data
• Tens of millions of queries from Y and M
• Click trails on URLs and ads
• Supported by Y, M, HP