Introduction: Panel on Data and Information

1
Introduction Panel on Data and Information

• David D. Lewis
• Ornarose, Inc.
• David D. Lewis Consulting
• Talk to be presented at Workshop on Strategic
  Research Directions in AI, Cornell Univ.,
  26-Jun-03

2
From Data to Information

• NLP, IR, speech recognition, image processing,
  signal processing,
• Converting raw data to form that supports
  information processing
• Database records, assertions, category labels,
  interpretations,
• Typically with a strong(er) semantic model,
  supporting strong(er) inferences

3
Canonicalization

• Ambiguity, synonymy, richness
• and analogous notions for nontextual data
• Limits ability to infer conclusions, find links,
  even take conditional actions (e.g. crude keyword
  spam filters)
• Similar meanings should have similar
  representations as data
• May be interleaved with inference /
  interpretation rather than done first

4
Condensation

• Raw data is bulky and mostly irrelevant to any
  given task, costing computing resources
• NLP text to structured representations
• IR find subset of documents, assign categories
• Image Processing lots of pixels -gt few image
  features

5
Learning C C

• Rules for canonicalization and condensation often
  easier to learn from data than develop manually
• Division of labor
• Human experts label data
• Computer searches for mapping rules
• Now dominant paradigm in IR, computational
  linguistics, vision,