INSYS 300 Natural Language Processing

1
INSYS 300 Natural Language Processing
2
Eliza
3
What is a language

• Symbols and rules for combining them
• Math
• Logic
• Regular expressions
• aaab ab
• Computer languages
• Is Natural Language a language in this sense?

4
Components of Natural Language (the traditional
model)

• Letters/Phonemes/Syllables
• Words (Concepts)
• Parts of Speech
• Phrases (Concepts)
• Sentences
• Discourse/Rhetoric/Conversation

5
Ontologies and Knowledge-bases

• Can we define all word associations (UMLS)

6
Levels of Natural Language Analysis

• Structure (Grammar)
• Semantics (Meaning)
• Pragmatics (Function)

7
Phrase Structure
S
NP
VP
8
Some problems with the basic model of language

• The levels are not cleanly separated
• Some parts of speech are ambiguous
• What exactly is an adverb

9
Natural Language Processing

• Component (Word) Identification
• Segmentation
• OCR, hand-writing recognition,
• Lexical semantics
• Dis-ambiguation
• Parsing
• Finding structure matching a grammar
• Could parse a program or a sentence
• Applications
• Question answering
• Summarization
• Language understanding
• Translation

10
Lexical Semantics

• What words mean
• Why dont dictionaries agree about the
  definitions of words?

11
Some Strategies and Issues for Parsing

• Strategies
• Transition networks
• Good for compiler design
• Constraint satisfaction
• Probability
• Difficulties
• Anaphora (pronoun reference)
• Pat and Jill went up the hill he fell down
• We need to know a lot to figure out he is Pat
• Agreement (they fell down)
• Garden path
• The man who hunts ducks out on weekends.
• True ambiguity
• I saw the man on the hill with the binoculars

12
Semantics

• What does something mean?
• Is there an absolute meaning?
• Given the words and the rules for combing them
  this is the implication
• Does meaning depend context, on the participants?
• Meaning is revealed in behavior
• The meaning is the way that people react to it
• Is meaning all relative? Its probably still
  based in individual and social needs

13
Question Answering

• Question analysis
• What type of answer is needed
• Yes/no, persons name, action
• What is the question about
• Who is the current king of France? (no such
  person)
• Convert question to query
• Where to find the answer
• Documents
• Databases/Knowledgebases/Ontologies
• Synthesize a response (Generation)
• Assemble facts
• Smooth the presentation

14
How do People Use Language(Pragmatics)

• Things arent always what they seem
• Its drafty in here may mean
• please close the door
• Highly contextualized
• E.g., Adjusting speech to the ambient noise level

15

• Assertion
• Would you spell your name?
• Request/Directive
• Would you open the window?
• Short Answer
• Verification Is it raining?
• Disjunctive Are you happy or sad?
• Concept completion Who did this?
• Feature specification What color is the
  dress?
• Quantification How many people were at the
  last class?
• Long Answer
• Definition What is an oxymoron''?
• Example Can you give me an example of
  electron bonding?
• Comparison What's the difference between a
  beagle and a terrier?
• Interpretation What does this mean?
• Causal antecedent What were the causes of
  the Civil War?
• Causal consequence What happened when you
  got elected?
• Goal orientation What were you trying to
  accomplish?
• Instrumental/procedural What are items for
  the agenda?

16
Some Strategies for Processing Language

• Logic
• Knowledgebases
• Statistical machine learning
• Needs a lot of training data

17
Machine Translation

• Cant just translate word by word or even phrase
  by phrase
• Art theft (English) gt Flight of art (French)
• Interlingua
• A common (semantic) representation for all
  language

18
Discourse Processing

• Creating explanations
• Narratives

19
Summarization

• Simple model gt Distill all the facts and then
  reconstruct them as needed
• Its difficult to do this because of context
• Extracts find the best set of sentences
• tfidf model of summarization
• Abstracts vs summaries
• Abstracts and Current awareness
• Indicative vs. informative abstracts