ICS481 Artificial Intelligence

1
ICS481 Artificial Intelligence

• Dr. Ken Cosh
• Lecture 10

2
This week

• Natural Language Understanding
• Getting the meaning of text and speech.
• Beyond pattern matching (such as with search
  engines)
• Focus on Syntax

3
Why NLU?

• Interfaces to databases (weather, financial)
• Automated Customer Service (banking, travel)
• Voice control of machines (PCs, Cars)
• Grammar and Style checking
• Summarisation (news, manuals)
• Email Routing
• Smarter Web Search
• Translating documents
• Etc.

4
Levels of Language Analysis

• Phonetics Sounds ? words
• Morphology Morphemes ? words (jump ed
  jumped)
• Syntax Word sequence ? sentence structure
• Semantics Sentence structure word meaning ?
  sentence meaning.
• Pragmatics Sentence meaning context ? deeper
  meaning
• Discourse and World Knowledge connecting
  sentences and background knowledge to utterances.

5
NLU Architecture
Frequency Spectogram
Speech Recognition
Word Sequence
Syntactic Analysis
Sentence Structure
Semantics Analysis
Partial Meaning
Pragmatics
Sentence Meaning
6
Syntactic Analysis

• Grammar captures the legal structures in a
  sentence
• Parsing involves finding legal structures for a
  sentence
• The result is a parse tree

7
Parse Tree
S
NP
VP
VP
VP
N
V
NP
NP
P
Art
N
N
John gave the book to Mary
8
Grammars

• Grammars are a set of rules for rewriting strings
  of symbols.
• S ? NP VP
• NP ? Name
• NP ? Art N
• Name ? John
• Art ? the
• N ? book

9
Grammars

• S ? NP VP
• The string S can be rewritten as a noun phrase NP
  and a verb phrase VP
• NP ? Name
• NP ? Art N
• A noun phrase can be rewritten either as a Name
  or as an Art(icle) followed by a N(oun)

10
Legal Grammar

• The sentence is legal if we can find a set of
  rewrite rules that, starting from the symbol S,
  generate the sentence. This is called parsing
  the sentence.
• The sequence of rules applied also give us the
  parse table.

11
Good Grammars

• Differentiate between correct sentences and
  incorrect sentences.
• The boy hit the ball
• The hit boy the ball
• Assigns meaningful structure to the sentences
• (The boy) (hit the ball)
• (The) (boy hit) (the ball)

12
Good Grammars

• Are compact and modular e.g. all these NPs can
  be used in any context in which NPs are allowed.
• NP ? Name John
• NP ? Art N the boy
• NP ? Art Adj N the tall girl
• NP ? Art N that VP the dog that barked
• In this case NP is a non-terminal symbol and
  the is a terminal symbol.

13
Regular Grammars

• Regular grammars are the most general grammars,
  where non terminal symbols can be broken down as
  follows
• A ? x or A ? xB, where A B are non-terminal
  symbols and x is a terminal symbol.
• NP ? Name
• NP ? Art N
• The following are all legal
• The cat likes tuna
• The cat the dog chased likes tuna
• The cat the dog the rat bit chased likes tuna!

14
Context Free Grammars

• Context free grammars are slightly more general.
  
• Regular Grammars are also Context Free.
• Rules allow non-terminal symbols to be broken
  into a string of terminal and non-terminal
  symbols.
• A ? ?
• The non-terminal symbol appears on its own on the
  lhs, and can be broken down independent of the
  context in which it appears.

15
Context Sensitive Grammar

• In context sensitive grammars, the context in
  which the non-terminal symbol appears, affects
  the way it can be broken down.
• Much of the structure of Natural Language can
  still be captured by the simpler context free
  rules, so we will concentrate on that.

16
A Simple Context Free Grammar

• S ? NP VP
• S ? S Conjunction S
• NP ? Pronoun
• NP ? Name
• NP ? Article Noun
• NP ? Number
• NP ? NP PP
• NP ? NP RelClause
• VP ? Verb
• VP ? Verb NP
• VP ? Verb Adj
• VP ? VP PP
• PP ? Prep NP
• RelClause ? that VP

Article ? the a an this that Prep ? to
in on near Conjunction ? and or
but Pronoun ? I you he me Noun ? book
flight meal Name ? Ken Chiang Mai Verb ?
book include prefer Adj ? first earliest
cheap
17
Very Simple Example

• S ? NP VP
• NP ? Name
• VP ? Verb
• Name ? Ken
• Verb ? Taught

18
Slightly More Advanced

• S ? NP VP
• NP ? Article Noun
• Article ? The
• Noun ? teacher
• VP ? VP PP
• VP ? Verb NP
• Verb ? taught
• NP ? Noun
• Noun ? A.I.
• PP ? Prep NP
• Prep ? to
• NP ? Noun
• Noun ? students

19
Backward Chaining vs Forward Chaining

• Backward Chaining is a top down approach, as we
  have used so far, beginning with S and working
  down to words.
• Forward Chaining starts with words and works
  bottom up to S.

20
Backward Chaining

• The teacher taught A.I. to students.

S
NP
VP
Art
Noun
VP
PP
Verb
NP
Prep
NP
Name
Noun
The teacher taught A.I. to
students.
21
Top Down Parsing

• The teacher taught A.I. to students.
• S ? NP VP (Could Work)
• NP ? Pronoun (Pronoun ? The) (1)
• NP ? Name (Name ? The) (2)
• NP ? Article Noun (Article The) (3)
• (Noun teacher)
• VP ? Verb (Verb taught but)(4)
• VP ? Verb NP (Verb taught) (5)
• NP ? Pronoun (Pronoun ? A.I.)
• NP ? Name (Name A.I.) (6)

22
Top Down Parsing - Problems

• Generates Sub Trees without checking input.
• NP ? Pronoun is tested when input is Ken, etc.
• Left-recursive rules lead to infinite loops
• NP ? NP PP
• When looking for an NP where there isnt one,
  this rule will loop forever.
• Grammar should be rewritten to avoid such
  recursion.
• Repeated Parsing of subtrees
• Some work can be saved by storing intermediate
  data.
• VP ? Verb leads to multiple processing of
  incorrect branch.

23
Bottom Up Parsing

• John Gave the Book to Mary

24
Bottom Up Parsing
Name
Verb
Art
N
V
Prep
Name

• John Gave the Book to Mary

25
Bottom Up Parsing
NP
VP
NP
NP
Name
Verb
Art
N
V
Prep
Name

• John Gave the Book to Mary

26
Bottom Up Parsing
S
VP
PP
NP
VP
NP
NP
Name
Verb
Art
N
V
Prep
Name

• John Gave the Book to Mary

27
Bottom Up Parsing
S
VP
S
VP
PP
NP
VP
NP
NP
Name
Verb
Art
N
V
Prep
Name

• John Gave the Book to Mary

28
Bottom Up Parsing
S
S
VP
S
VP
PP
NP
VP
NP
NP
Name
Verb
Art
N
V
Prep
Name

• John Gave the Book to Mary

29
Bottom Up Parsing
S
S
VP
S
VP
PP
NP
VP
NP
NP
Name
Verb
Art
N
V
Prep
Name

• John Gave the Book to Mary

30
Bottom Up Parsing

• Generates sub trees that cannot be extended to S,
  for example interpreting book as a verb.
• No problems with left recursion, but possible
  problems with empty right hand sides.
• All the facts are saved, avoiding unnecessary
  repetition.

31
Ambiguity

• A big big problem!!!
• Lexical ambiguity Book (V or N?)
• Later semantic ambiguity Bank (V or N or N?)
• Consider breaking this sentence down
• Mary saw John on the hill with a telescope

32
What did Mary see?

• Mary (((saw (John)) (on the hill)) (with a
  telescope))
• Mary ((saw (John)) (on the hill)) (with a
  telescope)))
• Mary ((saw (John (on the hill))) (with a
  telescope))
• Mary (saw ((John (on the hill)) (with a
  telescope)))
• Mary (saw (John (on the hill (with a telescope))))

33
Grammar too simple?

• Subjective / Objective case
• Me read the book
• They reads the book
• Sub categories
• John put the box in the corner
• the verb put expects a NP and a PP location.
• Movement
• The big man hurriedly put the blue ball in the
  red box
• What did the big man hurriedly put NP? in the
  box?
• Where did the big man hurriedly put the blue
  ball PP??

34
Building Grammar

• The grammar can be expanded by adding
  specificity.
• S ? NP/subj VP
• NP/subj ? Pronoun/subj
• NP/obj ? Pronoun/obj
• VP ? Verb NP/obj
• Combining these rules multiplies the size of the
  grammar.

35
Complex?

• Needs the help of expert linguists to create the
  rules.
• The rules of natural language are more complex
  than programming language.
• Understanding grammar is only one step towards
  understanding natural language.
• Semantics?
• Pragmatics?

36
Alternatives?

• A probabilistic approach
• Using statistical data to understand data.