Features and Unification

1
Features and Unification

• Follows (Old) Jurafsky Martin, Chapter 11
• Adapted from slides by Kathy McCoy, University of
  Delaware

2
A Simple Context Free Grammar Fragment

• NP ? Det N
• NP ? PropN
• Det ? a, the, this, those
• N ? book, dog, books, dogs
• PropN ? John, Mary
• V ? sneezed, visited, gave eat, eats
• S ? NP VP

• VP ? V
• (John sneezed)
• VP ? V NP
• (John visited Mary)
• VP ? V NP NP
• (John gave Mary a book)
• VP ? V NP PP
• (John gave a book to Mary)

3
Agreement

• Determiner/Noun Agreement
• This dog
• Those dogs
• Subject/Verb Agreement
• This dog eats
• Those dogs eat

• Our grammar also generates
• This dogs
• Those dog
• Our grammar also generates
• This dog eat
• Those dogs eats

4
Encoding Number Agreement in CFGs
Detsing ? this Detpl ? those Nsing ? dog Npl ?
dogs Vsing ? eats Vpl ? eat

• NPsing ? Detsing Nsing
• NPpl ? Detpl Npl
• VPpl ? Vpl NPsing
• VPpl ? Vpl NPpl
• VPsing ? Vsing NPsing
• VPsing ? Vsing NPpl
• Ssing ? NPsing VPsing
• Spl ? NPpl VPpl

5
Subcategorization

• Sneeze John sneezed
• John sneezed the bookNP
• Find Please find a flight to NYNP
• Please find
• Give Give meNPa cheaper fareNP
• Give with a flightPP
• Prefer I prefer to leave earlierTO-VP
• I prefer United has a flightS

6
Possible CFG Solution

• REPLACE
• VP ? V
• VP ? V NP
• VP ? V NP PP

• WITH
• VP ? VIntrans
• VP ? VTrans NP
• VP ? VTransPP NP PP
• VIntrans ? sneeze
• VTrans ? find
• VTransPP ? give

7
Encoding Number Agreement Subcats

• VP ? VIntrans/sing
• VP ? VIntrans/pl
• VP ? VTrans/sing NP
• VP ? VTrans/pl NP
• VP ? VTransPP/sing NP PP
• VP ? VTransPP/pl NP PP

• VIntrans/sing ? sneezes
• VIntrans/pl ? sneeze
• VTrans/sing ? finds
• VTrans/pl ? find
• VTransPP/sing ? gives
• VTransPP/pl ? give

But what about I sneeze, you sneeze, he
sneezes.
8
Features, informally

• View both words and grammar non-terminals as
  complex objects, each of which has a set of
  associated property-value pairs (called features)
  that can be manipulated.
• Det num sg ? this
• Det num pl ? those
• N num sg ? dog
• N num pl ? dogs
• Then a grammar can contain
• NP ? Det N but only if Det num N num

9
Feature Agreement

• OK
• NP ? Det N but only if Det num N num
• Better
• NP ? Det num a N num a
• Best
• NP num a ? Det num a N num a
• as well as
• S ? NP num a VP num a

10
Features and Feature Structures

• We can encode these properties by associating
  what are called feature structures with
  grammatical constituents.
• A feature structure is a set of feature-value
  pairs where
• features are atomic symbols
• values are either atomic symbols or (recursively
  embedded) feature structures

11
Example Feature Structures
12
Bundles of Features

• Feature Values can be feature structures
  themselves.
• This is useful when certain features commonly
  co-occur, as number and person.

13
Feature Structures as DAGs
14
Reentrant Structure

• Multiple features in a feature structure can
  share the same value. In this case they share
  structure, not just have the same value.
• Numerical indices indicate the shared value.

15
Reentrant DAGs make this clearer
16
Feature Paths

• It will also be useful to talk about paths
  through feature structures. As in the paths
• ltHEAD AGREEMENT NUMBERgt
• ltHEAD SUBJECT AGREEMENT NUMBERgt

17
Unification I

• Key operations on feature structures
• check the compatibility of two structures
• merge the information in two structures
• We can do both with a single operation called
  Unification.
• Unifying two feature structures produces a new
  feature structure that is more specific (has more
  information) than, or is identical to, each of
  the input feature structures.

18
The Unification Operation U

• Two feature structures can be unified if the
  component features that make them up are
  compatible.
• number sg U number sg number sg
• number sg U number pl fails!
• Structures are compatible if they contain no
  features that are incompatible.
• If so, unification returns the union of all
  feature/value pairs.

19
The Unification Operation

• Number sg U Number
• Number sg U Person 3

Number sg
20
The Unification Operation
U
Subject Agreement Person 3

Agreement Number sg Number sg Subject Agre
ement Person 3
21
The Unification Operation
Head Subject Agreement Number pl
U
Cat S Number sg Agreement
Person 3 Head Subject Agreement
1
1
Fail!
22
Properties of Unification

• Monotonic if some description is true of a
  feature structure, it will still be true after
  unifying it with another feature structure.
• Order independent (commutative) Unifying a set
  of feature structures in any order yields the
  same result.

23
Features, Unification, and Grammars

• To incorporate all this into grammars
• Each constituent has a feature-structure
  associated with it
• Each grammar rule has a (potentially empty) set
  of unification constraints associated with it.
• The set of unification constraints must be
  satisfied for the rule to be satisfied.

24
Unification Constraints

• X0 ? X1 Xn ? Grammar rule
• lt Xi feature path gt
• atomic value
• lt Xi feature path gt
• lt Xk feature path gt

? Set of constraints
25
Agreement

• NP ? Det Nominal
• lt Det AGREEMENT gt lt Nominal AGREEMENT gt
• lt NP AGREEMENT gt lt Nominal AGREEMENT gt
• Noun ? flight
• lt Noun AGREEMENT NUMBER gt SG
• Noun ? flights
• lt Noun AGREEMENT NUMBER gt PL
• Nominal ? Noun
• lt Nominal AGREEMENT gt lt Noun AGREEMENT gt
• Det ? this
• lt Det AGREEMENT NUMBER gt SG

26
Unification and Parsing

• Assume weve augmented our grammar with sets of
  unification constraints.
• What changes do we need to make to a parser to
  make use of them?
• Build feature structures and associate each with
  a subtree
• Unify feature structures as subtrees are created
  from smaller subtrees
• Block ill-formed constituents

27
Unification and Earley Parsing

• With respect to an Earley-style parser
• Build feature structures (represented as DAGs)
  and associate them with states in the chart
• Unify feature structures as states are advanced
  in the chart
• Block ill-formed states from entering the chart

28
Building Feature Structures

• Features of most grammatical categories are
  copied from head child to parent
• (e.g., from V to VP, Nom to NP, N to Nom)
• VP ? V NP
• lt VP HEAD gt lt V HEAD gt
• S ? NP VP
• lt NP HEAD AGREEMENT gt lt VP HEAD AGREEMENTgt
• lt S HEAD gt lt VP HEAD gt

S head NP head agreement
VP head agreement
1
2
2
1
29
Augmenting States with DAGs

• We just add a new field to the representation of
  the states
• S ? . NP VP, 0,0, Dag

30
Example

• NP ? Det . Nominal 0,1, DAG1
• Nominal ? Noun ., 1,2, DAG2

nominalhead noun head agreement
number sg
1
1
31
Review Earleys Algorithm I

• For each rule S?? in the grammar
• Add S??? to chart0,0
• For i 0 to N
• for edge in queuei
• if edge is incomplete and edge.next is a part
  of speech
• scanner(edge)
• if edge is incomplete and edge.next is not a
  POS
• predictor(edge)
• if edge is complete
• completer(edge)

32
Review Earleys Algorithm II
Predictor
Initialization
Scanner
Completer
33
Unifying States and Blocking

• Keep much of the Earley Algorithm the same.
• We want to unify the DAGs of existing states as
  they are combined as specified by the grammatical
  constraints.
• Alter COMPLETER when a new state is created,
  first make sure the individual DAGs unify. If
  so, then add the new DAG (resulting from the
  unification) to the new state.

34
Modifying Earley I

• Completer
• Before Completer adds new states to chart by
  finding states whose dot can be advanced
• (i.e., category of next constituent matches that
  of completed constituent)
• Now Completer will only advance those states if
  their feature structures unify.
• Extend test to enter a state in the chart
• Now DAGs may differ, so old check must be more
  complex
• Dont add states that have DAGs that are more
  specific than states in chart
• Q is new state subsumed by existing states?

35
Modifying Earley II