The Chomsky Hierarchy

1
The Chomsky Hierarchy
2
SentencesThe sentence as a string of
wordsE.g I saw the lady with the
binoculars string a b c d e b f
3

• The relations of parts of a string to each other
  may be different
• I saw the lady with the binoculars
• is stucturally ambiguous
• Who has the binoculars?

4

• I saw the lady with the binoculars
  a b c d e b fI saw the lady with the
  binoculars a b c d e b f

5

• How can we represent the difference?
• By assigning them different structures.
• We can represent structures with 'trees'.
• I
• read
• the book

6

• a. I saw the lady with the binoculars S  
  NP VP  V NP 
  NP PP
• I saw the lady with the binoculars I
  saw the lady with the binoculars

7

• b. I saw the lady with the binoculars S  
  NP VP  VP PP 
  
• I saw the lady with the binoculars I
  saw the lady with the binoculars

8

• birds fly
• S
• NP VP
• N V
• birds fly
• S ? NP VP
• NP ? N
• VP ? V

Syntactic rules
9

• S
•  
• NP VP
• birds fly
• a b
• ab string

10

• S
•  
• A B
• a b
• ab
• S ? A B
• A ? a
• B ? b

11

• Rules
•  
• Assumption
• natural language grammars are a rule-based
  systems
•  
• What kind of grammars describe natural language
  phenomena?
•  
• What are the formal properties of grammatical
  rules?
•  

12

• Chomsky (1957) Syntactic Struc-tures. The Hague
  Mouton
•  
• Chomsky, N. and G.A. Miller (1958) Finite-state
  languages Information and Control 1, 99-112
•  
• Chomsky (1959) On certain formal properties of
  languages. Information and Control 2, 137-167

13
Rules in Linguistics 1. PHONOLOGY /s/ ? ? ?
V ___V Rewrite /s/ as ? when /s/ occurs in
context V ____ V WithV auxiliary nodes,
? terminal nodes
14
Rules in Linguistics  2. SYNTAX S ? NP
VP VP ? V NP ? NRewrite S as NP VP in any
contextWith S, NP, VP auxiliary nodes V,
N terminal node
15

• PHONOLOGY (sound system)
•  
• Maltese Word-final devoicing
•  
• Orthography Pronunciation
• (spelling) (sound)
•  
• Sabet sab sa-bet sap
• Hobza hobz hob-za hops
• Vjaggi vjagg vjag-gi vjacc
•  
• voiced vd voiceless -vd
• b, z, g p, s, c
•  
•  
• vd ? -vd /____
•  
• (for end of word)

16

• MORPHOLOGY (word formation)
• Maltese Progressive assimilation in 3fsg
  imprefective (present)
• Marker for verb in 3rd person feminine singular
  imperfective t- (3fsgimpf she)
•  
• e.g. she breaks t-kisser
• I break n-kisser
•  
• t-kisser t-ressaq
• 3fsg-break 3fsg-move
• she breaks she moves
•  
• s-sakkar d-dur
• 3fsg-lock 3fsg-turn
• she locks she turns
•  
• t-sakkar t-dur
•  
• t ? s,d,etc. /____ s,d,etc.
• cor

17

• SYNTAX (phrase/sentence formation)
•  
• sentence
•  
• The boy kissed the girl
• Subject predicate
• noun phrase verb phrase
• art noun verb noun phrase
•  
• S ? NP VP
• VP ? V NP
• NP ? ART N
•  

18

• SEMANTICS (meaning)
•  
• The lion attacks the hunter
•  
• attack (a, b)
•  
• a ?y attack (y, b)
•  
•  
• ?z ?y attack (y, z) b
•  
• (with a the lion, b the hunter)
•  

19

• Chomsky Hierarchy
•  
• 0. Type 0 (recursively enumerable) languages
• Only restriction on rules left-hand side cannot
  be the empty string ( Ø ? .)
•  
• 1. Context-Sensitive languages -
  Context-Sensitive (CS) rules
•  
• 2. Context-Free languages - Context-Free (CF)
  rules
•  
• 3. Regular languages - Non-Context-Free (CF)
  rules
•  
• 0 ? 1 ? 2 ? 3
• a ? b meaning a properly includes b (a is a
  superset of b),
• i.e. b is a proper subset of a or b is in a

20

• Generative power
• 0. Type 0 (recursively enumerable) languages
• only restriction on rules left-hand side cannot
• be the empty string ( Ø ? .)
• - is the most powerful system
• 3. Type 3(regular language)
• - is the least powerful

21
Superset/subset relation
S1 S2
a c b d f g
a b
S1 is a subset of S2 S2 is a subset of S1
22

• Rule Type 3
•  Name Regular
•  Example Finite State Automata (Markov-process
  Grammar)
•  
• Rule type
• a) right-linear
• A ? xB or
• A ? x
• with
• A, B auxiliary nodes and
• x terminal node
•  
• b) or left-linear
• A ? Bx or
• A ? x
•  
• Generates ambn with m,n ? 1
•  
• Cannot guarantee that there are as many as as
  bs no embedding

23

• A regular grammar for natural language sentences
•  
• S ? the A
•  
• A ? cat B
• A ? mouse B
• A ? duck B
•  
• B ? bites C
• B ? sees C
• B ? eats C
•  
• C ? the D
•  
• D ? boy
• D ? girl
• D ? monkey
•  
• the cat bites the boy

24

• Regular grammars
•  
• Grammar 1 Grammar 2
• A ? a A ? a
• A ? a B A ? B a
• B ? b A B ? A b
•  
• Grammar 3 Grammar 4
• A ? a A ? a
• A ? a B A ? B a
• B ? b B ? b
• B ? b A B ? A b
•  
• Grammar 5 Grammar 6
• S ? a A A ? A a
• S ? b B A ? B a
• A ? a S B ? b
• B ? b b S B ? A b
• S ? ? A ? a

25

• Grammars non-regular
•  
• Grammar 6 Grammar 7
• S ? A B A ? a
• S ? b B A ? B a
• A ? a S B ? b
• B ? b b S B ? b A
• S ? ?

26

• Finite-State Automaton
• article noun
• NP NP1 NP2
• adjective

27

• NP
• article NP1
• adjective NP1
• noun NP2
• NP ? article NP1
• NP1 ?adjective NP1
• NP1 ? noun NP2

28

• A parse tree
• S root node
• NP VP non-
• terminal
• N V NP nodes
• DET N
• terminal nodes

29

• Rule Type 2
•  
• Name Context Free
•  
• Example
• Phrase Structure Grammars/
• Push-Down Automata
•  
•  
• Rule type
• A ? ?
• with
• A auxiliary node
• ? any number of terminal or auxiliary nodes
•  
•  
• Recursiveness (centre embedding) allowed
• A ? ?A?

30

• CF Grammar
•  
•  A Context Free grammar consists of
•  
• a) a finite terminal vocabulary VT
•  
• b) a finite auxiliary vocabulary VA
•  
• c) an axiom S ? VA
•  
• a finite number of context free rules of
• form A ? ?,
• where A ? VA
• and ? ? VA ? VT
•  
• In natural language syntax S is interpreted as
  the start symbol for sentence, as in S ? NP VP

31

• CF Grammars
•  
• The following languages cannot be generated by a
  regular grammar
•  
• Language 1 Language 2
•  anbn mirror image
•  
• ab abaaba
• aabb abbaabba
• Context-Free rules
• A ? a A a
• A ? a b
• A ? b A b

32

• Natural language
• Is English regular or CF?
• If centre embedding is required, then it cannot
  be regular
• Centre Embedding
• 1. The cat likes tuna fish
• a b
• 2. The cat the dog chased likes tuna fish
• a a b b
• 3. The cat the dog the rat bit chased likes tuna
  fish
• a a a b b b
• 4. The cat the dog the rat the elephant admired
  bit chased likes tuna fish
• a a a a
  b b b b
•  ab
• aabb

33

• Centre embedding
• S
•   NP VP
• the likes
• cat tuna
• a b
• ab

34

• S
•   NP VP
• likes
• NP S tuna
• the b
• cat NP VP
• a the chased
• dog b
• a
• aabb

35

• S
•   NP VP
• likes
• NP S tuna
• the b
• cat NP VP
• a chased
• NP S b
• the
• dog NP VP
• a the bit
• rat b
• a
•  
• aaabbb

36

• Natural language
• Is English regular or CF?
•  
• If centre embedding is required, then it cannot
  be regular
•  

37

• Centre Embedding
• 1. The cat likes tuna fish
• a b
• ab
•  
• 2. The cat the dog chased likes tuna
  fish
• a a b b
• aabb

38

• The cat likes tuna fish
• a b
• 2. The cat the dog chased likes ...
• a a b b

39

• 3. The cat the dog the rat bit
  chased likes ...
• a a a b b
  b
•  
• The cat the dog the rat the elephant
  admired bit chased likes ....
• a a a a
  b b b b
•  
• aaabbb
• aaaabbbb

40

• Natural language 2
•  
• More Centre Embedding
•  
• 1. If S1, then S2
• a a
•  
• 2. Either S3, or S4
• b b
•  
• 3. The man who said S5 is arriving today
• ?
•  
• 4. The man who said S6 is arriving the day after
• ?
•  
• Sentence with embedding
• If either the man who said S5 is arriving today
  or the man who said S5 is arriving tomorrow, then
  the man who said S6 is arriving the day after
•  

41

• Natural language 2
•  
• More Centre Embedding
•  
•  
• 1. If S1, then S2
• a a
•  
• 2. Either S3, or S4
• b b
•  
•  
• Sentence with embedding
•  
• If either the man is arriving today or the woman
  is arriving tomorrow, then the child is arriving
  the day after.
•  
• a if
• b either the man is arriving today
• b or the woman is arriving tomorrow

42

• CS languages
•  
• The following languages cannot be generated by a
  CF grammar (by pumping lemma)
•  
• anbmcndm
•  
•  
• Swiss German
•  
• A string of dative nouns (e.g. aa), followed by a
  string of accusative nouns (e.g. bbb), followed
  by a string of dative-taking verbs (cc), followed
  by a string of accusative-taking verbs (ddd)
•  
•  
• aabbbccddd
•  
• anbmcndm

43

• Swiss German
•   
• Jan sait das (Jan says that)
•  
•  
• mer em Hans es Huus hälfed aastriiche
• we Hans/DAT the house/ACC helped paint
• we helped Hans paint the house
•  
• abcd
•  
• NPdat NPdat NPacc NPacc Vdat Vdat Vacc Vacc
• a a b b c c d d