Parsing

1
Parsing
2
Front-End Parser
tokens
sourcecode
IR
scanner
parser
errors

• Checks the stream of words and their parts of
  speech for grammatical correctness

3
Front-End Parser
tokens
sourcecode
IR
scanner
parser
errors

• Determines if the input is syntactically well
  formed

4
Front-End Parser
tokens
sourcecode
IR
scanner
parser
errors

• Guides context-sensitive (semantic) analysis
  (type checking)

5
Front-End Parser
tokens
sourcecode
IR
scanner
parser
errors

• Builds IR for source program

6
Syntactic Analysis

• Natural language analogy consider the sentence

He
wrote
the
program
7
Syntactic Analysis
He
wrote
the
program
noun
verb
article
noun
8
Syntactic Analysis
He
wrote
the
program
noun
verb
article
noun
subject
predicate
object
9
Syntactic Analysis

• Natural language analogy

He
wrote
the
program
noun
verb
article
noun
subject
predicate
object
sentence
10
Syntactic Analysis

• Programming language

if
( b lt 0 )
a b
assignment
bool expr
if-statement
11
Syntactic Analysis

• syntax errors
• int foo(int i, int j))
• for(k0 i j )
• fi( i gt j )
• return j

12
Compiler Construction

• Sohail Aslam
• Lecture 11

13
Syntactic Analysis

• int foo(int i, int j))
• for(k0 i j )
• fi( i gt j )
• return j

extra parenthesis
Missing expression
not a keyword
14
Semantic Analysis

• Grammatically correct

He
wrote
the
computer
noun
verb
article
noun
subject
predicate
object
sentence
15
Semantic Analysis

• semantically (meaning) wrong!

He
wrote
the
computer
noun
verb
article
noun
subject
predicate
object
sentence
16
Semantic Analysis

• int foo(int i, int j)
• for(k0 i lt j j )
• if( i lt j-2 )
• sum sumi
• return sum

undeclared var
return type mismatch
17
Role of the Parser

• Not all sequences of tokens are program.
• Parser must distinguish between valid and invalid
  sequences of tokens.

18
Role of the Parser

• Not all sequences of tokens are program.
• Parser must distinguish between valid and invalid
  sequences of tokens.

19
Role of the Parser

• What we need
• An expressive way to describe the syntax
• An acceptor mechanism that determines if input
  token stream satisfies the syntax

20
Role of the Parser

• What we need
• An expressive way to describe the syntax
• An acceptor mechanism that determines if input
  token stream satisfies the syntax

21
Role of the Parser

• What we need
• An expressive way to describe the syntax
• An acceptor mechanism that determines if input
  token stream satisfies the syntax

22
Study of Parsing

• Parsing is the process of discovering a
  derivation for some sentence

23
Study of Parsing

• Mathematical model of syntax a grammar G.
• Algortihm for testing membership in L(G).

24
Study of Parsing

• Mathematical model of syntax a grammar G.
• Algortihm for testing membership in L(G).

25
Context Free Grammars

• A CFG is a four tuple G(S,N,T,P)
• S is the start symbol
• N is a set of non-terminals
• T is a set of terminals
• P is a set of productions

26
Why Not Regular Expressions?

• Reason regular languages do not have enough
  power to express syntax of programming languages.

27
Limitations of Regular Languages

• Finite automaton cant remember number of times
  it has visited a particular state

28
Example of CFG

• Context-free syntax is specified with a CFG

29
Example of CFG

• ExampleSheepNoise ? SheepNoise baa baa
• This CFG defines the set of noises sheep make

30
Example of CFG

• We can use the SheepNoise grammar to create
  sentences
• We use the productions as rewriting rules

31
Example of CFG

• SheepNoise ? SheepNoise baa baa

Rule Sentential Form
- SheepNoise
2 baa
32
Example of CFG

• SheepNoise ? SheepNoise baa baa

Rule Sentential Form
- SheepNoise
1 SheepNoise baa
2 baa baa
33
Example of CFG
Rule Sentential Form
- SheepNoise
1 SheepNoise baa
1 SheepNoise baa baa
2 baa baa baa

• And so on ...

34
Example of CFG

• While it is cute, this example quickly runs out
  intellectual steam
• To explore uses of CFGs, we need a more complex
  grammar

35
Example of CFG

• While it is cute, this example quickly runs out
  intellectual steam
• To explore uses of CFGs, we need a more complex
  grammar

36
More Useful Grammar
1 expr ? expr op expr
2 num
3 id
4 op ?
5
6
7 /
37
Backus-Naur Form (BNF)

• Grammar rules in a similar form were first used
  in the description of the Algol60 Language.

38
Backus-Naur Form (BNF)

• The notation was developed by John Backus and
  adapted by Peter Naur for the Algol60 report.
• Thus the term Backus-Naur Form (BNF)

39
Backus-Naur Form (BNF)

• The notation was developed by John Backus and
  adapted by Peter Naur for the Algol60 report.
• Thus the term Backus-Naur Form (BNF)

40
Derivation

• Let us use the expression grammar to derive the
  sentence
• x 2 y

41
Derivation x 2 y
Rule Sentential Form
- expr
1 expr op expr
2 ltid,xgt op expr
5 ltid,xgt expr
1 ltid,xgt expr op expr
42
Derivation x 2 y
Rule Sentential Form
2 ltid,xgt ltnum,2gt op expr
6 ltid,xgt ltnum,2gt ? expr
3 ltid,xgt ltnum,2gt ? ltid,ygt
43
Derivation

• Such a process of rewrites is called a
  derivation.
• Process or discovering a derivations is called
  parsing

44
Derivation

• Such a process of rewrites is called a
  derivation.
• Process or discovering a derivations is called
  parsing

45
Derivation
We denote this derivation as expr ? id
num id
46
Derivations

• At each step, we choose a non-terminal to replace
• Different choices can lead to different
  derivations.

47
Derivations

• At each step, we choose a non-terminal to replace
• Different choices can lead to different
  derivations.

48
Derivations

• Two derivations are of interest
• Leftmost derivation
• Rightmost derivation

49
Derivations

• Leftmost derivation replace leftmost
  non-terminal (NT) at each step
• Rightmost derivation replace rightmost NT at
  each step

50
Derivations

• Leftmost derivation replace leftmost
  non-terminal (NT) at each step
• Rightmost derivation replace rightmost NT at
  each step

51
Derivations

• The example on the preceding slides was leftmost
  derivation
• There is also a rightmost derivation

52
Rightmost Derivation
Rule Sentential Form
- expr
1 expr op expr
3 expr op ltid,xgt
6 expr ? ltid,xgt
1 expr op expr ? ltid,xgt
53
Derivation x 2 y
Rule Sentential Form
2 expr op ltnum,2gt ? ltid,xgt
5 expr ltnum,2gt ? ltid,xgt
3 ltid,xgt ltnum,2gt ? ltid,ygt
54
Derivations

• In both cases we have expr ? id num ? id

55
Derivations

• The two derivations produce different parse
  trees.
• The parse trees imply different evaluation orders!

56
Derivations

• The two derivations produce different parse
  trees.
• The parse trees imply different evaluation orders!