CS 2130

1
CS 2130

• Presentation 15
• Compiler Introduction

2
Overall Operation

• Parser is in control of the overall operation
• Demands scanner to produce a token
• Scanner reads input file into token buffer
  forms a token (How?)
• Token is returned to parser
• Parser attempts to match the token (How?)
• Failure Syntax Error!
• Success
• Does nothing and returns to get next token
• OR
• Takes Semantic Action

3
Overall Operation

• Semantic Action Lookup variable name
• If found okay
• If not Put in symbol table
• If semantic checks succeed, do code-generation
  (How?)
• Return to get next token
• No more tokens? Done!

4
Tokenization
Input File
Token Buffer
5
Example
main()
m
6
Example
main()
am
7
Example
main()
iam
8
Example
main()
niam
9
Example
main()
(niam
10
Example
main()
niam
Keyword main
11
Overall Operation

• Parser is in control of the overall operation
• Demands scanner to produce a token
• Scanner reads input file into token buffer
  forms a token (How?)
• Token is returned to parser
• Parser attempts to match the token (How?)
• Failure Syntax Error!
• Success
• Does nothing and returns to get next token
• OR
• Takes Semantic Action

12
Overall Operation

• Semantic Action Lookup variable name
• If found okay
• If not Put in symbol table
• If semantic checks succeed, do code-generation
  (How?)
• Return to get next token
• No more tokens? Done!

13
Rules

• ltC-PROGgt ? MAIN OPENPAR ltPARAMSgt CLOSEPAR
  ltMAIN-BODYgt
• ltPARAMSgt ? NULL
• ltPARAMSgt ? VAR ltVAR-LISTgt
• ltVARLISTgt ? , ltVARgt ltVARLISTgt
• ltVARLISTgt ? NULL
• ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt ltASSIGN-STMTgt
  CURLYCLOSE
• ltDECL-STMTgt ? ltTYPEgtltVARgtltVAR-LISTgt
• ltASSIGN-STMTgt ? ltVARgt ltEXPRgt
• ltEXPRgt ? ltVARgt
• ltEXPRgt ? ltVARgtltOPgtltEXPRgt
• ltOPgt ?
• ltOPgt ? -
• ltTYPEgt ? INT
• ltTYPEgt ? FLOAT

14
Demo
main() int a,b a b
Scanner
Token Buffer
Parser
15
Demo
main() int a,b a b
Scanner
Token Buffer
"Please, get me the next token"
Parser
16
Demo
main() int a,b a b
Scanner
m
Parser
17
Demo
main() int a,b a b
Scanner
am
Parser
18
Demo
main() int a,b a b
Scanner
iam
Parser
19
Demo
main() int a,b a b
Scanner
niam
Parser
20
Demo
main() int a,b a b
Scanner
(niam
Parser
21
Demo
main() int a,b a b
Scanner
niam
Parser
22
Demo
main() int a,b a b
Scanner
Token Buffer
Token main
Parser
23
Demo
main() int a,b a b
Scanner
Token Buffer
Parser
"I recognize this"
24
Parsing (Matching)

• Start matching using a rule
• When match takes place at a certain position,
  move further (get next token repeat the
  process)
• If expansion needs to be done, choose appropriate
  rule (How to decide which rule to choose?)
• If no rule found, declare error
• If several rules found the grammar (set of rules)
  is ambiguous
• Grammar ambiguous? Language ambiguous?

25
Scanning Parsing Combined
main() int a,b a b
Scanner
"Please, get me the next token"
Parser
26
Scanning Parsing Combined
main() int a,b a b
Scanner
Token MAIN
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt
27
Scanning Parsing Combined
main() int a,b a b
Scanner
"Please, get me the next token"
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt
28
Scanning Parsing Combined
main() int a,b a b
Scanner
Token OPENPAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt
29
Scanning Parsing Combined
main() int a,b a b
Scanner
Token CLOSEPAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltPARAMETERSgt ? NULL
30
Scanning Parsing Combined
main() int a,b a b
Scanner
Token CLOSEPAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltPARAMETERSgt ? NULL
31
Scanning Parsing Combined
main() int a,b a b
Scanner
Token CLOSEPAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt
32
Scanning Parsing Combined
main() int a,b a b
Scanner
Token CURLYOPEN
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE
33
Scanning Parsing Combined
main() int a,b a b
Scanner
Token INT
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtltVARgtltVAR-LISTgt ltTYPEgt ? INT
34
Scanning Parsing Combined
main() int a,b a b
Scanner
Token INT
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtltVARgtltVAR-LISTgt ltTYPEgt ? INT
35
Scanning Parsing Combined
main() int a,b a b
Scanner
Token INT
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtltVARgtltVAR-LISTgt ltTYPEgt ? INT
36
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ltVARgt
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtltVARgtltVAR-LISTgt ltVARLISTgt ? , ltVARgt
ltVARLISTgt ltVARLISTgt ? NULL
37
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ',' COMMA
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtltVARgtltVAR-LISTgt ltVARLISTgt ? , ltVARgt
ltVARLISTgt ltVARLISTgt ? NULL
38
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ltVARgt
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtltVARgtltVAR-LISTgt ltVARLISTgt ? , ltVARgt
ltVARLISTgt ltVARLISTgt ? NULL
39
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtltVARgtltVAR-LISTgt ltVARLISTgt ? , ltVARgt
ltVARLISTgt ltVARLISTgt ? NULL
40
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtltVARgtltVAR-LISTgt ltVARLISTgt ? , ltVARgt
ltVARLISTgt ltVARLISTgt ? NULL
41
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtltVARgtltVAR-LISTgt ltVARLISTgt ? , ltVARgt
ltVARLISTgt ltVARLISTgt ? NULL
42
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtltVARgtltVAR-LISTgt ltVARLISTgt ? , ltVARgt
ltVARLISTgt ltVARLISTgt ? NULL
43
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtltVARgtltVAR-LISTgt ltVARLISTgt ? , ltVARgt
ltVARLISTgt ltVARLISTgt ? NULL
44
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtltVARgtltVAR-LISTgt
45
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtltVARgtltVAR-LISTgt
46
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ltVARgt
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? ltVARgt
ltEXPRgt ltEXPRgt ? ltVARgt
47
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? ltVARgt
ltEXPRgt ltEXPRgt ? ltVARgt
48
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ltVARgt
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? ltVARgt
ltEXPRgt ltEXPRgt ? ltVARgt
49
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ltVARgt
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? ltVARgt
ltEXPRgt ltEXPRgt ? ltVARgt
50
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ltVARgt
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? ltVARgt
ltEXPRgt ltEXPRgt ? ltVARgt
51
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? ltVARgt
ltEXPRgt
52
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? ltVARgt
ltEXPRgt
53
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? ltVARgt
ltEXPRgt
54
Scanning Parsing Combined
main() int a,b a b
Scanner
Token CURLYCLOSE
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE
55
What happens?

• During/after parsing?
• Tokens get gobbled
• Screen blinks?
• Smoke comes out?
• Further checking and setup for checks
• Semantic actions
• Semantic checks
• Symbol tables
• What are semantic actions?
• Variables have attributes
• Declaration attached attributes to variables

56
Symbol Table

• int a,b
• Declares a and b
• within current scope
• Of type integer
• Use of a and b now legal

57
Semantic Actions

• What are typical Semantic Actions?
• How do they get invoked?
• What happens after a Semantic Action?

58
Typical Semantic Actions

• Enter variable declaration into symbol table
• Look up variables in symbol table
• Do binding of looked-up variables (scoping rules,
  etc.)
• Do type checking for compatibility
• Keep the semantic context of processing
• a b c ? t1 a b
• t2 t1 c

Semantic Context
59
How are Semantic Actions Called?

• Action symbols embedded in the grammar
• Each action symbol represents a semantic
  procedure
• Semantic procedures are called by parser at
  appropriate places during parsing
• These procedures do things and/or return values
• Semantic stack implements stores semantic
  records
• Semantic actions could do checking and/or storage
  or retrieval of information

60
Semantic Actions

• ltdecl-stmtgt ? lttypegtput-typeltvar-listgtdo-decl
• ltvar-listgt ? ltvargt, ltvar-listgtadd-decl
• ltvar-listgt ? ltvargt
• ltvargt ? IDproc-decl
• put-type puts given type on semantic stack
• proc-decl builds decl record for var
• add-decl builds decl-chain
• do-decl traverses chain on semantic stack
  using
• backwards pointers entering each var
  into
• symbol table

Name
Type
Scope
id1
1
3
do-decl ?
id2
1
3
id3
1
3
61
Semantic Actions

• What else can semantic actions do in addition to
  storing and looking up names in a symbol table?
• Do type checking for type compatibility
  assignment
• Two type of Semantic Actions
• Checking (Binding, Type Compatibility, Scoping,
  etc.)
• Translation (Generate temporary values, propagate
  them to keep semantic context).

62
Semantic Actions (Translation)

• Consider a b c d
• Grammar
• ltASSGNSTMTgt ? ltVARgt ltEXPRgtdo-assign
• ltEXPRgt ? ltVARgtltEXPRTAILgt
• ltVARgt ? IDprocess-id
• ltEXPRTAILgt ? ltOPgtprocess-opltVARgtdo-infixltEXPRT
  AILgt
• ltEXPTAILgt ? NULL

63
Call Chain to Semantic Actions

• process-id Puts semantic record for "a" on stack
• Q. Checking or Translation?

"a"
64
Call Chain to Semantic Actions

• process-id Puts semantic record for "a" on stack
• Q. Checking or Translation?
• A. Checking

"a"
65
Call Chain to Semantic Actions

• process-op Puts semantic record for "" on stack
• Q. Checking or Translation?

""
"a"
66
Call Chain to Semantic Actions

• process-op Puts semantic record for "" on stack
• Q. Checking or Translation?
• A. Checking

""
"a"
67
Call Chain to Semantic Actions

• process-id Puts semantic record for "b" on stack

"b"
""
"a"
68
Call Chain to Semantic Actions

• process-id Puts semantic record for "" on stack

""
"b"
""
"a"
69
Call Chain to Semantic Actions

• process-id Puts semantic record for "c" on stack

"c"
""
"b"
""
"a"
70
Call Chain to Semantic Actions

• do-infix
• Get temporary (say t1)
• Evaluate
• IR t1 a b

"c"
""
"b"
""
"a"
71
Call Chain to Semantic Actions

• do-infix
• Get temporary (say t1)
• Evaluate
• IR t1 a b

"t1"
""
"a"
72
Call Chain to Semantic Actions

• process-id Puts semantic record for "" on stack

""
"t1"
""
"a"
73
Call Chain to Semantic Actions

• process-id Puts semantic record for "d" on stack

"d"
""
"t1"
""
"a"
74
Call Chain to Semantic Actions

• do-infix
• Get temporary (say t2)
• Evaluate
• IR t2 t1 d

"d"
""
"t1"
""
"a"
75
Call Chain to Semantic Actions

• do-infix
• Get temporary (say t2)
• Evaluate
• IR t2 t1 d

"t2"
""
"a"
76
Call Chain to Semantic Actions

• do-assign
• Put value back into variable
• IR a ? t2

"t2"
""
"a"
77
Call Chain to Semantic Actions

• do-assign
• Put value back into variable
• IR a ? t2

78
Code Generation
Scanner
Request Token
Get Token
Parser
Start
Semantic Action
Semantic Error
Checking
Intermediate Representation
79
Life After IR
IR

• What happens next?

Optimizer
Optimized IR
Second Pass
Code Generator
Code
80
Optionally

• What happens next?

Code
Third Pass
Post Pass Optimizer
Better Code
81
One Pass vs. Two Pass
Scanner
Parser
Start

• Comparisons
• No optimization performed in one pass
• Need to have full IR available for optimization
• Most compilers are two pass

Semantic Action
Semantic Error
Code Generation
CODE
82
Code Generation (Basic)

• Get a register to evaluate t1
• say R5
• Load "a" in it (where is "a"?)
• LOAD R5, 4(R6) R6 frame ptr
• Add "b" to register holding "a"
• ADD R5, 5(R6)
• Store R5 into t1's location
• STOR R5, 6(R6)

t1
b
a
R6 (FP)
83
Typical Code Generator Actions

• Get register for evaluation
• Load operands into registers (maybe!)
• Perform evaluation
• Store results back (maybe!)
• Free registers
• Return to Loop
• Hidden Actions
• Address calculation for operands
• Use of a given instruction format

84
Code Generator

• Instruction selection (Load, Store, Add, Mult,
  etc.)
• Address calculation
• Instruction format selection
• Immediate mode
• Direct mode
• Register Indirect
• Base Offset
• Generating one or more instructions
• How?
• Use of pattern matching
• Post order traversal of expression tree

Constants Globals/Static Pointers Arrays
85
Summary

• Parser is the brain of the compiler
• Controls everything
• Most of the time is spent in keeping checking
  information (front-end job)
• Translation takes place in back end
• IR useful for code optimization advanced code
  generation
• Scanner, parser and code generator are automated
  (How and Why?)

86
Questions?
87
(No Transcript)