CS 2130

1
CS 2130

• Presentation 15
• Compiler Introduction

2
Big Picture

• Parsing Matching code we are translating to
  rules of a grammar. Building a representation of
  the code.
• Scanning An abstraction that simplifies the
  parsing process by converting the raw text input
  into a stream of known objects called tokens.
• Abstract Syntax Tree (e.g.) One of several
  possible ways of storing the program in a form
  that captures the essence of the desired
  operations
• Code Generation In general, when the parser
  finds a match some sequence of instructions in
  the target language may be generated
• Optimization Making the raw code generated better

3
Overall Operation

• Parser is in control of the overall operation
• Demands scanner to produce a token
• Scanner reads input file into token buffer
  forms a token (How?)
• Token is returned to parser
• Parser attempts to match the token (How?)
• Failure Syntax Error!
• Success
• Does nothing and returns to get next token
• OR
• Takes Semantic Action

4
Overall Operation

• Semantic Action Lookup variable name
• If found okay
• If not Put in symbol table
• If semantic checks succeed, do code-generation
  (How?)
• Return to get next token
• No more tokens? Done!

5
Tokenization
Input File
Token Buffer
6
Example
main()
m
7
Example
main()
am
8
Example
main()
iam
9
Example
main()
niam
10
Example
main()
(niam
11
Example
main()
niam
Keyword main
12
Overall Operation

• Parser is in control of the overall operation
• Demands scanner to produce a token
• Scanner reads input file into token buffer
  forms a token (How?)
• Token is returned to parser
• Parser attempts to match the token (How?)
• Failure Syntax Error!
• Success
• Does nothing and returns to get next token
• OR
• Takes Semantic Action

13
Overall Operation

• Semantic Action Lookup variable name
• If found okay
• If not Put in symbol table
• If semantic checks succeed, do code-generation
  (How?)
• Return to get next token
• No more tokens? Done!

14
Rules

• ltC-PROGgt ? MAIN OPENPAR ltPARAMSgt CLOSEPAR
  ltMAIN-BODYgt
• ltPARAMSgt ? NULL
• ltPARAMSgt ? VAR ltVAR-LISTgt
• ltVARLISTgt ? , VAR ltVARLISTgt
• ltVARLISTgt ? NULL
• ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt ltASSIGN-STMTgt
  CURLYCLOSE
• ltDECL-STMTgt ? ltTYPEgt VAR ltVAR-LISTgt
• ltASSIGN-STMTgt ? VAR ltEXPRgt
• ltEXPRgt ? VAR
• ltEXPRgt ? VARltOPgtltEXPRgt
• ltOPgt ?
• ltOPgt ? -
• ltTYPEgt ? INT
• ltTYPEgt ? FLOAT

15
Demo
main() int a,b a b
Scanner
Token Buffer
Parser
16
Demo
main() int a,b a b
Scanner
Token Buffer
"Please, get me the next token"
Parser
17
Demo
main() int a,b a b
Scanner
m
Parser
18
Demo
main() int a,b a b
Scanner
am
Parser
19
Demo
main() int a,b a b
Scanner
iam
Parser
20
Demo
main() int a,b a b
Scanner
niam
Parser
21
Demo
main() int a,b a b
Scanner
(niam
Parser
22
Demo
main() int a,b a b
Scanner
niam
Parser
23
Demo
main() int a,b a b
Scanner
Token Buffer
Token main
Parser
24
Demo
main() int a,b a b
Scanner
Token Buffer
Parser
"I recognize this"
25
Parsing (Matching)

• Start matching using a rule
• When match takes place at a certain position,
  move further (get next token repeat the
  process)
• If expansion needs to be done, choose appropriate
  rule (How to decide which rule to choose?)
• If no rule found, declare error
• If several rules found the grammar (set of rules)
  is ambiguous
• Grammar ambiguous? Language ambiguous?

26
Scanning Parsing Combined
main() int a,b a b
Scanner
"Please, get me the next token"
Parser
27
Scanning Parsing Combined
main() int a,b a b
Scanner
Token MAIN
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt
28
Scanning Parsing Combined
main() int a,b a b
Scanner
"Please, get me the next token"
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt
29
Scanning Parsing Combined
main() int a,b a b
Scanner
Token OPENPAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt
30
Scanning Parsing Combined
main() int a,b a b
Scanner
Token CLOSEPAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltPARAMETERSgt ? NULL
31
Scanning Parsing Combined
main() int a,b a b
Scanner
Token CLOSEPAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltPARAMETERSgt ? NULL
32
Scanning Parsing Combined
main() int a,b a b
Scanner
Token CLOSEPAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt
33
Scanning Parsing Combined
main() int a,b a b
Scanner
Token CURLYOPEN
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE
34
Scanning Parsing Combined
main() int a,b a b
Scanner
Token INT
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtVARltVAR-LISTgt ltTYPEgt ? INT
35
Scanning Parsing Combined
main() int a,b a b
Scanner
Token INT
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtVARltVAR-LISTgt ltTYPEgt ? INT
36
Scanning Parsing Combined
main() int a,b a b
Scanner
Token INT
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtVARltVAR-LISTgt ltTYPEgt ? INT
37
Scanning Parsing Combined
main() int a,b a b
Scanner
Token VAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtVARltVAR-LISTgt ltVARLISTgt ? , VAR
ltVARLISTgt ltVARLISTgt ? NULL
38
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ',' COMMA
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtVARltVAR-LISTgt ltVARLISTgt ? , VAR
ltVARLISTgt ltVARLISTgt ? NULL
39
Scanning Parsing Combined
main() int a,b a b
Scanner
Token VAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtVARltVAR-LISTgt ltVARLISTgt ? , VAR
ltVARLISTgt ltVARLISTgt ? NULL
40
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtVARltVAR-LISTgt ltVARLISTgt ? , VAR
ltVARLISTgt ltVARLISTgt ? NULL
41
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtVARltVAR-LISTgt ltVARLISTgt ? , VAR
ltVARLISTgt ltVARLISTgt ? NULL
42
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtVARltVAR-LISTgt ltVARLISTgt ? , VAR
ltVARLISTgt ltVARLISTgt ? NULL
43
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtVARltVAR-LISTgt ltVARLISTgt ? , VAR
ltVARLISTgt ltVARLISTgt ? NULL
44
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtVARltVAR-LISTgt ltVARLISTgt ? , VAR
ltVARLISTgt ltVARLISTgt ? NULL
45
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtVARltVAR-LISTgt
46
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltDECL-STMTgt ?
ltTYPEgtVARltVAR-LISTgt
47
Scanning Parsing Combined
main() int a,b a b
Scanner
Token VAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? VAR
ltEXPRgt ltEXPRgt ? VAR
48
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? VAR
ltEXPRgt ltEXPRgt ? VAR
49
Scanning Parsing Combined
main() int a,b a b
Scanner
Token VAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? VAR
ltEXPRgt ltEXPRgt ? VAR
50
Scanning Parsing Combined
main() int a,b a b
Scanner
Token VAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? VAR
ltEXPRgt ltEXPRgt ? VAR
51
Scanning Parsing Combined
main() int a,b a b
Scanner
Token VAR
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? VAR
ltEXPRgt ltEXPRgt ? VAR
52
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? VAR
ltEXPRgt
53
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? VAR
ltEXPRgt
54
Scanning Parsing Combined
main() int a,b a b
Scanner
Token ''
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE ltASSIGN-STMTgt ? VAR
ltEXPRgt
55
Scanning Parsing Combined
main() int a,b a b
Scanner
Token CURLYCLOSE
Parser
ltC-PROGgt ? MAIN OPENPAR ltPARAMETERSgt CLOSEPAR
ltMAIN-BODYgt ltMAIN-BODYgt ? CURLYOPEN ltDECL-STMTgt
ltASSIGN-STMTgt CURLYCLOSE
56
What happens?

• During/after parsing?
• Tokens get gobbled
• Screen blinks?
• Smoke comes out?
• Further checking and setup for checks
• Semantic actions
• Semantic checks
• Symbol tables
• What are semantic actions?
• Variables have attributes
• Declaration attached attributes to variables

57
Symbol Table

• int a,b
• Declares a and b
• within current scope
• Of type integer
• Use of a and b now legal

58
Semantic Actions

• What are typical Semantic Actions?
• How do they get invoked?
• What happens after a Semantic Action?

59
Typical Semantic Actions

• Enter variable declaration into symbol table
• Look up variables in symbol table
• Do binding of looked-up variables (scoping rules,
  etc.)
• Do type checking for compatibility
• Keep the semantic context of processing
• a b c ? t1 a b
• t2 t1 c

Semantic Context
60
How are Semantic Actions Called?

• Action symbols embedded in the grammar
• Each action symbol represents a semantic
  procedure
• Semantic procedures are called by parser at
  appropriate places during parsing
• These procedures do things and/or return values
• Semantic stack implements stores semantic
  records
• Semantic actions could do checking and/or storage
  or retrieval of information

61
Semantic Actions

• ltdecl-stmtgt ? lttypegtput-typeltvar-listgtdo-decl
• ltvar-listgt ? var, ltvar-listgtadd-decl
• ltvar-listgt ? var
• var ? IDproc-decl
• put-type puts given type on semantic stack
• proc-decl builds decl record for var
• add-decl builds decl-chain
• do-decl traverses chain on semantic stack
  using
• backwards pointers entering each var
  into
• symbol table

decl record
id3
Name
Type
Scope
id2
id1
1
3
do-decl ?
id1
id2
1
3
type
id3
1
3
62
Semantic Actions

• What else can semantic actions do in addition to
  storing and looking up names in a symbol table?
• Do type checking for type compatibility
  assignment
• Two type of Semantic Actions
• Checking (Binding, Type Compatibility, Scoping,
  etc.)
• Translation (Generate temporary values, propagate
  them to keep semantic context).

63
Semantic Actions (Translation)

• Consider a b c d
• Grammar
• ltASSGNSTMTgt ? VAR ltEXPRgtdo-assign
• ltEXPRgt ? VARltEXPRTAILgt
• VAR ? IDprocess-id
• ltEXPRTAILgt ? ltOPgtprocess-opVARdo-infixltEXPRTAI
  Lgt
• ltEXPTAILgt ? NULL

64
Call Chain to Semantic Actions
a b c d

• process-id Puts semantic record for "a" on
  stack
• Q. Checking or Translation?

"a"
65
Call Chain to Semantic Actions
a b c d

• process-id Puts semantic record for "a" on
  stack
• Q. Checking or Translation?
• A. Checking

"a"
66
Call Chain to Semantic Actions
a b c d

• process-op Puts semantic record for "" on
  stack
• Q. Checking or Translation?

""
"a"
67
Call Chain to Semantic Actions
a b c d

• process-op Puts semantic record for "" on
  stack
• Q. Checking or Translation?
• A. Checking

""
"a"
68
Call Chain to Semantic Actions
a b c d

• process-id Puts semantic record for "b" on
  stack

"b"
""
"a"
69
Call Chain to Semantic Actions
a b c d

• process-id Puts semantic record for "" on
  stack

""
"b"
""
"a"
70
Call Chain to Semantic Actions
a b c d

• process-id Puts semantic record for "c" on stack

"c"
""
"b"
""
"a"
71
Call Chain to Semantic Actions
a b c d

• do-infix
• Get temporary (say t1)
• Evaluate
• IR t1 a b

"c"
""
"b"
""
"a"
72
Call Chain to Semantic Actions
a b c d

• do-infix
• Get temporary (say t1)
• Evaluate
• IR t1 a b

"t1"
""
"a"
73
Call Chain to Semantic Actions
a b c d

• process-id Puts semantic record for "" on
  stack

""
"t1"
""
"a"
74
Call Chain to Semantic Actions
a b c d

• process-id Puts semantic record for "d" on
  stack

"d"
""
"t1"
""
"a"
75
Call Chain to Semantic Actions
a b c d

• do-infix
• Get temporary (say t2)
• Evaluate
• IR t2 t1 d

"d"
""
"t1"
""
"a"
76
Call Chain to Semantic Actions
a b c d

• do-infix
• Get temporary (say t2)
• Evaluate
• IR t2 t1 d

"t2"
""
"a"
77
Call Chain to Semantic Actions
a b c d

• do-assign
• Put value back into variable
• IR a ? t2

"t2"
""
"a"
78
Call Chain to Semantic Actions
a b c d

• do-assign
• Put value back into variable
• IR a ? t2

79
Code Generation
Scanner
Request Token
Get Token
Parser
Start
Semantic Action
Semantic Error
Checking
Intermediate Representation
80
Life After IR
IR

• What happens next?

Optimizer
Optimized IR
Second Pass
Code Generator
Code
81
Optionally

• What happens next?

Code
Third Pass
Post Pass Optimizer
Better Code
82
One Pass vs. Two Pass
Scanner
Parser
Start

• Comparisons
• No optimization performed in one pass
• Need to have full IR available for optimization
• Most compilers are two pass

Semantic Action
Semantic Error
Code Generation
CODE
83
Code Generation (Basic)
t1 a b

• Get a register to evaluate t1
• say R5
• Load "a" in it (where is "a"?)
• LOAD R5, 4(R6) R6 frame ptr
• Add "b" to register holding "a"
• ADD R5, 5(R6)
• Store R5 into t1's location
• STOR R5, 6(R6)

t1
b
a
R6 (FP)
84
Typical Code Generator Actions

• Get register for evaluation
• Load operands into registers (maybe!)
• Perform evaluation
• Store results back (maybe!)
• Free registers
• Return to Loop
• Hidden Actions
• Address calculation for operands
• Use of a given instruction format

85
Code Generator

• Instruction selection (Load, Store, Add, Mult,
  etc.)
• Address calculation
• Instruction format selection
• Immediate mode
• Direct mode
• Register Indirect
• Base Offset
• Generating one or more instructions
• How?
• Use of pattern matching
• Post order traversal of expression tree

Constants Globals/Static Pointers Arrays
86
Summary

• Parser is the brain of the compiler
• Controls everything
• Most of the time is spent in keeping checking
  information (front-end job)
• Translation takes place in back end
• IR useful for code optimization advanced code
  generation
• Scanner, parser and code generator are automated
  (How and Why?)

87
Questions?
88
(No Transcript)