Lexical and Syntax Analysis

1
Lexical and Syntax Analysis

• David Woolbright

2
Syntax Analysers or Parsers

• Usually based on a formal description of a
  language
• Context-free grammars or BNF are used to describe
  the syntax of a language
• Parsers are organized into two parts
• Lexical analyser small scale language
  constructs names, literals
• Syntax analyser large scale language constructs
  expressions, statements,

3
Why Separate?

• Simplicity lexical analysis is easier than
  syntax analysis
• Efficiency lexical analysis can be optimized
• Portability lexical analysis is platform
  dependent

4
Lexical Analysis

• Pattern matching
• Languages described by DFA
• Easily translated
• Acts as a front end for syntactic analysis
• Collects characters and outputs lexemes
  (character groupings).
• Internal codes are called tokens

5
Lexical Analysis

• Produces the next lexeme and token
• Skips comments and white space
• Inserts user-defined names into a symbol table
  (for later use by a compiler)
• Detect syntactic errors in tokens

6
Building a Lexical Analyser

• Write a formal description of the token patterns
  using a descriptive language related to regular
  expressions. Use a software tool to generate the
  lexical analyser
• Design a DFA for the token patterns and write a
  program that implements the DFA
• Design a DFA for the token patterns and
  hand-construct a table driven implementation of
  the DFA

7
(No Transcript)