Left and Right Linear Grammars Regular Expressions Scanner Implementation

1
Left and Right Linear GrammarsRegular
ExpressionsScanner Implementation

• CS 410 - Lecture 4
• West Virginia University
• Fall semester, 2002
• Frances L. Van Scoy
• fvanscoy_at_wvu.edu

2
Overview

• In this lecture we introduce regular expressions,
  a third way of defining some languages (along
  with grammars and automata).
• We show that left linear grammars, right linear
  grammars, finite state automata, and regular
  expressions are all equivalent in power.
• We show some techniques for implementing finite
  state automata and scanners.

3
Definition of Type 3 Grammars(Review)

• right linear
• A -gt a B or A -gt a,
• where A and B are in N and a is in S

• left linear
• A -gt B a or A -gt a,
• where A and B are in N and a is in S

4
A Right Linear Grammarfor Identifiers (Review)

• S -gt a T
• S -gt b T
• S -gt a
• S -gt b
• T -gt a T
• T -gt b T

• S gt a
• S gt a T gt a 1
• S gt a T gt a 1 T
• gt a 1 b T gt a 1 b 2

T -gt 1 T T -gt 2 T T -gt a T -gt b T -gt 1 T -gt 2
5
Constructing a Nondeterministic Finite State
Automaton from a Right Linear Grammar

• Let G (N, S, P, S).
• Construct a nondeterministic finite state
  automaton M (Q, S, d, S, F), where
• Q N U X, X not in N or S
• F X
• d is constructed by
• If A -gt a B is in P, then B is in d(A,a)
• If A -gt a is in P, then X is in d (A,a)

6
Example RLG to NDFSA

• d(S,a) T, X
• d(S,b) T,X
• d(S,1) F
• d(S,2) F
• d(T,a) T,X
• d(T,b) T,X
• d(T,1) T,X
• d(T,2) T,X

S -gt a T S -gt b T S -gt a S -gt b T -gt a T T -gt b T
T -gt 1 T T -gt 2 T T -gt a T -gt b T -gt 1 T -gt 2
7
A Left Linear Grammar for Identifiers (Review)

• S -gt S a
• S -gt S b
• S -gt S 1
• S -gt S 2
• S -gt a
• S -gt b

• S gt a
• S gt S 1 gt a 1
• S gt S 2 gt S b 2
• gt S 1 b 2 gt a 1 b 2

8
Constructing a Nondeterministic Finite State
Automaton from a Left Linear Grammar

• Let G (N, S, P, S).
• Construct a nondeterministic finite state
  automaton M (Q, S, d, X, F), where
• Q N U X, X not in N or S
• F S
• d is constructed by
• If A -gt B a is in P, then A is in d(B,a)
• If A -gt a is in P, then A is in d (X,a)

9
Example LLG to NDFSA

• S -gt S a
• S -gt S b
• S -gt S 1
• S -gt S 2
• S -gt a
• S -gt b

• d(X,a) S
• d(X,b) S
• d(X,1) F
• d(X,2) F
• d(S,a) S
• d(S,b) S
• d(S,1) S
• d(S,2) S

10
Definition of a Regular Expression

• Definition of a regular expression over alphabet
  S
• If a is in S then a is a r.e.
• If x is a r.e., then ( x ) is a r.e.
• If x is a r.e., then x is a r.e.
• If x and y are r.e., then x y is a r.e.
• If x and y are r.e., then x y is a r.e.
• Nothing else is a r.e.

11
Examples of Some Regular Expressions

• Identifier
• (a b) (a b 1 2)
• Real numeric literal
• 1 1 . 1 1
• 1 . 1
• Numeric literal
• 1 ( . 1 l )

12
Construction of a NDFSA with l-Transitions from a
RE (1)

• If a is in S then a is a r.e.
• If x is a r.e.,then ( x ) is a r.e.
• If x is a r.e., then x is a r.e.

x
l
13
Construction of a NDFSA with l-Transitions from a
RE (2)
x

• If x and y are r.e., then x y is a r.e.
• If x and y are r.e., then x y is a r.e.

l
l
l
l
y
l
x
y
l
l
14
Construction of a NDFSA from a NDFSA with
l-transitions
15
Summary of Conversions
Is a special case of
DFSA
NDFSA
NDFSA with l-transitions
Regular Expression
LLG
RLG
Shown in lecture
Not shown in lecture, but true
So these six ways of defining a language
are equivalent in power.
16
The Practical Question

• How do we implement a finite state automaton?
• A loop containing a case statement, with one case
  per state
• Or
• Lex (next weeks lectures)

17
Example of Implementation

• State X
• Loop
• case State is
• when X gt
• Case Input is
• When a gt
• When b gt
• When 1 gt
• When 2 gt
• End case
• When S gt
• Case Input is
• When a gt
• When b gt
• When 1 gt
• When 2 gt
• End case
• When Empty gt
• Case Input is

• d(X,a) S
• d(X,b) S
• d(X,1) F
• d(X,2) F
• d(S,a) S
• d(S,b) S
• d(S,1) S
• d(S,2) S

18
Summary
19
Quiz