CSE 3813 Introduction to Formal Languages and Automata

1
CSE 3813Introduction to Formal Languages and
Automata

• Chapter 1
• Introduction to the Theory of Computation
• These class notes are based on material from our
  textbook, An Introduction to Formal Languages and
  Automata, 4th ed., by Peter Linz, published by
  Jones and Bartlett Publishers, Inc., Sudbury, MA,
  2006. They are intended for classroom use only
  and are not a substitute for reading the textbook.

2
Acknowledgement

• Unless otherwise credited, all class notes on
  this website are based on the required textbook
  for this course, Linz, Peter. An Introduction to
  Formal Languages and Automata, 4th ed. Sudbury,
  Mass. Jones and Bartlett Publishers, 2006.
• These notes are intended solely for the use of
  the students in the CS 3813 class at Mississippi
  State University.
• Please assume any errors to be mine, and not the
  author of the textbook.

3
What does the title of this course mean?

• Grammar
• A set of rules for generating all and only the
  strings of a particular language
• Example the grammar (syntax rules) for the C
  language
• Formal language
• a subset of the set of all possible strings from
  from a set of symbols
• Example the set of all syntactically correct C
  programs
• Automata
• abstract, mathematical model of computer
• Examples finite automata, pushdown automata
  Turing machine, RAM, PRAM, many others
• We will consider each of these in this course

4
Focus of the course

• What are the fundamental capabilities and
  limitations of computers?
• To answer this, we will study abstract
  mathematical models of computers.
• These mathematical models abstract away many of
  the details of computers to allow us to focus on
  the essential aspects of computation.
• These models also allow us to develop a
  mathematical theory of computation.

5
What is computation?

• Computation is the execution of an algorithm by a
  computer.
• An algorithm is a sequence of primitive steps
  that can be specified explicitly.
• An algorithm can be performed mechanically, that
  is, by a machine
• It computes a function that transforms input into
  output.

6
Computer or Turing machine(Alan Turing 1936)
Read/write head
7
Automata

• An automaton has
• Input File
• Control Unit (with finite states)
• Temporary Storage
• Output

8
Finite automata

• Developed in 1940s and 1950s for neural net
  models of brain and computer hardware design
• Finite memory!
• Many applications
• text-editing software search and replace
• many forms of pattern-recognition (including use
  in WWW search engines)
• compilers recognizing keywords (lexical
  analysis)
• sequential circuit design
• software specification and design
• communications protocols

9
Finite automata

• For the computer engineers among us, you may
  think of finite automata as in-line filters.
• In an in-line filter, a signal comes in and the
  filter handles it, depending only upon the
  signals characteristics and the state the filter
  is in.
• The typical in-line filter has no auxiliary
  memory.
• The filter can change its state from one state to
  another, depending upon the signal it receives.
• By being in a different state the next time it
  receives a given signal, it can handle the same
  signal in different ways.

10
Pushdown automata

• Noam Chomskys work in the 1950s and 1960s on
  grammars for natural languages
• infinite memory, organized as a stack
• Applications
• compilers parsing computer programs
• programming language design

11
Turing machine

• Devised by Alan Turing
• Has infinite memory, organized as a tape, with a
  read/write head
• Most powerful automaton it can be proven that no
  computer can be more powerful than a Turing
  machine

12
Computational power
TM
LBA
PDA
FSA
13
Formal language
Alphabet finite set of symbols or characters
examples S a,b, binary,
ASCII String finite sequence of symbols from
an alphabet examples aab, bbaba,
also computer programs A formal language is a
set of strings over an alphabet. Examples of
formal languages over the alphabet S a, b
L1 aa, aba, aababa, aa L2 all strings
containing just two as and any number of bs A
formal language can be finite or infinite.
14
Formal languages (continued)

We often use string variables u aab, v
bbaba Operations on strings length u
3 reversal uR baa concatenation uv
aabbbaba The empty string, denoted ? , has some
special properties ? 0 ?w w? w
15
Formal languages (continued)

If w is a string, then wn stands for the string
obtained by repeating w n times. w0 ? S S
- ? L0 ? L1 L
16
Operations on languages
Set operations L1 ? L2 x x ? L1 or x ?
L2 is union L1 ? L2 x x ? L1 and x ?
L2 is intersection L1 - L2 x x ? L1 and
x ? L2 is difference ? - L is
complement L1 ? L2 (L1 - L2) ? (L2 - L1) is
symmetric difference
17
Operations on languages
String operations LR wR w ? L is
reverse of language L1L2 xy x ? L1, y
? L2 is concatenation of languages L
x ? x1xk k ? 0 and x1, , xk ? L L0 ?
L1 ? L2 . . . . is Kleene star or star
closure L L1 ? L2 . . . . is positive
closure
18
Some review questions

• What is ?, 01, 001 ? ?, 00, 10?
• What is the concatenation of 0, 11, 010 and ?,
  10, 010?
• What are the 5 shortest strings in the language
  0i1i i ? 0?
• What is the powerset of a, b, ab?

19
Back to Formal Languages

• An important example of a formal language
• alphabet ASCII symbols
• string a particular C program
• formal language set of all legal C programs
• The study of formal languages deals with
• Languages
• Grammars
• Automata

20
Grammars
Definition 1.1
A grammar G is defined as a quadruple G (V,
T, S, P) where V is a finite set of objects
called variables T is a finite set of objects
called terminal symbols S ? V is a special symbol
called the Start symbol P is a finite set of
productions or "production rules" Sets V and T
are nonempty and disjoint
21
Grammars
Production rules have the form x ? y where x is
an element of (V ? T) and y is in (V ? T) Given
a string of the form w uxv and a production
rule x ? y we can apply the rule, replacing x
with y, giving z uyv We can then say that
w ? z Read as "w derives z", or "z is derived
from w"
22
Grammars
If u ? v, v ? w, w ? x, x ? y, and y ? z, then we
say u ? z This says that u derives z
in an unspecified number of steps. Along the
way, we may generate strings which contain
variables as well as terminals. These are called
sentential forms.
23
Grammars
What is the relationship between a language and a
grammar?
Definition 1.2
Let G (V, T, S, P) The set L(G)
w ? T S ? w is the language generated by
G.
24
Grammars
Consider the grammar G (V, T, S, P), where V
S T a, b S S, P
25
Grammars
What are some of the strings in this language? S
? aSb ? ab S ? aSb ? aaSbb ? aabb S ? aSb ? aaSbb
? aaaSbbb ? aaabbb It is easy to see that the
language generated by this grammar is L(G)
anbn n ? 0 (See proof on pp. 22-23 in Linz)
26
Grammars
Let's go the other way, from a description of a
language to a grammar that generates it. Find a
grammar that generates L anbn1 n ? 0 So
the strings of this language will be b (0 a's
and 1 b) abb (1 a and 2 b's) aabbb (2 a's
and 3 b's) . . .
27
Grammars
In order to generate a string with no a's and 1
b, you might want to write rules for the grammar
that say S ? ab a ? ? But you can't do
this a is a terminal, and you can't change a
terminal, only variables
28
Grammars
So, instead of S ? ab a ? ? we create
another variable, A (we often use capital letters
to stand for variables), to use in place of the
terminal, a S ? Ab A ? ?
29
Grammars
Now you might think that we can use another S
rule here to generate the other part of the
string, the anbn part S ? aSb But you can't,
because that will generate ab, aabb, etc, which
are not strings in our language. Note, however,
that if we use A in place of S, that will solve
our problem A ? aAb
30
Grammars
So, here are our rules S ? Ab A ? aAb A ?
? The S ? Ab rule creates a single b terminal on
the right, preceded by other strings (including
possibly the empty string) on the left. The A ?
? rule allows the single b string to be
generated. The A ? aAb rule and the A ? ? rule
allows ab, aabb, aaabbb, etc. to be generated on
the left side of the string.
31
Language-recognition problem

• There are many types of computational problems.
  We will focus on the simplest, called the
  language-recognition problem.
• Given a string, determine whether it belongs to a
  language or not. (Practical application for
  compilers Is this a valid C program?)
• We study simple models of computation called
  automata, and measure their computational power
  in terms of the class of languages they can
  recognize.

32
Automata, languages, and grammars

• In this course, we will study the relationship
  between automata, languages, and grammars.
• Recall that a formal language is a set of strings
  over a finite alphabet.
• Automata are used to recognize languages.
• Grammars are used to generate languages.
• All of these concepts fit together.

33
Classification of automata, languages, and
grammars
34
Computability Theory

• Besides developing a theory of classes of
  languages and automata, we will study the limits
  of computation. We will consider the following
  two important questions
• What problems are impossible for a computer to
  solve?
• What problems are too difficult for a computer to
  solve in practice (although possible to solve in
  principle)?

35
Uncomputable (undecidable) problems

• Many well-defined (and apparently simple)
  problems cannot be solved by any computer
• Examples
• For any program x, does x have an infinite loop?
• For any two programs x and y, do these two
  programs have the same input/output behavior?
• For any program x, does x meet its specification?
  (i.e., does it have any bugs?)

36
Intractable problems

• We will learn how to mathematically characterize
  the difficulty of computational problems.
• There is a class of problems that can be solved
  in a reasonable amount of time and another class
  that cannot (What good is it for a problem to be
  solvable, if it cannot be solved in the lifetime
  of the universe?)
• The field of cryptography, for example, relies on
  the fact that the computational problem of
  breaking a code is intractable

37
Why study the theory of computing?

• This is the core mathematics of CS, and has not
  changed in over 30 years.
• There are many applications, especially in design
  of compilers and programming languages.
• It is important to be able to recognize
  uncomputable and intractable problems.
• We need to know this in order to be a computer
  scientist, and not simply a computer programmer.