Theory of Computation

1
Theory of Computation ????
2
Instructor ??? E-mail yen_at_ee.ntu.edu.tw
Web http//www.ee.ntu.edu.tw/yen Time
220-510 PM, Tuesday Place BL 114 Office
hours by appointment Class web page
http//www.ee.ntu.edu.tw/yen/courses/TOC-2007.ht
m  
3
textbook

• Introduction to Automata Theory, Languages, and
  Computation
• John E. Hopcroft,
• Rajeev Motwani,
• Jeffrey D. Ullman,
• (2nd Ed. Addison-Wesley, 2001)

4
1st Edition
Introduction to Automata Theory, Languages, and
Computation John E. Hopcroft, Jeffrey D.
Ullman, (Addison-Wesley, 1979)
5
3rd Edition

• Introduction to Automata Theory, Languages, and
  Computation
• John E. Hopcroft,
• Rajeev Motwani,
• Jeffrey D. Ullman,
• (Addison-Wesley, 2006)

6
Grading

• HW/Project 20
• Midterm exam. 40
• Final exam. 40

7
Why Study Automata Theory?

• Finite automata are a useful model for important
  kinds of hardware and software
• Software for designing and checking digital
  circuits.
• Lexical analyzer of compilers.
• Finding words and patterns in large bodies of
  text, e.g. in web pages.
• Verification of systems with finite number of
  states, e.g. communication protocols.

8
Why Study Automata Theory? (2)

• The study of Finite Automata and Formal Languages
  are intimately connected. Methods for specifying
  formal languages are very important in many areas
  of CS, e.g.
• Context Free Grammars are very useful when
  designing software that processes data with
  recursive structure, like the parser in a
  compiler.
• Regular Expressions are very useful for
  specifying lexical aspects of programming
  languages and search patterns.

9
Why Study Automata Theory? (3)

• Automata are essential for the study of the
  limits of computation. Two issues
• What can a computer do at all? (Decidability)
• What can a computer do efficiently?
  (Intractability)

10
Applications
Pattern recognition
Prog. languages
Supervisory control
Computer-Aided Verification
Comm. protocols
Compiler
circuits
Quantum computing
...
Theoretical Computer Science Automata Theory,
Formal Languages, Computability, Complexity
11
Aims of the Course

• To familiarize you with key Computer Science
  concepts in central areas like
• - Automata Theory
• - Formal Languages
• - Models of Computation
• - Complexity Theory
• To equip you with tools with wide applicability
  in the fields of CS and EE, e.g. for
• - Complier Construction
• - Text Processing
• - XML

12
Fundamental Theme

• What are the capabilities and limitations of
  computers and computer programs?
• What can we do with computers/programs?
• Are there things we cannot do with
  computers/programs?

13
Studying the Theme

• How do we prove something CAN be done by SOME
  program?
• How do we prove something CANNOT be done by ANY
  program?

14
Example The Halting Problem (1)

• Consider the following program. Does it terminate
  for all values of n ? 1?
• while (n gt 1)
• if even(n)
• n n / 2
• else
• n n 3 1

15
Example The Halting Problem (2)

• Not as easy to answer as it might first seem.
• Say we start with n 7, for example
• 7, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5,
• 16, 8, 4, 2, 1
• In fact, for all numbers that have been tried
• (a lot!), it does terminate . . .
• . . . but in general?

16
Example The Halting Problem (3)

• Then the following important undecidability
  result should perhaps not come as a total
  surprise
• It is impossible to write a program that decides
  if another, arbitrary, program terminates
  (halts) or not.
• What might be surprising is that it is possible
  to prove such a result. This was first done by
  the British mathematician Alan Turing.

17
Our focus
18
Topics

• 1.  Finite automata, Regular languages, Regular
  grammars deterministic vs. nondeterministic,
  one-way vs. two-way finite automata,
  minimization, pumping lemma for regular sets,
  closure properties.
• 2.  Pushdown automata, Context-free languages,
  Context-free grammars deterministic vs.
  nondeterministic, one-way vs. two-way PDAs,
  reversal bounded PDAs, linear grammars, counter
  machines, pumping lemma for CFLs, Chomsky
  normal form, Greibach normal form, closure
  properties.
• 3.

19
Topics (contd)

• 3. Linear bounded automata, Context-sensitive
  languages, Context-sensitive grammars.
• 4. Turing machines, Recursively enumerable sets,
  Type 0 grammars variants of Turing machines,
  halting problem, undecidability, Post
  correspondence problem, valid and invalid
  computations of TMs.
•  

20
Topics (contd)

• 5. Basic recursive function theory
• 6. Basic complexity theory Various resource
  bounded complexity classes, including NLOGSPACE,
  P, NP, PSPACE, EXPTIME, and many more.
  reducibility, completeness.
• 7. Advanced topics Tree Automata, quantum
  automata, probabilistic automata, interactive
  proof systems, oracle computations, cryptography.

21
Who should take this course?

• YOU

22
Languages

• The terms language and word are used in a strict
  technical sense in this course
• A language is a set of words.
• A word is a sequence (or string) of symbols.
• ? (or ?) denotes the empty word, the sequence of
  zero symbols.

23
Symbols and Alphabets

• What is a symbol, then?
• Anything, but it has to come from an alphabet
  which is a finite set.
• A common (and important) instance is ? 0, 1.
• ? , the empty word, is never an symbol of an
  alphabet.

24
Computation
memory
CPU
25
temporary memory
input memory
CPU
output memory
Program memory
26
Example
temporary memory
input memory
CPU
output memory
Program memory
compute
compute
27
temporary memory
input memory
CPU
output memory
Program memory
compute
compute
28
temporary memory
input memory
CPU
output memory
Program memory
compute
compute
29
temporary memory
input memory
CPU
Program memory
output memory
compute
compute
30
Automaton
temporary memory
Automaton
input memory
CPU
output memory
Program memory
31
Different Kinds of Automata

• Automata are distinguished by the temporary
  memory
• Finite Automata no temporary memory
• Pushdown Automata stack
• Turing Machines random access memory

32
Finite Automaton
temporary memory
input memory
Finite Automaton
output memory
Example Vending Machines
(small computing power)
33
Pushdown Automaton
Stack
Push, Pop
input memory
Pushdown Automaton
output memory
Example Compilers for Programming Languages
(medium computing power)
34
Turing Machine
Random Access Memory
input memory
Turing Machine
output memory
Examples Any Algorithm
(highest computing power)
35
Power of Automata
Finite Automata
Pushdown Automata
Turing Machine
Less power
More power
Solve more computational problems