A Introductory Romp Through Automata and Computation

1
A Introductory Romp ThroughAutomataand
Computation
2
Topics

• Greetings
• Our Beginning (for Computation)
• Skipping Ahead with Three Major Concepts
• Regular Languages
• Context-Free Languages
• Recursively Enumerable Languages, Some Things
  Cant Be Done
• Summary with Chomskys Hierarchy
• Example Exercises if Time Permits

3

• Our Beginning
• Algorithm? What is that?
• Hilberts problems at the Paris conference in
  00
• Problem 10

4

• Skipping Ahead with Three Major Concepts
• Three Major Concepts in Automata
• Languages
• Grammars
• Automatons
• Well need some definitions (Definitions are
  good.)
• Languages
• What is a language? Easy to say?
• Some examples English, C, XML, Morse code
• Need some precision.
• Will need some preliminaries ideas, i.e.,
  definitions.

5

• Skipping Ahead with Three Major Concepts
• alphabet
• non-empty set of symbols.
• here, well denote these sets by the symbol ? , a
  capital sigma,
• (sigma for symbol, get it?)
• An important example ? 0, 1All algorithms
  can be expressed with that simple alphabet
• Non-example ? ?
• string
• a finite sequence of symbols from an alphabet ?
• Ex 0101
• Ex ?, the string with no symbols
• String theory ?
• substring, string length, prefix of a string,
  suffix of a string, powers of a string
• operations on strings concatenation of two
  strings, reverse of a string,

6

• Skipping Ahead with Three Major Concepts
• ? and ?
• ? w w is a non-empty string from ?
• ? ? ? ? // i.e., the empty string is in ?
• Finally,
• language -
• if ? is an alphabet, a language L on ? is a
  subset of ?
• Ex.
• ? (, ) ,
• L (k)k k is a non-negative integer
• Operations on languages
• On two or more languages Languages are sets so
  we have Set-theoretic operations ?, ?, - ,
  concatenation
• On a single language reverse, concatenation,
  powers of a language, positive () and star ()
  closures of languages.

7

• Skipping Ahead with Three Major Concepts
• Grammars
• a way to represent a language without listing all
  the strings
• described by 4 entities G (V, T, S, P) where
• V is a non-empty finite set of variables
• T is a non-empty finite set of terminals (think
  of T as the alphabet for the language the grammar
  generates
• S is a single variable called the start variable
• P is a finite set of productions, which specify
  how the strings are constructed
• Ex G (V, T, S, P) where V S, T (, ) ,
  S, and P is
• S ? (S) ?
• Read S can have the form (S) or S can have the
  form ? (substitution rules for S)

8

• Skipping Ahead with Three Major Concepts
• Grammars (contd)
• language generated by a grammar G - set of
  strings that result from applying the productions
  of the G.
• Ex G (V, T, S, P) where V S, T (, ) ,
  S, and P is
• S ? (S)
• S ? ?
• L(G) (k)k k a non-negative integer
• To see this, try to get the string ((()))
• just apply the first production 3 then
• the last production 1 time
• S ? (S) ? ((S)) ? (((S))) ? (((?))) ((()))
• Clearly can do this for any k

9

• Skipping Ahead with Three Major Concepts
• Automatons
• the third big idea
• think of an automaton as a machine
• Automaton Notes
• have a finite state control unit
• different types of automatons differ in the type
  of (temporary) storage
• Also differ in the way output can be produced.
• How They Work (more or less)
• Input Notes
• string of symbols from alphabet,
• cant be written
• Storage Notes
• can be read and written
• unlimited in general

• Output
• - acceptors yes/no
• or
• - transducers strings

10

• Skipping Ahead with Three Major Concepts
• How They Work (contd)
• Control Unit Notes
• takes the machine from one state to another
  depending on input and current state
• operates in discrete states
• next state determined by a transition function
  (can be thought of as rules)
• Operating (more or less) Procedure Notes
• Input a string w
• Output yes/no if acceptor
• string x if transducer
• state initial state q0
• while (not halted)
• q state, a next symbol in w
• state, output rule (q, a),
• Remind you of a computer?

11

• Regular Languages
• Different classes of languages obtained by
  putting restrictions on
• the storage mechanism of automatons and/or
• the form of the production rules of grammars,
• Regular languages can be described several
  (equivalent) ways. Ours will use the dfa.
• An Application
• detect programming language code constructs
  (variable names, etc.)
• Deterministic Finite Acceptor (dfa)
• an automaton where the transition function has a
  rule stating the next state for each (q, a) pair
  (where q is a state of the machine and a is an
  input symbol)

12

• Regular Languages
• More technically, dfa
• a set of 5 entities M (Q, T, ?, q0, F) where
• Q is a finite set of machine states
• T is a set of terminals (input symbols)
• ? ?(q, a) q1 next state of M for current
  state q and input symbol a
• q0 is the initial state of the machine
• F is a subset of Q that are accepting states
• Ex M (Q, T, ?, q0, F) where
• Q q0, q1, q2
• T 0, 1
• ? (the state transition rules) can be seen easily
  from the transition graph
• q0 is the start state
• F qf

13

• Regular Languages
• language accepted by a dfa
• the set of strings which when consumed end put
  the dfa in a final state
• What is the language accepted by M?
• Ans the set of all strings that start with any
  number of 1s (including no 1s) and ending in
  01, i.e., L(M) 101 w a string of 0s and
  1s
• regular language
• a language L (set of strings
• from an alphabet) for which
• there is a dfa M to accept it.

14

• Regular Languages
• Non-deterministic Finite Acceptors (nfa)
• an automaton where their transition function is
  not required to have a rule for each (q, a)
  pair.
• transitions can occur without any input symbol
  being consumed, i.e., transitions to the next
  state may occur on the empty string.
• language accepted by an nfa
• (same as for dfa) i.e, the set of strings, which
  when consumed, put the automaton in a final
  state.
• dfas are (trivially) nfas

15

• Regular Languages
• DFAs NFAs!
• dfas and nfas are equivalent
• By this we mean they accept the same languages
• Can interpret this to mean that if you are given
  any nfa M1, there is a dfa M2 such that L(M2)
  L(M1)

16

• Regular Languages
• Regular grammars
• left-linear grammar (llg) - a grammar G (V, T,
  S, P) where each production has the formA ? Bx
  or A ? x where A, B are variables and x is a
  string of T symbols (or ? )
• Ex. G given by these productions
• S ? B01
• B ? B0 B1
• L(G) w01 w any strings of 0s and 1s

17

• Regular Languages
• Regular grammars
• right-linear grammar (rlg) - a grammar G (V, T,
  S, P) where each production has the formA ? xB
  or A ? x where A, B are variables and x is a
  string of T symbols (or ? )
• Ex. G given by these productions
• S ? 1S 0S A
• A ? 01
• L(G) w01 w any strings of 0s and 1s
• regular grammar a left or right linear grammar.

18

• Regular Languages
• Results
• Languages generated by rlgs are precisely the
  same as languages generated by llgs.
• Languages generated by regular languages are
  precisely those languages than can be accepted by
  dfas.
• So, we have If ? is a alphabet, the following
  are equivalent
• regular language on ?
• languages on ? accepted by dfas
• languages on ? accepted by nfas
• languages on ? that can be generated by
  right-linear grammars
• languages on ? that can be generated by
  left-linear grammars

19

• Context-Free Languages
• context-free grammar (CFG)
• a grammar G (V, T, S, P) such that all
  productions have the form A ? x where x is a
  string from V ? T.
• context-free language
• a language for which there is a CFG to generate
  it.
• Notes
• regular grammars are context-free, i.e., regular
  grammars are a restricted form of CFG hence,
• Regular languages (RLs) are CFLs.

20

• Context-Free Languages
• Important Application programming and markup
  languages can be specified by context-free
  grammars.
• Ex G (V, T, S, P) where V S, T (, ) ,
  S, and P is given by
• S ? (S)
• S ? ?
• L(G) (k)k k a non-negative integer
• Same example as earlier. (Section of many
  programming languages.)

21

• Context-Free Languages
• NPDAs
• stack a data structure whose data entries and
  removals are from only one end.
  (Last-in-first-out).
• non-deterministic pushdown automaton (npda) a
  machine whose temporary storage is a stack.

22

• Context-Free Languages
• can be described more formally by , a 7-tuple M
  (Q, T, ? , ?, q0, ?, F) where
• Q (finite no. of states), T (finite input symbol
  set), q0 (start state), F (set of final states)
  are as for dfas and nfas,
• ? is the set of stack symbols (not in dfas or
  nfas), and
• ?is the symbol at the bottom of a stack at the
  beginning of computation
• ? is the transition function (different from
  dfas and nfas)

23

• Context-Free Languages
• The state transitions of an npda have different
  characteristics than dfas or nfas. An example
  
• Notes
• on each state transition arrow i, t/r
• where
• i is the next input symbol or ? (i.e., an input
  symbol is not required to be consumed)
• t is the symbol on the top of the stack
• r is a string from ? that replaces t on the top
  of the stack (can be ?)

24

• Context-Free Languages
• Notes (contd)
• for a particular (i, t) there may be more than
  one transition from a particular state
• hence non-deterministic
• L(M) akbk k any non-negative integer
• So if a ( and b )
• L(M) (k)k k any non-negative integer
• Try to trace through aabb.

25

• Context-Free Languages
• deterministic pushdown automaton (dpda) like an
  npda but the transition function is restricted
  such that
• for any state q, input symbol a, and stack symbol
  b, there can be at most one transition from state
  q
• if there is a ? - transition from a state q when
  the top of the stack is b, there can be no other
  transition from q when the top of the stack is b
• dpdas are restricted dpdas and dfas are
  restricted dpdas.

26

• Context-Free Languages
• Ex
• this is a dpda that accepts the same language as
  the previous npda
• L(M) akbk k any non-negative integer
• So if a ( and b )
• L(M) (k)k k any non-negative integer
• Again, try to trace through aabb

27

• Context-Free Languages
• Facts
• npdas CFGs
• the set of languages generated by CFGs is
  precisely those languages which are accepted by
  npdas.
• so npdas accept CFLs
• dpdas and npdas are not equivalent.
• the languages accepted by dpdas are called
  deterministic context-free (DCFLs)
• We have so far RLs ? DCFLs ? CFLs

28

• Turing Machines, Recursively Enumerable
  Languages, Some Things Cant Be Done
• Now were back to the beginning, sort of
• Alan Turing was working on Hilberts problems (as
  were many others) and in 1936 had a paper that
  introduced the
• (Standard) Turing Machine (STM) an automaton in
  which the temporary storage, input string, and
  output string (if any) are on a single infinite
  length tape

29

• Turing Machines, Recursively Enumerable
  Languages, Some Things Cant Be Done
• The STM automaton
• We assume that, at the beginning of operation,
• the tape begins with the input string already on
  it
• the STM knows where the first input symbol
  (leftmost one) is
• the input string has finite length
• the infinite portions of the input tape not
  containing any input symbols has blank symbols

30

• Turing Machines, Recursively Enumerable
  Languages, Some Things Cant Be Done
• How It Works, Transition Diagram, Example
• Ex STM as an acceptor
• Accepts L(M) akbk k any non-negative
  integer
• Notes
• same language as the npda and dpda examples but
  looks much different
• if we were instead to make our primary focus the
  string on the tape after the STM halted, we would
  be interpreting the STM as a transducer,
• the notataion on the transition graph edges (i.e,
  b/x ? (or ?) ) are to be interpreted as
• read next input symbol b from the tape
• replace b with x
• traverse to the right (?) (or left (?)) on the
  tape
• Try to trace aabb

31

• Turing Machines, Recursively Enumerable
  Languages, Some Things Cant Be Done
• How It Works, Transition Diagram, Example
• Ex STM as a transducer.
• If we focus on the contents of the tape at the
  end of execution (after the TM halts) then we are
  viewing the machine as a transducer.
• Note Noone programs TMs for the practicality
  of it. They are for theoretical considerations.

32

• Turing Machines, Recursively Enumerable
  Languages, Some Things Cant Be Done
• Another Transition Diagram and Example
• An STM may or may not halt. To halt means the
  machine enters a state for which there is no
  transition to another state with the given input
  symbol.
• Ex STM that doesnt halt. (Try to trace
  through and watch it cycle.)

33

• Turing Machines, Recursively Enumerable
  Languages, Some Things Cant Be Done
• Playing Around With the Tape Mechanism
• Can a more powerful machine (accept more
  languages) be constructed by altering the tape
  mechanism?
• Add More Tapes
• a tape to simulate RAM
• a tape to simulate hard disk
• a tape to simulate any ol input/output device
• Taking on the look of computers as we know them
• More Powerful?This might be thought to make a
  machine that is more powerful than the STM
• more powerful, i.e., in the sense, it accepts
  more languages,
• can perform more transducer functions,
• NOT the case
• But it that can be shown that the machine is
  indeed not more powerfulnot to be the case

34

• Turing Machines, Recursively Enumerable
  Languages, Some Things Cant Be Done
• Turings Thesis
• Any computation that can be carried out by
  mechanical means can be carried out by some
  STM.
• Hence,
• if you know of something a computer can do, then
  there is an STM to do it (Turings thesis), also,
  or
• if you can think of a mechanical
  computation/algorithm to solve a problem, an STM
  can do it, hence
• if youre asked whether some language is accepted
  by a STM, ask yourself if you could think of a
  way to verify that a particular string of the
  language is indeed in the language. If so, then
  there is an STM to accept that language.

35

• Turing Machines, Recursively Enumerable
  Languages, Some Things Cant Be Done
• TM not very versatile it seems
• (So far it looks like for any particular
  mechanical computation we can find an STM to do
  it, but that that STM can do only that particular
  mechanical computation, and not others.)
• That doesnt seem very useful past that one
  computation
• Need a machine that can execute many (different)
  computations
• Like our computers, which can run all types of
  programs
• What if we could make a TM M1 be able to simulate
  the operation of any STM M2 (and its transition
  function) on any input I to M2?

36

• Turing Machines, Recursively Enumerable
  Languages, Some Things Cant Be Done
• Universal Turing Machine
• A three-tape TM U where
• Tape 1 can be made to contain the transition
  function of any STM M via a sequence of 0s and
  1s
• Tape 2 will contain the input I to M.
• Tape 3 contains a copy of the current state of M
  during the simulation of M by U.
• U executes by
• From tape 3, determining a current state of M
• From tape 2, determining the next input symbol
• From tape 1, determining a next state of M and
  how to alter and move on tape 2
• altering tapes 3 and 2 accordingly to what it
  found out from tape 1
• repeating these steps
• Then U can be seen to simulate any STM and a
  string can be viewed to represent a mechanical
  computation/algorithm
• Since multi-tape TMs are no more powerful than
  STMs, there is an STM to do what U does.

37

• Turing Machines, Recursively Enumerable
  Languages, Some Things Cant Be Done
• Conclusions, notes
• U can be seen to behave very much like how we see
  computers to behave, and
• there is a STM that behaves very much like
  computers as we know them
• universal Turing machine very important idea in
  the development of computers
• Well now,
• Looks like STMs can make any computation we
  could conjure and state succinctly.
• It that right?

38

• Turing Machines, Recursively Enumerable
  Languages, Some Things Cant Be Done
• Halting Problem
• One way of stating it
• Given an STM M and an input w for M, will M halt
  (i.e., enter a state for which there are no more
  transitions defined in the transition)?
• Another way
• Given an STM M and an input w for M, is there a
  mechanical computation/an algorithm/an STM that
  tells us whether or not M halts.
• The hard cruel fact
• It was shown that there is no such
  TM/algorithm/mechanical computation that
  satisfies the halting problem
• So,
• there are problems that are easily stated for
  which there is no mechanical computation/algorithm
  /program to solve it.

39

• Turing Machines, Recursively Enumerable
  Languages, Some Things Cant Be Done
• Chomskys Hierarchy (almost)
• recursively enumerable languages (REL)
• a language L for which there is an STM M such
  that for each string in L, M halts in a final
  state. (Note that there is no requirement for
  what M does on a string not in L. It may or may
  not halt. If M were required to halt, the
  language is said to be recursive.)
• unrestricted grammar
• a grammar G (V, T, S, P) is said to be
  unrestricted if each production is of the form
• L ? R
• where
• L is a non-empty string of variables and
  terminals, and
• R is a non-empty string of variables and
  terminals or ?

40

• Turing Machines, Recursively Enumerable
  Languages, Some Things Cant Be Done
• Chomskys Hierarchy (almost) (contd)
• Facts
• the languages generated by the unrestricted
  grammars is precisely the recursively enumerable
  languages.
• a CFG is a UG CFLs ? RELs
• RLs ? DCFLs ? CFLs ? RELs (Chomsky)

41
THE END