THCSA Review

1
THCSA Review

• Final exam
• Friday, May 28
• Liongate PCG room

2
THCSA exam

• Nature of the exam
• 1 hour, two of three questions.
• No books, notes or calculators.
• Show all relevant work for partial credit.
• Read questions carefully, choose the one you can
  score highest on.
• Study materials
• Lectures course notes
• Tutorials solutions
• Past exams, websites
• Books on reserve Sipser, Linz, Hopcroft, etc.

3
THCSA overview

• Formal languages and models of computing
• Language and grammar basics
• Regular languages and regular expressions
• Chomsky hierarchy of languages
• Models of computing
• Finite automata (DFAs and NFAs)
• Pushdown automata (PDAs)
• Turing machines
• Cellular automata

4
Introduction to languages

• A language is a collection, or set of strings
• The set of characters which make up the strings
  is called the alphabet, e.g. A a,b.
• Four simple examples of languages over the
  alphabet A are
• the null (empty) set, ?
• the zero length string ?
• the alphabet, A
• the closure of the alphabet, A (all strings)

5
Combining languages

• We discussed three main ways of combining
  languages
• Union L ? M contains all elements which
  appear in either L or M.
• Product contains all possible ways of
  combining strings of two sets
• L ? M cat(s, t) s ? L and t ? M.
• Closure contains all possible products of a
  set with itself
• L L0 ? L1 ? L2 ? (L0 ?)
• Other ways intersection, positive closure, etc.

6
Typical kinds of questions

• Apply the notions of union, product and closure
  to simple languages, e.g.
• 1) find L ? M , L ? M , L if L a,bb and M
  ab, a, ba
• 2) find L such that L ? L aa, abb, bba, bbbb
• Use the definitions of union, product, and
  closure to prove some formal relations for sets,
  e.g.
• Show L L ? L

7
Basics of grammars

• Another way of describing languages is through a
  set of production rules called a grammar
• Grammars consist of four parts
• Non-terminals symbols of the grammar,
  usually in upper case
• Terminals symbols of the language to be
  made, usually numbers or lower case
• Start symbol - special non-terminal (usually S)
• Production rules set of rules for taking
  some combinations of symbols into others
• ? ? ?

8
Infinite languages

• Many grammars can lead to the same language.
• The grammar for a finite language can be made by
  simply having a production for each string.
• To produce an infinite language, a grammar must
  be recursive, where some productions are used
  repeatedly.
• A production is called recursive if its left side
  occurs on its right side. A grammar is indirectly
  recursive if it takes two or more steps for a
  symbol, e.g. S, to reproduce itself.

9
Combining grammars

• If we have grammars for two languages, M N
• M A ? ? N B ? ?
• we can find grammars to combine the languages.
• Union M ? N S ? A B
• followed by the grammars of M N
• Product M ? N S ? AB
• followed by the grammars of M N
• Closure M S ? ? SA
• followed by the grammar of M

10
Ambiguity and meaning

• The meaning of a string often depends on how it
  was derived, like 3 4 2.
• In some languages, each string has only one
  possible derivation, so the meaning is
  unambiguous.
• A grammar is ambiguous if some string has two
  different derivations (parse trees).
• Ambiguity is something to avoid in computer
  languages!

11
Typical questions

• Given an infinite language, find a grammar which
  will produce it.
• Find the language described by a particular
  grammar.
• Determine if a particular string can be produced
  by a given grammar.
• Decide whether a grammar is recursive or
  indirectly recursive.

12
Regular languages

• A simple family of languages can be constructed
  which are called regular. They are made by
  induction, starting with a simple basis
• Basis ?, ? and a are regular languages for
  all a ? A.
• More complex ones are made by combining these. If
  L and M are regular, then so are
• L ? M, L ? M and L
• A shorthand way of describing regular languages
  is by using regular expressions.

13
Regular expressions

• Regular expressions have an equivalent basis (?,
  ?, a) and are also built inductively.
• Regular Expression Language 
• R S L (R S ) L (R ) ? L (S )
• R ? S , R S L (R ? S ) L (R ) ?L (S )  
• R L (R ) L (R )
• Example
• the language of the regular expression a bc
  is
• L(a bc) a, b, bc, bc2, ..., bcn, ....

14
Typical kinds of questions

• Given a regular language, find a corresponding
  regular expression, and vice versa.
• Prove simple formal relations for regular
  expressions, e.g.
• R? ?R R
• R R R

15
Beyond regular languages

• Chomsky discovered that there are many classes of
  language, and that they can grow in complexity,
  with regular languages being the simplest.
• Chomsky hierarchy
• Regular
• Context-free
• Context sensitive
• Phrase structure

16
Grammar classifications

• All the productions for a particular class of
  grammar have to be of a certain type
• Regular S ? ? w T wT
• at most 1 NT on right side
• Context free S ? ?
• must have 1 NT on left side
• Context sensitive ? ? ? ? ? ?
• no shortening rules
• Phrase structure ? ? ?
• no restrictions!

17
Typical kinds of questions

• Given a grammar, decide what kind of language it
  is.
• List the various kinds of grammar in order of
  increasing complexity and give their allowed
  production rules.

18
Models of Computing

• There are computing models which are able to
  recognise each level of language in the Chomsky
  hierarchy. As the languages grow more complex,
  the computing models become more powerful.
• DFA and NFA - Regular languages
• Pushdown automata - Context-free
• Bounded Turing Ms - Context sensitive
• Turing machines - Unrestricted

19
Finite automata
Input
Finite automaton
Control

• The simplest model of computing is the finite
  automaton, which consists of a finite control
  unit and an input string.
• Some FAs are deterministic (DFAs) while others
  are non-deterministic (NFAs).
• DFAs and NFAs are both capable of recognising
  the regular languages.
• Finite automata are useful in simple circuits and
  in searching for particular patterns in data.

20
Elements of finite automata

• Finite automata are described by
• 1) The set of states
• 2) Which is the start state?
• 3) Which are the final (accepted) states?
  Double circled!
• 4) What are the transition functions?
• Transition functions are represented by
• T(i, a) j

21
Determinism vs non-determinism

• Each DFA node has one transition for each letter
  of the alphabet NFAs can have more than one for
  a particular letter, or even none. NFAs can
  also have transitions with no letter ?.
• DFAs are always in a unique state after a
  particular string, while NFAs could be in many
  possible states.
• We can find a DFA equivalent to any NFA, but it
  might have many more states. The DFA needs a
  state to correspond to any set of states which
  the NFA could be in.

22
Beyond regular languages

• If a language is regular, then there is a DFA
  which recognises it. We can use this to tell if
  some languages are regular.
• The pumping lemma says that if a string is long
  enough, it can be written in the form xyz, so
  that all strings of the form
• xykz for all k ? 0
• are also in the language. If this cannot be
  done, then the language is not regular!
• This follows because the string y arises from an
  internal loop of the finite automaton, and we can
  go around the loop any number of times.

23
Representing regular languages

• There are many ways of representing regular
  languages
• 1) Listing of elements
• 2) Regular expression
• 3) NFAs or DFAs
• 4) Regular grammar
• We discussed how to go between any two of them
• NFAs ? DFAs Reg. exp. ? NFAs
• We also showed how to find a unique, minimum
  state DFA by eliminating indistinguishable
  states.

24
Typical kinds of questions

• Draw a DFA or an NFA to recognise a particular
  regular language.
• Is a particular automaton deterministic?
• Describe one way to check if a language is
  regular.
• Give an application for a finite automaton.
• Transform one description of a particular regular
  language (i.e., a DFA, NFA, regular expression,
  grammar) into another. For example, change an
  NFA into a DFA.
• Does a particular DFA have the minimum number of
  states? If not, find the minimum state DFA.

25
Pushdown Automata (PDA)
Pushdown automaton
Input
Control
Read/write stack memory Last in, first out LIFO

• PDAs are like finite automata, but they have an
  additional stack memory which allows them to
  recognise a larger family of languages, the
  context free languages (e.G. An bn ).

26
Designing PDAs

• The finite state control is identical to that of
  the finite automata, but PDAs also require
  commands for the stack pop, push(?), nop
• Based on its present state, i, the input
  character, a, and the top stack character, C, the
  PDA decides the next state, j, and what to do to
  the stack?
• Shorthand ? i, a, C, push(A), j?
• Graphically

a, C / push(A)
i
j
27
PDAs

• To follow the actions of the PDA, we need to
  follow the present state, the remaining input and
  the stack contents (these are known as the
  instantaneous description).
• For example ?0, abba, YZ?
• Like finite automata, PDAs can be deterministic
  or non-deterministic. Unlike previously, it
  makes a difference for PDAs. Non-deterministic
  PDAs accept all context free languages but
  deterministic PDAs cannot!

28
Applications of PDAs

• Most computer languages are context free, and the
  compilers of these languages contain a parser
  which is generally a deterministic PDA.
• The parser checks the program syntax is ok, i.e.
  that the program makes sense, and figures out
  what the program means.

29
Typical PDA questions

• Describe a stack memory and the three basic
  operations for it.
• Construct a PDA, graphically and using the
  transition functions, to recognise a particular
  context free language.
• Construct a PDA to do a particular operation on
  its stack.
• Describe a practical application of a pushdown
  automaton.

30
Turing Machines
Infinite tape
Tape head
Control
Usual finite state control

• Turing machines have an infinite read/write tape
  rather than the stack of PDAs. Also, the input
  begins on the tape, so no separate input is
  needed.
• Turing machines recognise the phrase structure
  languages, i.e. any grammatical language. They
  are usually used for performing tasks.

31
Turing machine tape

• For Turing machines, we have to keep track of
  what is read from and written to the tape, and
  also how the tape head is moved for the next step
  (one cell to the left, right or stay).
• Turing machines stop when the halt state is
  reached, or when there is no valid move
  available.
• The input starts on the tape, with the rest of
  the cells blank. By convention, the tape head
  starts on the leftmost non-empty cell.

32
Turing machine instructions

• We can represent a full TM instruction as
• ?i, a, b, L, j?
• If the current state of the machine is i,
• and if the symbol in the current tape cell is a,
  
• then write b into the current tape cell, move
  left one cell, and go to state j.
• Graphically,

33
Turing machine variations

• Turing machines can also be deterministic or
  non-deterministic, but this does not make them
  any more or less powerful.
• However, if the tape is restricted so that you
  can only see use the tape with the input, the TM
  becomes less powerful (linear bounded TM) and can
  only recognise context sensitive languages.
• Many other TM variations are equivalent to the
  original TM. This includes multi-track,
  multi-tape, multi-head, multi-dimensional tape
  and the off-line Turing machine.

34
Universality

• Turing machines are useful because of their
  universality. We can design a universal Turing
  machine (UTM) which is programmable and can do
  anything that any other Turing machine can do.
• In fact, we believe the Church-Turing thesis
• Anything which is intuitively computable can be
  computed by a Turing machine.
• No one has ever invented a more powerful
  computing model than a Turing machine.
• However, there are some things no computer can
  do!

35
Typical kinds of questions

• Why is a TM more powerful than a PDA?
• Construct a TM to do a simple task, e.g.
  addition, etc.
• What does the Church-Turing thesis assert?
• What is a universal Turing machine?
• What is a linear bounded TM and what languages
  can it recognise?

36
Cellular automata (not examinable)

• Similar to a Turing machine where each cell
  evolves in every step. This is usually based on
  simple rules which depend on the contents of the
  cell and its surrounding cells.
• The rules can lead to many different behaviours
• regular, fractal, random or chaotic.
• They can be in one or more dimensions. E.g. the
  game of life is a two dimensional CA.
• CAs can be deterministic or probabilistic, and
  can be used to model problems with local
  interactions, e.g. traffic flow, crowd movement,
  etc.