Theoretical Computer Science A

1
Theoretical Computer Science A

• Lecturer Dr Robert Crittenden
• Institute of Cosmology and Gravitation
• Mercantile 7.10, x5625
• Robert.Crittenden_at_port.ac.uk
• Course website
• www.tech.port.ac.uk/staffweb/crittenr/thcsa.html
• Check here for announcements, exam marks,
  lectures, course notes, old exams, useful links,
  etc.

2
Recommended books and references

• Lecture notes and tutorials online J. Britt
  and B. Bassett
• Theory of Computation An Introduction J.L.
  Hein, (1996).
• Introduction to the Theory of Computation M.
  Sipser, (1997).
• Feynman lectures on Computation R. Feyman
  (2000) focus on Chapter 3.
• Godel, Escher, Bach An Eternal Golden
  Braid D.R. Hofstadter, (2000).
• A New Kind of Science S. Wolfram (2002).

3
What the course is about

• A Models of computation Automata, Turing
  machines, etc. How does a computer
  work? What is a computer language? How is
  English different from C?
• B What are the limits of computing?
  What can we compute in principle?
  What can we compute in practice?
• This course is about learning to think abstractly
• Difficult stuff, so it is essential to keep up
  with lectures and tutorials!

4
The Challenge

• This is a tough course - usually with a high
  failure rate. You need to work and attend
  tutorials!
• It leads on to THCSB which is even more abstract
  and has had a very high failure rate work now
  and you will benefit for two years!
• The unseen benefits from this course will stay
  with you for many years so it is not all bad!

5
Course overview

• The course discusses two very closely related
  concepts models of computing and languages
• The computing models are simplified as much as
  possible, so as to boil them down to their most
  important elements. Some of the models will be
  more powerful than others.
• Languages are effectively sets of strings which
  are built by particular rules, called a grammar.
  The more complex the grammar, the more
  complicated the language can be.

6
Models of computing

• Finite state machines (automata) Pattern
  recognition Simple circuits (e.g. elevators,
  sliding doors)
• Automata with stack memory (pushdown
  autom.) Parsing computer languages
• Automata with limited tape memory
• Automata with infinite tape memory Called
  Turing machines Most powerful model
  possible Capable of solving anything that is
  solvable

7
Chomsky hierarchy of grammars

• Regular grammars
• Context free grammars
• Context sensitive grammars
• Unrestricted grammars
• Well define what these mean later, but the
  important point is that the grammars become more
  complex as we go down the list, and contain those
  above it.

8
Computers recognise languages!

• Computers can be made to recognize, or accept,
  the strings of a language.
• There is a correspondence between the power of
  the computing model and the complexity of
  languages that it can recognize!
• Finite automata only accept regular grammars.
• Push down automata can also accept context free
  grammars.
• Turing machines can accept all grammars.
• This is why we study both in this course!

9
The course structure

• Intro to languages
• Regular languages
• Finite Automata (finite state machines)
• Regular grammars
• Context free languages and pushdown automata
• Turing Machines
• Cellular Automata and the Game of Life

10
Introduction to Languages
11
Sets

• A set is a collection of things called its
  elements. If x is an element of set S, we can
  write this
• x ? S
• A set can be represented by naming all its
  elements, for example
• S x, y, z
• There is no particular order or arrangement of
  the elements, and it doesnt matter if some
  appear more than once. These are all the same
  set
• x, y, zy, x, zy, y, x, z, z

12
Combining Sets

• A set with no elements is called the empty set,
  or a null set. It is denoted by ? .
• If every element of set A is also an element of
  set B, then A is called a subset of B
• A ? B
• The union of two sets is the set with all
  elements which appear in either set
• C A ? B
• The intersection of two sets is the set with all
  the elements which appear in both sets
• C A ? B

13
Alphabets and strings

• A string a sequence of symbols that are placed
  next to each other in juxtaposition
• The set of symbols which make up a string are
  taken from a finite set called an alphabet
• E.g. a, b, c is the alphabet for the string
  abbacb.
• A string with no elements is called an empty
  string and is denoted ?.
• If A is an alphabet, the infinite set of all
  strings made up from A is denoted A.
• For example, if Aa, then A?, a, aa, aaa,

14
Languages

• A language is a set of strings.
• If A is an alphabet, then a language over A is a
  collection of strings whose components come from
  A.
• So A is the biggest possible language over A,
  and every other language over A is a subset of
  A.

15
Examples of languages

• Four simple examples of languages over an
  alphabet A are the sets ?, ?, A, and A.
• For example, if Aa then these four simple
  languages over A are
•  ?, ?, a, and ?, a, aa, aaa, .
• Recall ? is the empty string while ? is the
  empty set. A is an infinite set.

16
Example English

• The alphabet is A a,b,c,d,ex,y,z
• The English language is made of strings formed
  from A e.g. fun, excitement.
• We could define the English Language as the set
  of strings over A which appear in the Oxford
  English dictionary (but it is clearly not a
  unique definition).

17
Concatenation

• The natural operation of concatenation of strings
  places two strings in juxtaposition.
• For example, if then the concatenation of the two
  strings aab and ba is the string aabba.
• Use the name "cat " to denote this operation.
  cat(aab, ba) aabba.

18
Combining Languages

• Languages are sets of strings, so they can be
  combined by the usual set operations of union,
  intersection, difference, and complement.
• Also we can combine two languages L and M by
  forming the set of all concatenations of strings
  in L with strings in M.

19
Products of languages

• This new language is called the product of L and
  M and is denoted by L ? M.
• A formal definition can be given as follows
•  L ? M cat(s, t) s ? L and t ? M.
• For example, if L ab, ac and M a, bc,
  abc, then the product LM is the language
•  
• L ? M aba, abbc, ababc, aca, acbc, acabc.

20
Properties of products

• The following simple properties hold for any
  language L
•   L? ? ?? L L.
•   L ? ? ? ?.L ?.
•  The product is not commutative. In other words,
  we can find two languages L and M such that L ? M
  ? M? L.
• The product is associative. In other words, if
  L, M, and N are languages, then
• L ? (M ? N) (L ? M) ? N
•  

21
Powers of languages

• If L is a language, then the product L ? L is
  denoted by L2.
• The language product Ln for every n ? 0, 1, 2,
  is as follows
• L0 ?
• Ln L ? Ln-1 if n gt 0

22
Example

• For example, if L a, bb then the first few
  powers of L are
• L0 ?
• L1 L a, bb
• L2 L ? L aa, abb, bba, bbbb
• L3 L ? L2 aaa, aabb, abba, abbbb,
  bbaa, bbabb, bbbba, bbbbbb

23
Closure of a language

• If L is a language over A (i.e. L ? A) then the
  closure of L is the language denoted by L and is
  defined as follows
•  L L0 ? L1 ? L2 ?.
• The positive closure of L is the language denoted
  by L and defined as follows
•  
• L L1 ? L2 ? L3 ?.

24
L vs. L

• It follows that L L ? ?.
• But its not necessarily true that
• L L - ?.
• For example, if we let our alphabet be A a
  and our language be L ?, a, then
• L L.
• Can you find a condition on a language L such
  that L L - ??

25
Closure of an alphabet

• The closure of A coincides with our definition of
  A as the set of all strings over A. In other
  words, we have a nice representation of A as
  follows
•  
• A A0 ? A1 ? A2 ?.
•  

26
Properties of Closure

• Let L and M be languages over the alphabet A.
  Then 
• a)      ? ? ?
• b)      L L ? L (L)
• c)      ?? L if and only if L L
• d)      (L ? M) (L ? M) (L ? M)
• e)      L ? (M ? L) (L ? M) ? L