Models of Computation: Automata and Formal Languages

1
Models of Computation Automata and Formal
Languages

• Sam M. Kim

2
What is it about?
3
Where do we use it?
almost
Everywhere!
Everywhere!
Everywhere!
Everywhere!
Everywhere!
Everywhere!
4
Preliminaries

• Notational Convention
• Leading lower case English letters a, b, c,
  denote symbols.
• Trailing lower case English letters .., u, v, w,
  x, y, z denote strings of symbols.
• Upper case English letters, ? and ? denote sets.
• For a string x, x denotes the length of x.
• ? (or ?) denotes the null string.
• ? , which is called Kleene star (or closure) of
  ? , denotes the set of all strings
• over the alphabet ? including the null
  string , ?, and ? ? - ?.
• xi denotes the repeating string of x i times,
  and xR denotes the reverse of string x. For
  example, (ab)3 ababab, (ab)0 ?, (abcd)R
  dcba.

5
Preliminaries (conted)

• Set Specification
• Explicit A 2, 8, ? a, b, c
• By set properties A x property list of x
• Examples
• A x x ? a, b and x has no
  consecutive as
• B n n is an even number
• C xxR x ? 0, 1 00, 11, 0110,
  1001, 1111, 001100, . . .
• D anbncn n ? 0 ?, abc, aabbcc,
  aaabbbccc, . . .

6
Proof Techniques

• The truth table for implication if P then Q
  (denoted by P ? Q ) is defined
• as follows

• A proposition is a declarative statement that is
  either true or false.
• We are interested in proof techniques for
  propositions of he form
• P ? Q, or P ? Q (i.e., P ? Q and Q ? P ).

7
Proof Techniques (conted)

• 1. Trivial proof Prove, if possible, that Q is
  true (i.e., tautology). Notice that
• if Q is true, then P ? Q is true independent
  of P.
• Example Suppose that the premise
  Graduates of the Computer
• Science Department get a good job is
  true. Then the statement
• If the weather is good then the
  graduates of the Computer Science
• Department get a good job. is true.
• 2. Vacuous proof Prove P is false. Notice
  that if P is false, then P ? Q is
• true independent of Q.
• Example If elephants fly, then they
  will lay eggs.

8
Proof Techniques (contd)

• 3. Indirect proof Prove that Q ? P is true.
  Notice that ( P ? Q ) ? (Q ? P).
• Example The proof for If it rained, then
  the lawn will be wet. is equivalent
• to the proof for If the lawn is not wet,
  then it did not rain.
• 4. Proof by cases Let P1, P2, and P3 be
  propositions. Then the proposition
• (P1 ? P2 ? P3) ? Q
  
• can be proved by three cases P1 ? Q, P2 ?
  Q, or P3 ? Q, because
• (P1 ? P2 ? P3) ? Q ? (P1 ? Q) ?
  (P2 ? Q) ? (P3 ? Q)
• 5. Proof by contradiction We want to prove that
  a proposition P is false. With the
• assumption that P is true, we show that P ?
  Q is incorrectly false, contradicting a
• proven fact. So we must lay the blame on the
  assumption that P is true.

9
Proof Techniques (conted)

• Dealing with universal/existential
  quantifications
• Let P(x1, x2, ., xn), n ? 1, be a
  predicate with Boolean variables x1, x2, , xn.
• For example
• GT(x, y) true if x gt y, false otherwise
• Prime(n) true if n is a prime number,
  false otherwise.
• We will use ? and ? to denote for all (or
  for every) and there exists, respectively.
  Symbol ? is called the universal quantifier, and
  ? the existential quantifier.

10
Proof Technique (conted)

• 6. Proof by example To prove that a
  proposition of the form ? x P(x)
• (i.e., there exists an x which satisfies
  P(x)), it is enough to show an example x which
  satisfies P(x). However, to disprove the
  proposition we should present
• a generalized proof that P(x) is false for
  arbitrary x, which is not easy.
• 7. Proof by counter example To disprove that a
  proposition of the form ? x P(x) (i.e., for
  all x, P(x) is true), it is enough to show an x
  which does not satisfy P(x).
• 8. Proof by generalization To prove a
  proposition of the form ? x P(x), we need to
  show that P(x) holds for arbitrary x.

11
Proof Techniques (conted)

• 8. Proof by induction Proof by induction is
  performed in three steps
• (1) base, (2) hypothesis (or assumption),
  and (3) induction.
• Example Prove that ? n ? 0 ?ni0 i n(n1)
  /2
• (1) Base when n 0, ? 0i0 i 0, and
  n(n1)/ 2 0. Thus the equality holds.
• (2) Hypothesis Suppose that ? m lt n it
  satisfies ?mi0 i m(m1) /2.
• (3) Induction step We prove that for n m1,
  ?ni0 i n(n1) /2.
• Clearly, ?ni0 i ?mi0 i n, and ?mi0
  i m (m1)/2 by the hypothesis.
• It follows that
• ?mi0 i n m(m1)/2 n m(m1)/2 (m1)
  (m1)(m2)/2 n(n1)/2.

12
Proof Techniques (conted)

• Constructive proof vs non-constructive proof
  Constructive proof actually shows the information
  that the problem requires, while non-constructive
  proof
• shows the existence of such information
  indirectly.
• Example We want to prove the statement
  There is a prime number p gt 2.
• Constructive proof 5 is a prime, which is
  greater than 2.
• Non-constructive proof Every natural
  number can be represented as a
• product of prime numbers. This fact
  implies that, if there is no prime
• number greater than 2, any natural
  number n should be equal to 2i for some
• integer i. However, we know that there
  exist natural numbers that are not
• power of 2, which contradicts to a know
  fact. It follows that there exists a
• prime number greater than 2.

13
Proof Technique (conted)

• Other useful Proof techniques
• Pigeonhole Principle Suppose that we have n
  pigeonholes and n1 items to be placed in the
  pigeonholes. Then however the items placed, at
  least one pigeonhole will contain at least two
  items. (This can be easily proved by the proof by
  induction technique.)
• 9. Proof by the pigeonhole principle (example)
  Show that in a graph G with n vertices, every
  path of length greater than or equal to n has a
  cycle.
• Proof Let the nodes of G be the
  pigeonholes and the nodes appearing on the
• path be the items. We put an item in a
  mail box (i.e., a node) if that item appears on
  the path. If a path length is greater than or
  equal to n, then it has more than n items. By the
  pigeonhole principle at least one box will
  contain
• at least two items (i.e., nodes on the
  path), which implies a cycle on the path.

14
Proof Techniques (conted)

• 10. Proof by counting This technique is used
  to demonstrate that particular set is larger than
  others.
• Example Prove that, for all n ? 1, the
  rows and columns of n ? n bit matrix do not
  exhaust all possible binary numbers of length n.
• Proof. Obviously, there are 2n different
  binary numbers of length n, but only 2n rows and
  columns in n ? n bit matrix. Since 2n lt 2n ,
  for all n gt 2, the statement is true for all n gt
  2. For n 2 or 1, we can prove by exhaustive
• argument (this part of the proof is left
  for the reader).

15
Proof Techniques (conted)

• 11. Diagonalization technique This is a
  technique that, given a set of n lists, each of
  length m, construct a list of length m which is
  not contained in the set.
• Example Find a binary number of length n
  which does not appear on any column or row of an
  n ? n bit matrix (recall the preceding example).
• Solution Let mii be the complement of the
  bit at i-th column and i-th row.
• The matrix does not contain the binary
  number v m11m22mnn.
• Clearly, i-th row or column has bit mii at
  its i-th position, while v has mii,
• the complement at the same position.
  Hence, there is no n bits number in
• the matrix that is equal to v.

16
A Rumination on the Proof by Induction

• Using the proof by induction technique, prove
  that the pigeonhole principle is true.

(2) Induction hypothesis Suppose that the
pigeonhole principle is true for any number of
pigeonholes n lt n. (3) Induction step We show
that when the number of pigeonholes is n1 and
the number of items m gt n1, the principle
also holds. Consider the array of n1
pigeonholes as shown below. Case (a) When the
last pigeonhole contains no more than one item
(i.e., either zero or one item). The first n
pigeonholes together contain more than n items.
By the above induction hypothesis, among the
first n pigeonholes there exists at least one
pigeonhole which contains more than one item. The
principle holds. Case (b) When the last
pigeonhole contains more than one item. This last
pigeonhole witnesses that the principle is true.
17
A remark on the proof by induction technique
The proof by induction technique is a good
approach to show a statement of the form
something happens for all integers n ? k,
where something can be the pigeon principle,
the Sperners lemma (see the homework), or a
mathematical equation. If it is understood, the
lower bound k does not appear in the statement,
like in the pigeonhole principle(k 1) and the
Sperners lemma(k 2). The logic involved in
the proof is like going up the stairway. Imagine
that we stand at the bottom of a stairway, and
know how to move one step up from any i-th stair
to (i1)-th stair. Then we know how to move up to
n-th stair, which is our destination. So the
proof by induction proceeds as follows 1.
Induction base we prove that something can
happen for n k, i.e., we show we can go to the
bottom of the stairway. Now, if we can show
that if it happens for any n, k ? n lt n, it
can also happen for n 1, we are all done! We
know that it happens for k by the induction
base. Since it happens for k, by the above proven
fact it should also happen for k 1. Since It
happens for k 1, it should also happen for k
2, and so on. Finally, since it happens for n
1, it should also happen for n, the final
destination. So they usually set up this logical
ground with the following so called induction
hypothesis 2. Induction hypothesis suppose that
something happens for all nlt n (to be
specific, it is k ? n lt n). Then the proof
shows in the final induction step that if it
happens for n lt n, it also happens for n 1,
which includes n. 3. Induction step Supposing
that something happens for any nlt n, we show
that it also happens for n 1.
18
A Rumination on the Proof by Induction
Prove that ? n ? 0 ?ni0 i n(n1) /2
(3) Induction step We prove that for n
n1, ?ni0 i n(n1) /2.
(2) Hypothesis Suppose that ? n lt n
it satisfies ?mi0 i n(n1) /2.

• Base when n 0, ? 0i0 i 0, and n(n1)/ 2
  0.
• Thus the equality holds.

0 1 2 . . . . . . .
n (n1)
n(n1)/2
(n1) (n1)(n2)/2 n(n1)/2
19
Rumination (conted)
Prove the pigeonhole principle for n ? 1
pigeonholes m gt n items.
(3) Induction we prove The principle holds for n
n1 holes and m gt n items.
(2) Hypothesis suppose that the principle Holds
for all n lt n holes and m gt n items.