91'304 Foundations of Theoretical Computer Science

1
91.304 Foundations of (Theoretical) Computer
Science

• David Martin
• dm_at_cs.uml.edu

This work is licensed under the Creative Commons
Attribution-ShareAlike License. To view a copy of
this license, visit http//creativecommons.org/lic
enses/by-sa/2.0/ or send a letter to Creative
Commons, 559 Nathan Abbott Way, Stanford,
California 94305, USA.
2
About These Notes

• Designed to be used with Sipsers Introduction to
  the Theory of Computation
• Available through Lecture notes link on course
  web page
• Note that examples various other things are not
  included here
• Prepared with TeXPoint, see http//raw.cs.berkeley
  .edu/texpoint/

3
Basic Objects and Notation

• N 1, 2, 3, 4, ...
• Some texts include 0 Sipser doesn't
• Z ..., -2, -1, 0, 1, 2, ...
• Z N
• R set of all real numbers
• Q set of all rational (quotient) numbers
• PosEven 2n n 2 N
• x predicate(x)
• Tiny constraints are sometimes added to the left
  of , as in
• PosEven n 2 N n 2 0
• Sometimes is used instead of

4
Scope of Intentional Notation

• Variables inside x pred(x) are local
• Think of the specification as a mathematical
  program
• We will see many programming languages this term
  DFAs, NFAs, Regex, PDAs, TMs, C, ...
• Mathematical notation is a type of precise
  specifier i.e., a programming language
• Turns out it is far more powerful than our
  ordinary programming languages we'll prove this
  later

5
Scope of Intentional Notation

• 2n n 2 N n 2 N n 2 0
• These are two different programs that produce the
  same set

different vars
6
Set Operations

• Union Disjunction Or
• 0, 3, 6, 9 0, 2, 4, 6, 8 ?
• Å Intersection Conjunction And
• 0, 3, 6, 9 \ 0, 2, 4, 6, 8 ?

7
Set Operations

• ComplementPosOdd PosEvenc ... depending
• For a set A,Universe is implicit. Be
  careful!!!
• Set difference PosOdd N - PosEven 1, 3,
  5, ...
• For sets A B,
• A B x x 2 A and x ? B A Å Bc

8
More Set Operations

• CardinalityIf A is a set, then 1 is not very
  precise, but oh well. Well improve upon this
  later

9
More on Sets

• The empty set
• The number of matters
• ? ? ,
• Elements of a set
• 5 2 1, 2,3, , 5 , 2
• 3 ? 1, 2,3, , 5 , 2
• 2 1, 2,3, , 5 , 2
• Subset
• 1, 5 µ 1, 2,3, , 5 , 2
• µ 1, 2,3, , 5 , 2
• 2,3 1, 2,3, , 5 , 2

10
More Set Operations

• Cartesian Product (aka Cross Product)If A and B
  are sets, then
• A B (a,b) a 2 A and b 2 B
• Note that the operator "preserves structure" by
  wrapping parentheses and commas around its
  arguments
• 1,3 c, d, f ?
• If A n and B m, then A B ?

11
More Set Operations

• Generalizing to more sets

12
More Set Operations

• Power setP(A) 2A x x µ A
• Important equivalence
• x µ A means the same as x 2 P(A)
• Examples
• P(1,2) , 1, 2, 1,2
• P() ?
• P(N) ?
• If An then P(A) ?
• (Implicit that n ? 1)

13
Propositional logic

• Variables stand for truth values
• Simple procedure for evaluating truth value of
  statement
• x y x Ç y x Æ y x Ç x
• f f f f t
• f t t f t
• t f t f t
• t t t t t

14
Propositional Logic

• x y yÇx xÇy (xÇy)Æ(yÇx)f f t t tf t f t
  ft f t f ft t t t t

15
Propositional Logic

• x y y!x x!y xyf f t t tf t f t ft f t f
  ft t t t t
• The statement x!y is true unless x is true and y
  is false. In particular, it's true even when x
  is false and y is false.
• The statement x!y is a claim that "x being true
  forces y to be true". That claim can itself be
  either true or false. The claim does not say
  what happens when x is false.

16
Propositional Logic

• x!y
• read as "x implies y" or "if x, then y"
• xy
• means "x and y have the same truth values" they
  are always in agreement.
• read as "x if and only if y" or "x iff y" or "x
  is equivalent to y"
• Examples all statements below are true
• 5 7 12 ! 52 25
• username unknown ! login denied
• password incorrect ! login denied
• login denied 9 password incorrect
• 2 2 5 ! 52 25
• 2 2 5 52 10

17
Propositional Logic

• These implications capture coincidence, not
  necessarily causality, but not necessarily mere
  coincidence either
• We'll use double lined arrows ) to emphasize the
  causality part of the relationship
• Usually when our statements concern variables
• Example x gt 5 ) x2 gt 25 () instead of !)
• Speaking of which...

18
Predicate Logic (With Quantifiers)

• A predicate takes some inputs and is either true
  or false once the inputs are specified
• P(x,y) x Æ y
• Q(x) x2 lt 27
• (the types of the inputs should be explicitly or
  implicitly specified)

19
Predicate Logic (With Quantifiers)

• "For all" universal quantifier 8
• 8x P(x) means that for every possible x, P(x) is
  true
• Once P's behavior is known and the universe of
  possible values of x is known, the statement 8x
  P(x) is either true or false, so it is a
  predicate
• Example 8x x2 lt 27 is false when x ranges over
  the elements of N
• Counterexample 62 27
• It is true when x ranges over 1,2,3,4,5
• Can prove by checking each x

20
Predicate Logic (With Quantifiers)

• "There exists" existential quantifier 9
• 9x P(x) means that P(x) is true for one or more
  possible values of x
• Once P's behavior is known and the universe of
  possible values of x is known, the statement 9x
  P(x) is either true or false, so it is a
  predicate
• Example 9x x2 (x-1)2 gt 27 is true when x
  ranges over the elements of N

21
Combining Quantifiers

• (8 x 2 N) (9 y 2 N) y 3x ?
• (8 x 2 N) (9 y 2 N) 3y x ?
• (9 e 2 R) (8 x 2 R) x e x ?
• (8 x 2 R) (9 i 2 R) x i 1 ?
• (9 x 2 Q) x x 2 ?
• (8 x 2 Q) x x ? 2 ?
• You can't prove a 8 statement over an infinite
  set by enumerating cases you have to use a
  different argument

22
Relations

• A relation is a predicate that takes two (or
  more) inputs
• Examples
• "lt" between two elements of N
• r2 x2 y2 . The relation is "the points x, y
  lie on a circle of radius r centered at the
  origin", on three elements of R
• Relations need not be specified by a formula, and
  they need not be infinite
• (1,2), (2,1), (5,4)
• x r y , the program x always takes longer to run
  than the program y
• The numbers p, q 2 R are related if p/q is a
  power of 2
• If some relation doesn't have a standard syntax
  (like the last example), we invent a benign name
  for it like and use infix notation
• 3 6 but (6 9) under that definition of

23
Statements about Relations

• A relation is reflexive if this statement is
  true
• (8 x) x x
• A relation is symmetric if
• (8 x,y) xy ) yx
• A relation is antisymmetric if
• (8 x,y) (xy Æ yx) ) xy
• A relation is transitive if
• (8 x,y,z) (xy Æ yz) ) xz
• A relation is an equivalence relation if is
  reflexive, symmetric, and transitive

24
Examples

• over N
• lt over N
• a b meaning a2b2 over Z
• a b meaning a - b lt 3 over N
• over R
• r over the set of all C programs

25
Relevance

• We will work with some relations having to do
  with how computation happens
• We will often work to discover the truth or
  falsity of statements that use 8 and/or 9
  quantifiers
• Example if A and B are sets, then these three
  statements each say exactly the same thing
• AB
• (A µ B) Æ (B µ A)
• (8 x) x 2 A ! x 2 B Æ (8 x) x 2 B ! x 2 A

26
Functions

• f A ! B is a statement saying f is a function
  that maps A to B
• inputs are in A, outputs are in BIf x 2 A, then
  f(x) is the associated element of B
• 8 x 2 A 9 y 2 B f(x)yevery input produces
  some output
• The function consists of both the type statement
  f A ! B and the actual associations
• ! does not mean implies in this notation

27
Functions

• g N ! R via g(x) x1/3
• h N ! true, false via
• f 1,2 ! R viaf(1) ?f(2) -37
• Note that functions dont have to specified or
  even specifiable by formula

28
One-to-one and Onto Functions

• f A ! B is one-to-one if8 x,y 2 A x ? y !
  f(x) ? f(y)
• f A ! B is onto if8 y 2 B 9 x 2 A f(x)
  y
• f, g, h on previous page?

29
Characters, Strings, Languages

• An alphabet is a finite set, usually called ?
• Example ? a, b, c
• Example ? 0, 1
• A string is a finite ordered sequence of zero or
  more characters from an alphabet
• Example abcabab
• Empty string ? (epsilon)
• The unique string with length 0
• Think of this as ""
• Some books use the symbol ? instead of ?
• Note that is not a string at all

30
Operations on Strings

• Concatenation
• 0101 11 010111
• Sometimes written without ' '
• Particularly with variables
• Example if x and y are strings thenxy x y
• ? is the identity for this operation
• For every string x,x ? ? x x
• Thus 11???1?1
• Note that concatenation does not mark the place
  where the two strings are joined
• 0 11 01 1 011 ?

31
Operations on Strings

• Exponentiation. Inductive definition
• Basis For every string x,
• x0 ? (not 1)
• Induction step if x is a string and n 0 is a
  whole number, thenxn1 x xn
• Exponents may only be whole numbers x1.5 is
  undefined
• (001)3 001001001
• Parentheses for grouping only
• ?5 ?

32
Operations on Strings

• Reversal. Inductive definition
• Basis ?R ?
• Induction step if x is a string and c is a
  character, then(xc)R ?
• (011011)R 110110

33
Languages

• A language is a set of strings.Suppose ?a,b.
  Examples
• A (the empty language)
• B abba, babb, ?, aaaaaaaaaaaaaaaaaaaa
• C x x contains an even number of as
• ?
• Note ? ? !!
• Convention we typically use lower-case letters
  (x,y,z) for string variables and upper-case
  letters (A,B,C) for language variables

34
Operations on Languages

• Concatenation
• A B x y x 2 A Æ y 2 B
• Similar to Cartesian product, but not same
• A 0, 001 and B 01, ?
• A B (0,01), (0,?), (001,01), (001,?)
• A B ?
• ( A B is not necessarily AB )
• A ? ( \cdot \emptyset ???)

35
Operations on Languages

• Reversal
• AR xR x 2 A
• Exponentiation. Inductive definition
• Basis if A is a language, then A0 ?
• Induction step If A is a language and n 0 is
  a whole number, thenAn1 A An
• A A3 ?
• B aab, bb B2 ?
• C x x contains an even number of asC3 ?
  

36
Operations on Languages

• (Kleene Star)
• If A is a language, then
• Very important operation
• Think of A as set of all concatenations of zero
  of more things from A
• B aab, bb B ?
• Note that is an operator on languages, not
  strings (for now)
• But that exponentiation applies to both

37
Important Idiom ?

• Every alphabet ? is a finite set of 1-character
  strings, so ? is automatically a small language
• So ? means set of all concatenations of zero of
  more things from ?
• In other words set of all strings formed from
  the alphabet ?
• ?a ??, a, aa, aaa, aaaa, ...
• ?0,1 ??, 0, 1, 00, 01, 10, 11, 000, 001,
  ...
• Lexicographical order shortest strings first,
  then sorted by dictionary order

38
Important Idioms

• Equivalent statements
• Let x be a string
• Let x 2 ?
• Equivalent statements
• Let B be a language
• Let B µ ?
• Alternative Let B ... ?
• P(?) is the set of all languages over ?

39
Orientation

• Strings will be the inputs, outputs, and source
  codes of our programs
• Languages will be the birds-eye views of the
  overall behavior of particular programs
• Each language will include particular strings of
  interest and exclude others
• The language might be a specification of what
  some program is desired to do or it might be a
  description of what a program actually does
• For human communications
• Strings sentences or utterances
• Language set of those strings that make sense
• Program a person who speaks the language

40
More Language Examples

• Let L1x2 0,1 x is a multiple of 3
• Let ? be the ASCII alphabet and
• Let L2 p 2 ? gcc does not report syntax
  errors when compiling p
• Let L3 p 2 ? p is a syntactically correct
  C program
• We might hope that L2 L3
• Let L4 p 2 L2 p eventually prints Hello
  when you run it
• L4 is uncomputable (well see why after 12 weeks
  or so!)

41
Orientation

• Is C more powerful than Java?
• This is a question about computational models
• To formalize, we compare the set of all behaviors
  of all possible C programs to the set of all
  behaviors of all possible Java programs
• In other words we compare sets of languages
• Are they the same? Subsets? Disjoint?

42
Language Classes

• A language class C over an alphabet ? is a set of
  languages over ?
• The class of all languages over ? is P(?)
• So C µ P(?)
• FIN A µ ? A 0 Ç A 2 N
• The class of all finite languages
• ALL P(?) A A µ ?
• The class of all languages
• Human communication version the class of
  Indo-European languages, or the class of Romance
  languages

43
Picture so far
ALL
Each point is a language in this Venn diagram
FIN
44
Warning

• Students often confuse strings, languages, and
  classes of languages
• Every time you encounter an object you need to
  (correctly) know which type it is supposed to be
• If you are working on the wrong plane, nothing
  will make sense at all
• Remember
• string what a program is computing with at one
  moment strings are always finite
• language a characterization of the programs
  overall behavior languages are often infinite
• class a characterization of computational power
  what these type of programs are able to do
  classes are usually infinite