Chapter 2. Basic structures: Sets, Functions, Sequences and Sums

1
Chapter 2. Basic structures Sets, Functions,
Sequences and Sums

• Cheng-Chia Chen

2
2.1 Sets

• Basic structure upon which all other (discrete
  and continuous ) structures are built.
• a set is a collection of objects.
• an object is anything of interest, maybe itself a
  set.
• Definition 1.
• A set is a collection of objects.
• The objects in a set are called the elements or
  members of the set.
• If x is a member of a set S, we say S contains
  x.
• notation x ÎS vs x Ï S
• Ex In 1,2,3,4,5, the collection of 1,3 5 is a
  set.

3
Set description

• How to describe a set?
• 1. List all its member.
• the set of all positive odd integer lt 10 ?
• The set all decimal digits ?
• the set of all upper case English letters ?
• The set of all nonnegative integers ?
• 2. Set builder notation
• P(x) a property (or a statement or a
  proposition) about objects. e.g., P(x) x gt
  0 and x is odd
• then x P(x) is the set of objects satisfying
  property P.
• P(3) is true gt 3 Î x P(x)
• P(2) is false gt 2 Ï x P(x)

4
Conventions

• N def x x is a natural numbers
  0,1,2,3,...
• N def 1,2,3,...
• Z def ...,-3,-2,-1,0,1,2,3,...
• R def the set of real numbers.
• Problem
• The same set may have many different
  descriptions.
• x 0 lt x lt10 /\ x is odd
• 1,3,5,7,9, 5,3,1,9,7
• 9,7,1,3,5.

5
Set predicates

• Definition 2.
• Two sets S1, S2 are equal iff they have the same
  elements
• S1 S2 iff "x (x Î S1 lt-gt x Î S2)
• Ex 1,3,5 1,5,3 1,1,3,3, 5
• Graphical representation of sets
• Venn Diagrams
• rectangle Universal set U
• circles sets
• points particular elements
• point x inside circle S gt x Î S
• point x outside S gt x Ï S.

6
Set predicates (contd)

• Null set Æ def the collection of no
  objects.
• Def 3 empty set for-all x x Ï Æ.
• Def 3. subset
• A Í B iff all elements of A are elements of B.
• A Í B ? for-all x (x Î A ? x Î B)).
• Def 3 A Ì B def A Í B /\ A ¹ B.
• Exercise Show that
• 1. For all set A (Æ Í A)
• 2. (A Í B /\ B Í A) ? (A B)
• 3. A Í Æ ? A Æ
• Diagram representation of the set inclusion
  relationship.

7
Size or cardinality of a set

• Def. 4 A the size(cardinality) of A
  of distinct elements of A.
• Ex
• 1,3,3,5 ? ?
• the set of binary digits ?
• N ? Z ? 2i i ? N ?
• R ?
• Def. 5.
• A set A is finite iff A is a natural number
  o/w it is infinite.
• Two sets are of the same size (cardinality) iff
  there
• is a 1-1 onto mapping between them.

8
countability of sets

• Def.
• A set A is said to be denumerable iff A N.
• A set is countable iff either A n for some n
  in N or A N.
• Exercise Show that
• 1. N Z Q 4,5,6,...
• 2. R (-1, 1) (0,1)
• 3. (0,1) is uncountable
• By exercise 1,2,3,
• R is not countable.
• Q and Z is countable.

9
The power set

• Def 6.
• If A is a set, then the collection of all subsets
  of A is also a set, called the power set of A and
  is denoted as P(A) or 2A.
• Ex
• P(0,1,2) ?
• P() ?
• P(1,2,..., n) ?
• Elements in a set are not ordered. But sometimes
  we need to distinguish between (1,3,4) and
  (3,4,1) --gt ordered n-tuples

10
Cartesian Products

• Def. 7 n-tuple
• If a1,a2,...,an (n gt 0) are n objects, then
  (a1,a2,...,an) is a new object, called an
  (ordered) n-tuple with ai as its ith element.
• Any ordered 2-tuple is called a pair.
• (a1,a2,...,am) (b1,b2,...,bn) means
• (1) m n and
• (2) ai bi for all 1 i m.

11
Cartesian product

• Def. 8 Cartesian product
• A x B def (a,b) a ? A and b ? B
• A1 x A2 x ...x An def (a1,...,an) ai ? Ai
  for all 1 i n.
• Ex A 1,2, B a,b,c , C 0,1
• 1. A x B ? 2. B x A ?
• 3. A x ? 4. A x B x C ?
• Def. 8.1 Any subset of AB is called a relation
  from A to B.
• problem 1. when will A x B B x A ?
• 2. A x B ?

12
The diagonalization principle (D.P.)

• R a binary relation on A (i.e. a subset of AxA)
  .
• D (the diagonal set for R ) def x x Î A
  and (x,x) Ï R.
• For each x Î A, let Ra (the raw of a) b Î
  A (a,b) Î R.
• Then D ? Ra for all a Î A.
• Example Let A a,b,c,d,e,f and R

Ra b,d Rbb,c Rcc Rdb,c,e,f Ree,f Rf
a,c,d,e
D a,d,f. D¹Rx since for each xÎA, xÎD iff
(x,x)ÏR iff xÏRx.
13
Generalization of the diagonalization principle

• A, B two sets with a onto mapping f A ?B.
• R a relation from B to A (i.e., a subset of
  BxA).
• For all x ? A, let
• Rf(x) y y ? A ? (f(x),y) ? R ? A, and
• D x x ? A ? (f(x),x) ? R ? A.
• Then
• D ? Rf(x) for all x ? A.
• Pf Analogous to the previous one.
• Notes
• 1. C Rf(x) x ? A Ry y ? B ? 2A is
  a set of
• subsets of A.
• 2. C ? 2A (? D ? 2A but D ? C ).

14
Generalization of the diagonalization principle
fA ?B R BxA
Rfa b,d Rfbb,c Rfcc Rfdb,c,e,f Rfee,
f Rffa,c,d,e
D a,d,f. D¹Rfx since for each f(x)ÎB, xÎD
iff (f(x),x)ÏR iff xÏRf(x).
15
Show that A ? 2A

• Pf (1) The case that A is finite is trivial
  since 2A 2A gt A and there is no
  bijection b/t two finite sets with different
  sizes.
• (2) Assume A 2A, i.e., there is a bijection
  f A ?B, where B is 2A.
• Define the so-called diagonal set D in terms
  of f as follows
• Let D x ? A x Ï Rf(x)
  ,which is f(x) .
• Now
• () D is a subset of A and
• () D ? f(x) for any x ? A
• since x ? D iff x Ï f(x) for each x ? A.
  
• gt D is not a subset of A (? f is onto
  to 2A),
• a contradiction to ().
• Hence f must not exist!!

16
Application of the diagonalization principle

• Theorem The set 2N is uncountable.
• pf 1. direct from A ¹ 2A with A N.
• 2. another proof suppose 2N is denumerable.
  Then
• there is a bijection f N -gt 2N.
• let 2N S0,S1,S2,... where Si f(i).
• Now the diagonal set
• k k Ï Sk .
• By diagonalization
• principle, D ¹ Sk for any k,
• but D is a subset of N,
• by assumption
• D must equal to Sk for
• some k. a contradiction!

17
Application of the diagonalization principle
Theorem The real set (0,1) is uncountable. pf
If n is a real in (0,1), then it can be
represented as an infinite sequence n
0.n1n2n3 where each nk is one of 0,,9. Ex
½ 0.500000 1/3 0.3333 p-3
0.14159. Now suppose the set (0,1) is
denumerable. Then there must exist a bijection f
N ? (0,1). We now apply diagonization principle
to define a number d from f as follows let d
0.d0d1d2. where dk is the 9s complement of the
kth digit of the fraction part of f(k).
18

• Now it can be shown that d ¹ f(n) for all n since
  dn 9 f(n)n ¹ f(n)n, where f(n)k is the kth
  digit of the fraction part of f(n).
• This contradicts the facts that d ? (0,1) and
  f(N) (0,1).

19
The Halting Problem (Section 3.1, 6th edition
p176)

• L any of your favorite programming languages
  (C, C, Java, BASIC, etc. )
• Problem write an L-program HALT(P,X), which
  takes another L-program P(-) and string X as
  input, and
• HALT(P,X) returns true if P(X) halts and
• returns false if P(X) does not halt.

20
How hard is the halting problem ?

• Consider the program
• int f( int n )
• if ( n 1 ) return 1
• if (n 2 0 ) // n is even
• f(n 2)
• else // n is odd
• f( n x 3 1)
• It is conjectured that f(n) halts (returns 1) for
  all n 1.
• Ex f(10) ? 5 ? 16 ? 8 ? 4 ? 2 ? 1 (halts)

21
Halt(P,X) does not exist

• Ideas leading to the proof
• Problem1 What about the truth value of the
  sentence
• L L is false
• Problem 2 Let S X X? X. Then does S
  belong to S or not ?
• The analysis S ? S gt S ? S S ? S gt S ? S.
• Problem 3 ??? 1. ??????? 2. ?????????
• ?? 1. 2. ???????
• Problem 4 ???? ???????? gt ????????????
• gt ?????????????gt
  ???????? .
• ??????????
• Conclusion
• 1. S is not a set!!
• 2. If a language is too powerful, it may produce
  expressions that is meaningless or can not be
  realized.
• Question If HALT(P,X) can be programmed, will it
  incur any absurd result like the case of S?
• Ans yes!!

22
H(P) a program that HALT(-,-) cannot predicate
Notes 1. H(P) is simply L if(HALT(P,P)) then
goto L 2. H uses HALT(-,-) as a subroutine.
3. H(P) halts iff HALT(P,P) returns false iff
P(P) does not halt. 4. Let input P be H ? H(H)
halts iff H(H) does not halt. ? HALT is not a
correct implementation!
23
1.5 Set operations

• union, intersection,difference , complement,
• Definition.
• 1. AÈ B x x ? A or x ? B
• 2. AÇ B x x ? A and x ? B
• 3. If A Ç B gt call A and B disjoint.
• 4. A - B x x ? A but x ? B
• 5. A U - A
• Venn diagram representations
• Ex U 1,..,10, A 1,2,3,5,8
• B 2,4,6,8,10
• gt AÈ B , AÇ B , A - B , A ?

24
Set identities (page 124, 6th edition)

• Identity laws A È A A ? U A
• Domination law U È A A ? A
• Idempotent law A ?A A AÈA A
• complementation A A
• commutative A?B B ?A AÈBBÈA
• Associative A?(B?C) (A?B)?C
  ___?_____
• Distributive A È (B ? C) ?
  ____?_____
• De Morgan laws (AUB) _?_
• (A ? B) A U
  B
• Absorption law AÈ(A ? B) A A ? (AÈB) A
• Complement law AÈA U A?A

25
Prove set equality

• 1. Show that (A È B) A Ç B by show that
• 1. (A È B) Í A ÇB
• 2. A ÇB Í (A È B)
• pf
• (1. By definition and logic reasonng) Let x be
  any element in (A È B) . Then
• x ? (AUB) iff (x ? AUB) iff x ? A and x ?
  B iff x ?A and x ? B.
• 2. show (1) by using set builder and logical
  equivalence.
• (AUB) x x ? AUB x x ?A and x ? B
  
• x x ?A Ç x x ?A A
  Ç B.

26
Membership Table

• 3. Show distributive law A U (B ? C) (AUB) ?
  (AUC) by using membership table.
• Let x be any element. Then we need only consider
  8 cases as to whether x is a member of A or B or
  C. In all cases, we find that x ?A U (B ? C) iff
  x?(AUB)?(AUC) . Hence the equality holds.

x ?A x? B x? C x? AUB x? AUC x? B?C x? A U (B ? C) x? (AUB)?(AUC)
0 0 0 0 0 0 0 0
0 0 1 0 1 0 0 0
0 1 0 1 0 0 0 0
0 1 1 1 1 1 1 1
1 0 0 1 1 0 1 1
1 0 1 1 1 0 1 1
1 1 0 1 1 0 1 1
1 1 1 1 1 1 1 1
27
Set equality reasoning

• 4 show (AÈ (BÇ C)) (C È B) Ç A by set
  identities.
• pf
• (AÈ (BÇ C))
• A Ç (B Ç C )
• (B Ç C ) Ç A
• (B ÈC) Ç A

28
Generalized set operations

• Def. 6
• A1,A2,...An n sets
• B A1,A2,...An
• 1. A1È A2È An ÈB Èi1,..n Ai def ?
• 2. A1Ç A2Ç An ÇB Ç i1,..n Aidef ?
• quiz if B gt ÈB ? ÇB ?
• Venn diagram representation of A1È A2È A3 and A1Ç
  A2Ç A3 .
• Example
• A 0,2,4,6,8, B 0,1,2,3,4, C0,3,6,9 gt
• A1È A2È A3 ?
• A1Ç A2Ç A3 ?

29
Set representation (in computer)

• Unordered list or array
• union, intersection, time consuming.
• Bit string
• U a1,...,an is the universal set.
• A any subset of U
• A can be represented by the string
• s(A) x1 x2 x3....xn where
• xi 1 if a1 ? A and 0 o/w.
• fast set operations
• suitable only if U is not large.
• constant size representation.

30
1.6 Functions

• Def. 1 functions A, B two sets
• 1. a partial function f from A to B is a set of
  pairs (x, y) ? AxB
• s.t., for each x ? A there is at most one y
  ? B s.t. (x,y) ? f.
• 2. f is a (total) function if for for each x ?
  A there is exactly one
• y ? B with (x,y) ? f.
• 3. if (x,y) ? f, we write f(x) y.
• 4. f A ?B means f is a function from A to B.
• Def. 2. If fA ? B then
• 1. A the domain of f 2. B the codomain of
  f
• if f(a)b gt
• 3. b is the image of a 4. a is
  the preimage of b
• 5. range(f) ? 6.
  preimage(f) ?

31
Types of functions

• Def 4. f? A x B S a subset of A,
• T a subset of B
• 1. f(S) def ? y ?B ?x ? S with (x,y) ?
  f
• 2. f-1(T) def ? x ?A ?y ? T with (x,y)
  ? f
• Def. 1-1, onto, injection, surjection,
  bijection
• f A -gt B.
• f is 1-1 (an injection) iff ?
• f is onto (surjective, a surjection) iff ?
• f is 1-1 onto ltgt f is bijective (a bijection,
  1-1 correspondence)

32
Properties of functions

• If there is an onto mapping from A to B, then
  there is a 1-1 mapping from B to A.
• If there is a 1-1 mapping from A to B then
  there is an onto mapping from B to A.
• The onto ( ) and the 1-1( ) relations between
  sets are all preorders (i.e., refexive and
  transitive ) and are converse to each other.
• If there is a onto mapping from A to B and a onto
  mapping from B to A, then there is a 1-1 and onto
  mapping from A to B.
• pf (1) Let fA?B an onto. For each b ? B, let
  g(b) x ? A f(x) b ? Ø ? the function
  hB?A defined by h(b) any x ? g(b) is 1-1.
• (2,3) Similar to (1). (4) is hard (as an
  exercise?).

33
Real valued functions

• FA? B is a real valued function iff A and B are
  subsets of R.
• f1,f2 real valued functions
• gt 1. f1f2 (x) ?
• 2. f1 f2 (x) f1(x) x f2(x).
• 3. f is increasing iff ?
• 4. f is strictly increasing iff ?

34
operations on functions

• A, B, C any sets f A ?B g B ? C, then
• 1. identity function
• idA A ?A s.t. idA(x) x for all x in
  A.
• 2. composition of g and f
• gf A ? C s.t. gf(x) g(f(x)) for x in A.
• 3. inverse If f is a bijection, then
• f-1 B ? A s.t. f-1(y) x iff f(x) y
  for y in B.
• Graphical representations

35
Properties of operations on functions

• f A ? B. g B? C h C? D, then
• 1. idA f f f idB
• 2. f(gh) (fg) h --- assoc.
• 3. (fg)-1 g-1 f-1
• If f A ? A is a bijection then
• 4. f f-1 f-1 f idA .
• Sequence
• finite A-sequence a n (or 1,n) ?A
• w A-sequence a N (or N) ? A.
• B indexed A-sequence fB?A.

36
The growth of functions ( Section 3.2, 6th
edition)

• Summation rules
• S ai def a1 a2...an (...)
• a (ad) (a2d)... ?
• sol let S a (ad) (a2d)... (a
  (n-1)d).
• Then S (a (n-1)d) (a(n-2)d) (ad)a
• 2S (2a (n-1)d) . (2a nd)
  (2a nd) x n.
• gt S a (n-1)dn/2 (a1 an)n/2.
• a ar arr ...
• sol let S a ar ar2... a r (n-1).
• Then rS ar ar2 ar3 arn
• (1-r)S a - arn.
• gt S a(1-rn)/(1-r).

37
The growth of functions

• If f is a positive valued function, then what do
  the following terms mean?
• O(f) means ?
• W(f) means ?
• Q(f) means ?
• useful in the analysis of the efficiency of
  computer algorithms.

38
Problem, algorithm and Complexity

• A problem is a general question
• description of parameters input
• description of solutionoutput
• description of how the outputs are related to the
  inputs.
• An algorithm is a step by step procedure to get
  the related output (i.e., answer) from the given
  input.
• a recipe
• a computer program
• We want the most efficient algorithm
• fastest (mostly)
• most economical with memory (sometimes)
• expressed as a function of problem input size

39
Example Traveling Salesman Problem

• Parameters
• Set of cities
• Inter-city distances

40
Solution output

• Solution
• The shortest tour passing all cities
• Example a,b,d,c,a has length 27

41
Problem Size

• What is appropriate measure of problem size?
• m nodes?
• m(m1)/2 distances?
• Use an encoding of the problem
• alphabet of symbols a,b,c,d,0-9, .
• strings abcd1059693.
• Measures
• Problem Size length of encoding.
• Time Complexity how long an algorithm takes, as
  function of problem size.

42
Time Complexity

• What is tractable?
• A function f(n) is O(g(n)) whenever
• ? c gt 0 ? n0 gt 0 s.t. f(n) c g(n) for
  all n gt n0.
• A polynomial time algorithm is one whose time
  complexity is O(p(n)) for some polynomial p(n).
• An exponential time algorithm is one whose time
  complexity cannot be bounded by a polynomial
• (e.g., nlog n or 2n).

43
Tractability

• Basic distinction
• polynomial time tractable
• exponential time intractable

problem size
10 20 30 40 50 60
n .00001s .00002s .00003s .00004s .00005s .00006s
n2 .0001s .0004s .0009s .0016s .0025s .0036s
n3 .001s .008s .027s .064s .125s .216s
n5 .1s 3.2s 24.3s 1.7 m 5.2 m 13.0 m
2n .001s 1.0s 17.9s 12.7d 35.7 y 366c
3n .059 s 58 m 6.5 y 3855 c 2108c 1.310 13 c
execution time in microseconds(µs)
44
Effect of Speed-ups for different order of
functions

• Wait for faster hardware!
• Consider maximum problem size you can solve in an
  hour.

present 100 times faster 1000 times faster
n N1 100 N1 1000 N1
n2 N2 10N2 31.6 N2
n3 N3 4.64 N3 10 N3
n5 N4 2.5 N4 3.98 N4
2n N5 N56.64 N59.97
3n N6 N64.19 N62.29
45
Asymptotic notations

• Asymptotic analysis
• care the value of f(n) only when n is very large.
• discard the difference of f and g when limit f/g
  is bounded by two positive numbers (a,b).
• O((g(n)) f(n) ? c and k gt 0 s.t. f(n) ?
  cg(n) for all n gt k
• Q((g(n)) f(n) ?c2,c1 and k gt 0 s.t.
  c2g(n) ? f(n) ? c1 g(n) for all n gt k
• W((g(n)) f(n) ?c and k gt 0 s.t. c g(n) ?
  f(n) for all n gt k

46
Examples

• Ex1 Show that f(x) x2 2x 1 is O(x2).
• pf Let c 4, k 1, It is easy to check that if
  x gt k, then f(x) x2 2x 1 x2 2 x2 x2
  c x2.
• Hence by definition, f(x) O(x2).
• Ex2 Show that f(x) x2 2x 1 is W(x2).
• pf Let c 1, k 1. Obviously, x22x1 x2
  cx2 for all x gt1k. Hence f(x) W(x2).

47

• Ex2 Show that f(x) x2 2x 1 is Q(x2).
• pf Let c1 1,c24, k 1.
• c1x2 x22x1 c2 x2 for all x gt1k. Hence
  f(x) Q(x2).
• Proposition f Q(g) iff f O(g) and f W(g).
• pf (gt) direct from definition.
• (lt) if f O(g) Q(g)
• gt ? c1, k1 s.t f(x) c1 g(x) for all x gt k1.
• ? c2, k2 s.t f(x) c2 g(x) for all x gt
  k2.
• let k max(k1,k2) gt
• c2 g(x) f(x) c1 g(x) for all x gt k.
• Hence f(x) Q(g).

48
Using the asymptotic notations

• We use f(n) D(g(n)) to mean f(n) ?D(g(n)),
  where D Q,O,W.
• analogy
• O (asymptotic upper bound) ?
• f(n) O(g(n)) behaves like f(n) ? g(n)
• W (asymptotic lower bound) ?
• f(n) W(g(n)) behaves like f(n) ? g(n)
• Q (asymptotic tight bound
• f(n) Q (g(n)) behaves like f(n) g(n)

49
Exercises

• Let f and g be positive real function. Show that
• f O(g) iff g W(f)
• 2. f Q(g) iff limit n?? g/f is bounded above 0