Son of Formal Foundations: Numbers and Stuff

1
Son of Formal FoundationsNumbers and Stuff

• Helpful Reading
• Rosen 4.1-4.5 (has some extra set notation we
  have not covered yet)

2
Discrete Mathematics

• Discrete mathematics involves principles for
  discretized (noncontinuous) spaces
• Important such space integers
• We will study some more important number theory
  principles
• Counting
• Determine the size of some finite discrete space
• Probability
• Counting gives sizes of event sample space

3
Counting Theory

• Important for combinatorics (the study of
  arrangements of objects)
• Important for probability
• Important for complexity of algorithms
• Leads to two more proof techniques
• Combinatorial proof Show two expressions are
  equivalent as two different methods for counting
  the number of elements in a group
• Pigeonhole principle Lower bound on the minimum
  number of elements in a group

4
Rules of Counting Theory

• Sum rule Counting the number of ways to solve
  two independent problems
• Product rule Counting the number of ways to
  solve two problems in sequence
• Inclusion-exclusion Counting the number of ways
  to solve two problems whose solutions may overlap
• Many things can be counted this way

5
The Sum Rule

• If there are n1 ways to do Task 1 and n2 ways to
  do Task 2, and the problems cannot be solved
  simultaneously (i.e. no solution to Task 1 also
  solves Task 2), then total no. of ways to do any
  one of the tasks is n1 n2
• How many ways are there to pick a single project
  from two possible sets of projects of sizes 7 and
  11? (7 11 18)
• How many possible choices are there for a meal if
  there are four menu sections, each with 15
  choices? (15 15 15 15 60)
• Extends to more than two independent tasks

6
The Product Rule

• If a task can be broken down into two independent
  subtasks, with n1 and n2 ways, respectively, to
  complete the subtasks, then the total number of
  ways to do the original task is n1 n2
• Count even integers btw 0 and 99 (inclusive)
• Task 1 Choose ones place, 5 choices (0, 2, 4, 6,
  8)
• Task 2 Choose tens place, 10 choices
• Total of 5 10 50 even integers btw 0 and 99
• Count the number of 7-digit telephone numbers
• Seven tasks (each task is to choose a different
  digit), ten possible choices per task, 107 total
  ways

7
Inclusion-Exclusion Principle

• If there are ways to solve two tasks
  simultaneously, the sum rule gives us an
  overcount of the number of ways to solve one task
  OR the other
• Must use the Inclusion-Exclusion Principle, which
  states that the number of ways to solve either
  Task 1 or Task 2 is equal to (ways to solve Task
  1 ways to solve Task 2 - ways to solve both
  simultaneously)

8
Quintessential Example

• Example How many bit strings (I.e. strings of 0s
  and 1s) of length 5 either start with a 0 or end
  with a 1?
• Task 1 Number of bit strings that start with 0
• Really just 24 ways of picking since the 0 is
  fixed
• Task 2 Number of bit strings that end with 1
• Again, 24 ways since the 1 is fixed
• Total by sum rule 2 24 25 32
• If you write out all the answers, you should get
  24 (try it) -- Why?

9
Example, continued

• Inclusion-Exclusion forces you to account for the
  set of bit strings that both start with 0 and end
  with 1 -- you overcounted those bit strings with
  the sum rule
• How many such extras are there?
• 2 2 2 8 since the 0 and 1 are fixed
• 32 - 8 24, the answer we should have gotten in
  the first place

10
Pigeonhole Principle

• If there are k 1 or more objects placed in k
  boxes, at least one box must have more than one
  object in it
• In general, if there are N items placed in k
  boxes, at least one box has éN / kù items
• How many students must be in a room to INSURE
  that 3 of them have the same birth month? (Answer
  25)
• IN CLASS We will prove these via proof by
  contradiction

11
Combinatorics

• Counting the number of arrangements of objects in
  a group
• There are two major combinatoric units
• A combination of items is a grouping of the items
  in which the order of the items is ignored
• A permutation of items is a grouping of the items
  in which the item order is important

12
P(n, r) and C(n, r)

• P(n, r) is the number of permutations of r
  elements from an n-element set
• P(n, r) n(n-1)(n-2)(n-3)(n-r2)(n-r1) by
  direct application of the product rule
• Also happens that P(n,r) n! / (n-r)!
• C(n, r) is the number of combinations of r
  elements from an n-element set
• P(n, r) C(n, r) number of orderings of the
  elements
• Number of orders for r elements r! (product
  rule)
• Thus, C(n, r) P(n, r) / r! n! / (r! (n-r)! )

13
Combinatorial Proof

• Prove two expressions are equal by showing they
  are different ways to count the number of items
  in the same group
• Prove C(n1, k) C(n,k-1) C(n,k) Pascals
  identity
• Suppose that there is an aquarium with n1 fish
• Exactly C(n1, k) groups of k fish in the
  aquarium
• Now, suppose we remove one of the fish
• C(n, k-1) groups of k-1 fish to which we could
  add the exiled fish, and C(n, k) existing groups
  of k fish
• The total number of these two groups of k fish
  must be same as the number of groups over all n1
  fish
• Thus, C(n1, k) C(n, k-1) C(n, k)

14
Probability

• The probability of an event E, which is a subset
  of a finite sample space S of equally likely
  outcomes, is p(E) E / S
• Let X be some discrete random variable whose
  values are limited to the space S
• Then the expected value of X is equal to

15
More Probability

• p(not E) 1 - p(E)
• p(E or F) p(E) p(F) if E and F are
  independent (by the sum rule)
• p(E and F) p(E)p(F) if E and F are independent
  (by the product rule)
• p(E F) p(E and F)/p(F) conditional prob.
• Read as probability of E given F
• p(E F) p(E) if E and F are independent
• p(E F) ( of outcomes in E that are also in
  F) divided by ( of outcomes in F)

16
Preview Using Probability Ideas in Algorithm
Analysis

• The average runtime of some algorithm is the
  expected value of its runtime for a problem of
  size n
• Let X be the number of operations executed by the
  algorithm over some input of size n, the average
  runtime, then just find the expected value of X
• Useful for deterministic as well as randomized
  algorithms

17
Brief Example Analyzing Linear Search Algorithms

• Find runtime (in number of operations executed by
  the algorithm) of a linear search in which the
  probability of the desired item being in the list
  is q, and each item is equally likely to be the
  desired item
• Assume it takes 2i 1 operations to get to the
  ith item in the list, and 2n 2 operations to
  show that an item is not in the list
• Each of these operations is assumed to take the
  same amount of time

18
Answer to Search Example

• For items in the list, possible x are
• 2(1) 1 3, 2(3) 1 5, , 2(n) 1
• For each of these x, p(X x) q/n equal
  likelihood
• p(X 2n 2) (1-q)
• Thus, expected runtime
• (3 5 7 2n 1)(q/n) (2n 2)(1-q)
• By the magic of algebra, this reduces to
• q(n 2) (2n 2)(1-q) (2-q)n 2
• q 0 - expect of operations (2n 2)
  reasonable
• q 1 - expect of operations (n 2)