Title: Chapter 7: Advanced Counting Techniques
1Chapter 7Advanced Counting Techniques
27.1 Recurrence Relations
- A recurrence relation (R.R., or just recurrence)
for a sequence an is an equation that expresses
an in terms of one or more previous elements a0,
, an-1 of the sequence, for all nn0. - A recursive definition, without the base cases.
- A particular sequence (described non-recursively)
is said to solve the given recurrence relation if
it is consistent with the definition of the
recurrence. - A given recurrence relation may have many
solutions.
7.1 Recurrence Relations
3Recurrence Relation Example
- Consider the recurrence relation
- an 2an-1 - an-2 (n2).
- Which of the following are solutions? an
3n an 2n - an 5
7.1 Recurrence Relations
4Example Applications
- Recurrence relation for growth of a bank account
with P interest per given period - Mn Mn-1 (P/100)Mn-1
- Growth of a population in which each organism
yields 1 new one every period starting 2 periods
after its birth. - Pn Pn-1 Pn-2 (Fibonacci relation)
7.1 Recurrence Relations
5Solving Compound Interest RR
- Mn Mn-1 (P/100)Mn-1
- (1 P/100) Mn-1
- r Mn-1 (let r 1 P/100)
-
7.1 Recurrence Relations
6Tower of Hanoi Example
- Problem Get all disks from peg 1 to peg 2.
- Only move 1 disk at a time.
- Never set a larger disk on a smaller one.
Peg 1
Peg 2
Peg 3
7.1 Recurrence Relations
7Hanoi Recurrence Relation
- Let Hn moves for a stack of n disks.
- Optimal strategy
- Move top n-1 disks to spare peg. (Hn-1 moves)
- Move bottom disk. (1 move)
- Move top n-1 to bottom disk. (Hn-1 moves)
- Note Hn 2Hn-1 1
7.1 Recurrence Relations
8Solving Tower of Hanoi RR
7.1 Recurrence Relations
9Finding Recurrence Relation
- Ex Find a recurrence relation and give initial
conditions for the number of bit strings of
length n that do not have two consecutive 0s. How
many such bit strings are there of length 5?
7.1 Recurrence Relations
10Codeword Enumeration
- ExConsider a string of decimal digits a valid
codeword if it contains an even number of 0
digits. For example, 1230407869 is valid, whereas
120987045608 is not valid. Let an be the number
of valid n-digit codewords. Find a recurrence
relation for an .
7.1 Recurrence Relations
11Catalan Numbers
- Ex Find a recurrence relation for Cn , the
number of ways to parenthesize the product of n1
numbers, x0, x1,, xn, to specify the order of
multiplication. For example, C3 5.
7.1 Recurrence Relations
127.2 Solving Recurrences
General Solution Schemas
- A linear homogeneous recurrence of degree k with
constant coefficients (k-LiHoReCoCo) is a
recurrence of the form an c1an-1
ckan-k,where the ci are all real, and ck ? 0. - The solution is uniquely determined if k initial
conditions a0ak-1 are provided.
7.2 Solving Recurrences
13Solving LiHoReCoCos
- Basic idea Look for solutions of the form an
rn, where r is a constant. - This requires the characteristic equation rn
c1rn-1 ckrn-k, i.e., rk - c1rk-1 - - ck
0 - The solutions (characteristic roots) can yield an
explicit formula for the sequence.
7.2 Solving Recurrences
14Solving 2-LiHoReCoCos
- Consider an arbitrary 2-LiHoReCoCo an c1an-1
c2an-2 - It has the characteristic equation (C.E.) r2 -
c1r - c2 0 - Thm. 1 If this CE has 2 roots r1?r2, then an
a1r1n a2r2n for n0for some constants a1, a2.
7.2 Solving Recurrences
15Example
- Solve the recurrence an an-1 2an-2 given the
initial conditions a0 2, a1 7. - Solution
7.2 Solving Recurrences
16Example Continued
- To find a1 and a2, solve the equations for the
initial conditions a0 and a1 -
- Simplifying, we have the pair of equations
- which we can solve easily by substitution
-
- Final answer
Check an0 2, 7, 11, 25, 47, 97
7.2 Solving Recurrences
17Example
- Find an explicit formula for the Fibonacci
numbers.
7.2 Solving Recurrences
18The Case of Degenerate Roots
- Now, what if the C.E. r2 - c1r - c2 0 has only
1 root r0? - Theorem 2 Then, an a1r0n a2nr0n, for all
n0,for some constants a1, a2. - Ex
7.2 Solving Recurrences
19k-LiHoReCoCos
- Consider a k-LiHoReCoCo
- Its C.E. is
- Thm.3 If this has k distinct roots ri, then the
solutions to the recurrence are of the form - for all n0, where the ai are constants.
7.2 Solving Recurrences
20Example
7.2 Solving Recurrences
21Degenerate k-LiHoReCoCos
- Suppose there are t roots r1,,rt with
multiplicities m1,,mt. Then - for all n0, where all the a are constants.
7.2 Solving Recurrences
22Example
7.2 Solving Recurrences
23LiNoReCoCos
- Linear nonhomogeneous RRs with constant
coefficients may (unlike LiHoReCoCos) contain
some terms F(n) that depend only on n (and not on
any ais). General form - an c1an-1 ckan-k F(n)
The associated homogeneous recurrence
relation(associated LiHoReCoCo).
7.2 Solving Recurrences
24Solutions of LiNoReCoCos
- A useful theorem about LiNoReCoCos
- If an p(n) is any particular solution to the
LiNoReCoCo - Then all its solutions are of the form an
p(n) h(n),where an h(n) is any solution to
the associated homogeneous RR
7.2 Solving Recurrences
25Example
- Find all solutions to an 3an-12n. Which
solution has a1 3? - Notice this is a 1-LiNoReCoCo. Its associated
1-LiHoReCoCo is an 3an-1, whose solutions are
all of the form an a3n. Thus the solutions to
the original problem are all of the form an
p(n) a3n. So, all we need to do is find one
p(n) that works.
7.2 Solving Recurrences
26Trial Solutions
- If the extra terms F(n) are a degree-t polynomial
in n, you should try a degree-t polynomial as the
particular solution p(n). - This case F(n) is linear so try an cn d.
- (for all n)
(collect
terms) So - So is a solution.
- Check an1 -5/2, -7/2, -9/2,
7.2 Solving Recurrences
27Finding a Desired Solution
- From the previous, we know that all general
solutions to our example are of the form -
- Solve this for a for the given case, a1 3
-
-
- The answer is
7.2 Solving Recurrences
28Example
7.2 Solving Recurrences
297.3 Divide Conquer R.R.s
- Main points so far
- Many types of problems are solvable by reducing a
problem of size n into some number a of
independent subproblems, each of size ??n/b?,
where a?1 and bgt1. - The time complexity to solve such problems is
given by a recurrence relation - T(n) aT(?n/b?) g(n)
7.3 D-C Recurrence Relations
30DivideConquer Examples
- Binary search Break list into 1 sub-problem
(smaller list) (so a1) of size ??n/2? (so b2). - So T(n) T(?n/2?)c (g(n)c constant)
- Merge sort Break list of length n into 2
sublists (a2), each of size ??n/2? (so b2),
then merge them, in g(n) T(n) time. - So T(n) 2T(?n/2?) cn (roughly, for some c)
7.3 D-C Recurrence Relations
31DivideConquer Examples
- Finding the Maximum and Minimum Break list into
2 sub-problem (smaller list) (so a2) of size
??n/2? (so b2). - So T(n) 2T(?n/2?)2 (g(n)2 constant)
7.3 D-C Recurrence Relations
32Fast Multiplication Example
- The ordinary grade-school algorithm takes T(n2)
steps to multiply two n-digit numbers. - This seems like too much work!
- So, lets find an asymptotically faster
multiplication algorithm! - To find the product cd of two 2n-digit base-b
numbers, c(c2n-1c2n-2c0)b and
d(d2n-1d2n-2d0)b, - First, we break c and d in half
- cbnC1C0, dbnD1D0, and then... (see
next slide)
7.3 D-C Recurrence Relations
33Derivation of Fast Multiplication
(Multiply out polynomials)
(Factor last polynomial)
7.3 D-C Recurrence Relations
34Recurrence Rel. for Fast Mult.
- Notice that the time complexity T(n) of the fast
multiplication algorithm obeys the recurrence - T(2n)3T(n)?(n) i.e.,
- T(n)3T(n/2)?(n)
- So a3, b2.
Time to do the needed adds subtracts of
n-digit and 2n-digit numbers
7.3 D-C Recurrence Relations
35The Master Theorem
- Consider a function f(n) that, for all nbk for
all k?Z,,satisfies the recurrence relation - f(n) a f (n/b) cnd
- with a1, integer bgt1, real cgt0, d0. Then
7.3 D-C Recurrence Relations
36Examples
- Consider a function f(n) that, for all n2k for
all k?Z,,satisfies the recurrence relation - f(n) 5f(n/2) 3. Then
- Complexity of Merge Sort
7.3 D-C Recurrence Relations
37Example
- Recall that complexity of fast multiply was
- T(n)3T(n/2)?(n)
- Thus, a3, b2, d1. So a gt bd, so case 3 of the
master theorem applies, so -
- which is ?(n1.58), so the new algorithm is
strictly faster than ordinary T(n2) multiply!
7.3 D-C Recurrence Relations
38Example
- The Closest-Pair Problema set of n points,
-
- How can this closest pair of points be found in
an efficient way? - T(n)2T(n/2)7n
7.3 D-C Recurrence Relations
397.4 Generating Functions
- Definitiongenerating function for the sequence
of real numbers is the infinite
series
7.4 Generating Functions
40Examples
- What is the generating function of the sequence
1,1,1,1,1,1? - What is the generating function of the sequence
?
7.4 Generating Functions
41Examples
- The function f(x)1/(1?x) is the generating
function of the sequence 1,1,1,for xlt1. - The function f(x)1/(1?ax) is the generating
function of the sequence 1,a,a2,for axlt1.
7.4 Generating Functions
42Theorem
Convolution of ak and bk
7.4 Generating Functions
43Example
- What sequence has the generating function
- f(x)1/(1?x)2 ?
7.4 Generating Functions
44Extended Binomial Coefficient
7.4 Generating Functions
45Extended Binomial Theorem
-
- Can be proved using Maclaurin series.
- Examples
7.4 Generating Functions
46Example
- Find the number of solutions of
- Sol Find the coefficient of x17 ,
- The answer is
7.4 Generating Functions
47Example
- Solve the recurrence relation
- Sol Let G(x) be the generating function of
ak ,
7.4 Generating Functions
48Example(Contd)
7.4 Generating Functions