Recurrence Equations

1
Recurrence Equations

• Algorithm Design Analysis
• 4

2
In the last class

• Recursive Procedures
• Analyzing the Recursive Computation.
• Induction over Recursive Procedures
• Proving Correctness of Procedures

3
Recurrence Equations

• Recursive algorithm and recurrence equation
• Solution of the Recurrence equations
• Guess and proving
• Recursion tree
• Master theorem
• Divide-and-conquer

4
Recurrence Equation Concept

• A recurrence equation
• defines a function over the natural number n
• in term of its own value at one or more integers
  smaller than n
• Example Fibonacci numbers
• FnFn-1Fn-2 for n?2
• F00, F11
• Recurrence equation is used to express the cost
  of recursive procedures.

5
Linear Homogeneous Relation
is called linear homogeneous relation of degree k.
Yes
No
6
Characteristic Equation

• For a linear homogeneous recurrence relation of
  degree k
• the polynomial of degree k
• is called its characteristic equation.
• The characteristic equation of linear homogeneous
  recurrence relation of degree 2 is

7
Solution of Recurrence Relation

• If the characteristic equation
  of the recurrence relation
  has two distinct roots s1 and s2,
  then
• where u and v depend on the initial
  conditions, is the explicit formula for the
  sequence.
• If the equation has a single root s, then, both
  s1 and s2 in the formula above are replaced by s

8
Fibonacci Sequence
f11 f21 fn fn-1 fn-2
1, 1, 2, 3, 5, 8, 13, 21, 34, ......
Explicit formula for Fibonacci Sequence The
characteristic equation is x2-x-10, which has
roots
Note (by initial conditions)
which results
9
Determining the Upper Bound

• Example T(n)2T(?n/2?) n
• Guess
• T(n)?O(n)?
• T(n)?cn, to be proved for c large enough
• T(n)?O(n2)?
• T(n)?cn2, to be proved for c large enough
• Or maybe, T(n)?O(nlogn)?
• T(n)?cnlogn, to be proved for c large enough

Try and fail to prove T(n)?cn
T(n)2T(?n/2?)n ? 2c(?n/2?)n ?
2c(n/2)n (c1)n
T(n) 2T(?n/2?)n ? 2(c?n/2? lg
(?n/2?))n ? cn lg (n/2)n cn
lg n cn log 2 n cn lg n cn n
? cn log n for c?1
Note the proof is invalid for T(1)1
10
Recursion Tree

T(size)
nonrecursive cost
T(n)
n
The recursion tree for T(n)T(n/2)T(n/2)n
11
Recursion Tree Rules

• Construction of a recursion tree
• work copy use auxiliary variable
• root node
• expansion of a node
• recursive parts children
• nonrecursive parts nonrecursive cost
• the node with base-case size

12
Recursion tree equation

• For any subtree of the recursion tree,
• size field of root
• Snonrecursive costs of expanded nodes
• Ssize fields of incomplete nodes
• Example divide-and-conquer
• T(n) bT(n/c) f(n)
• After kth expansion

13
Evaluation of a Recursion Tree

• Computing the sum of the nonrecursive costs of
  all nodes.
• Level by level through the tree down.
• Knowledge of the maximum depth of the recursion
  tree, that is the depth at which the size
  parameter reduce to a base case.

14
Recursion Tree
Work copy T(k)T(k/2)T(k/2)k

T(n)nlgn
T(n)
n
n/2d
(size ?1)
At this level T(n)n2(n/2)4T(n/4)2n4T(n/4)
15
Recursion Tree for
T(n)3T(?n/4?)?(n2)
cn2
cn2
c(n/4)2
c(n/4)2
c(n/4)2
log4n
c(n/16)2
c(n/16)2
c(n/16)2
c(n/16)2
c(n/16)2
c(n/16)2
c(n/16)2
c(n/16)2
c(n/16)2



T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
Total O(n2)
Note
16
Verifying Guess by Recursive Tree
Inductive hypothesis
17
Common Recurrence Equation

• Divide and Conquer
• T(n) bT(n/c) f(n)
• Chip and Conquer
• T(n) T(n - c) f(n)
• Chip and Be Conquered
• T(n) bT(n - c) f(n)

18
Recursion Tree for
T(n)bT(n/c)f(n)
f(n)
f(n)
b
f(n/c)
f(n/c)
f(n/c)
b
logcn
f(n/c2)
f(n/c2)
f(n/c2)
f(n/c2)
f(n/c2)
f(n/c2)
f(n/c2)
f(n/c2)
f(n/c2)



T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
T(1)
Note
Total ?
19
Solving the Divide-and-Conquer

• The recursion equation for divide-and-conquer,
  the general caseT(n)bT(n/c)f(n)
• Observations
• Let base-cases occur at depth D(leaf), then
  n/cD1, that is Dlg(n)/lg(c)
• Let the number of leaves of the tree be L, then
  LbD, that is Lb(lg(n)/lg(c)).
• By a little algebra LnE, where Elg(b)/lg(c),
  called critical exponent.

20
Divide-and-Conquer the Solution

• The recursion tree has depth Dlg(n)/ lg(c), so
  there are about that many row-sums.
• The 0th row-sum is f(n), the nonrecursive cost of
  the root.
• The Dth row-sum is nE, assuming base cases cost
  1, or ?(nE) in any event.
• The solution of divide-and-conquer equation is
  the nonrecursive costs of all nodes in the tree,
  which is the sum of the row-sums.

21
Little Master Theorem

• Complexity of the divide-and-conquer
• case 1 row-sums forming a geometric series
• T(n)??(nE), where E is critical exponent
• case 2 row-sums remaining about constant
• T(n)??(f(n)log(n))
• case 3 row-sums forming a decreasing geometric
  series
• T(n)??(f(n))

22
Master Theorem
The positive ? is critical, resulting gaps
between cases as well

• Loosening the restrictions on f(n)
• Case 1 f(n)?O(nE-?), (?gt0), then
• T(n)??(nE)
• Case 2 f(n)??(nE), as all node depth contribute
  about equally
• T(n)??(f(n)log(n))
• case 3 f(n)??(nE?), (?gt0), and f(n)?O(nE?),
  (???), then
• T(n)??(f(n))

23
Using Master Theorem
24
Looking at the Gap

• T(n)2T(n/2)nlgn
• a2, b2, E1, f(n)nlgn
• We have f(n)?(nE), but no ?gt0 satisfies
  f(n)?(nE?), since lgn grows slower that n? for
  any small positive ?.
• So, case 3 doesnt apply.
• However, neither case 2 applies.

25
Proof of the Master Theorem
Case 3 as an example
(Note in asymptotic analysis, f(n)??(nE?) leads
to f(n) is about (nE?), ignoring the
coefficients.
Decreasing geo. series
26
Home Assignment

• pp.143-
• 3.7
• 3.8
• 3.9
• 3.10