CS 3343: Analysis of Algorithms

1
CS 3343 Analysis of Algorithms

• Lecture 6 Analyzing recursive algorithms

2
Outline

• Analyzing recursive algorithms
• Defining recurrence
• Solving recurrence
• Recursion tree method
• Substitution method
• Master method

3
Merge sort
T(n) 2 T(n/2) T(n)
subproblems
Work dividing and Combining
subproblem size
4
Power

• int pow (b, n)
• m n gtgt 1
• ppow(b,m)pow(b,m)
• if (n 2)
• return p b
• else
• return p

int pow (b, n) m n gtgt 1 p pow (b, m) p
p p if (n 2) return p
b else return p
T(n) T(n/2) T(1)
T(n) 2T(n/2) T(1)
5
3-way-merge-sort

• 3-way-merge-sort (A1..n)
• If (n lt 1) return
• 3-way-merge-sort(A1..n/3)
• 3-way-merge-sort(An/31..2n/3)
• 3-way-merge-sort(A2n/31.. n)
• Merge A1..n/3 and An/31..2n/3
• Merge A1..2n/3 and A2n/31..n

• Is this algorithm correct?
• Whats the recurrence function for the running
  time?
• What does the recurrence function solve to?

6
Unbalanced-merge-sort

• ub-merge-sort (A1..n)
• if (nlt1) return
• ub-merge-sort(A1..n/3)
• ub-merge-sort(An/31.. n)
• Merge A1.. n/3 and An/31..n.

• Is this algorithm correct?
• Whats the recurrence function for the running
  time?
• What does the recurrence function solve to?

7
Solving recurrence

• Recurrence tree (iteration) method
• - Good for guessing an answer
• Substitution method
• - Generic method, rigid, but may be hard
• Master method
• - Easy to learn, useful in limited cases only
• - Some tricks may help in other cases

8
Recursion-tree method

• A recursion tree models the costs (time) of a
  recursive execution of an algorithm.
• The recursion-tree method can be unreliable, just
  like any method that uses ellipses ().
• It is good for generating guesses of what the
  runtime could be.
• But Need to verify that the guess is right. ?
  Induction (substitution method)

9
Recursion tree example 1
Solve T(n) 2T(n/2) dn, where d gt 0 is
constant.
10
Recursion tree example 1
Solve T(n) 2T(n/2) dn, where d gt 0 is
constant.
T(n)
11
Recursion tree example 1
Solve T(n) 2T(n/2) dn, where d gt 0 is
constant.
12
Recursion tree example 1
Solve T(n) 2T(n/2) dn, where d gt 0 is
constant.
13
Recursion tree example 1
Solve T(n) 2T(n/2) dn, where d gt 0 is
constant.
dn
dn/2
dn/2
dn/4
dn/4
dn/4
dn/4

Q(1)
14
Recursion tree example 1
Solve T(n) 2T(n/2) dn, where d gt 0 is
constant.
dn
dn/2
dn/2
h log n
dn/4
dn/4
dn/4
dn/4

Q(1)
15
Recursion tree example 1
Solve T(n) 2T(n/2) dn, where d gt 0 is
constant.
dn
dn
dn/2
dn/2
h log n
dn/4
dn/4
dn/4
dn/4

Q(1)
16
Recursion tree example 1
Solve T(n) 2T(n/2) dn, where d gt 0 is
constant.
dn
dn
dn/2
dn
dn/2
h log n
dn/4
dn/4
dn/4
dn/4

Q(1)
17
Recursion tree example 1
Solve T(n) 2T(n/2) dn, where d gt 0 is
constant.
dn
dn
dn/2
dn
dn/2
h log n
dn/4
dn/4
dn
dn/4
dn/4


Q(1)
18
Recursion tree example 1
Solve T(n) 2T(n/2) dn, where d gt 0 is
constant.
dn
dn
dn/2
dn
dn/2
h log n
dn/4
dn/4
dn
dn/4
dn/4


Q(1)
leaves n
Q(n)
19
Recursion tree example 1
Solve T(n) 2T(n/2) dn, where d gt 0 is
constant.
dn
dn
dn/2
dn
dn/2
h log n
dn/4
dn/4
dn
dn/4
dn/4


Q(1)
leaves n
Q(n)
Total Q(n log n)
20
Recursion tree example 2
Solve T(n) 2T(n/2) 1.
21
Recursion tree example 2
Solve T(n) 2T(n/2) 1.
T(n)
22
Recursion tree example 2
Solve T(n) 2T(n/2) 1.
23
Recursion tree example 2
Solve T(n) 2T(n/2) 1.
24
Recursion tree example 2
Solve T(n) 2T(n/2) 1.
1
1
1
1
1
1
1

Q(1)
25
Recursion tree example 2
Solve T(n) 2T(n/2) 1.
1
1
1
h log n
1
1
1
1

Q(1)
26
Recursion tree example 2
Solve T(n) 2T(n/2) 1.
1
1
1
1
h log n
1
1
1
1

Q(1)
27
Recursion tree example 2
Solve T(n) 2T(n/2) 1.
1
1
1
2
1
h log n
1
1
1
1

Q(1)
28
Recursion tree example 2
Solve T(n) 2T(n/2) 1.
1
1
1
2
1
h log n
1
1
4
1
1


Q(1)
29
Recursion tree example 2
Solve T(n) 2T(n/2) 1.
1
1
1
2
1
h log n
1
1
4
1
1


Q(1)
leaves n
Q(n)
30
Recursion tree example 2
Solve T(n) 2T(n/2) 1.
1
1
1
2
1
h log n
1
1
4
1
1


Q(1)
leaves n
Q(n)
Total Q(n)
31
Substitution method
The most general method to solve a recurrence
(prove O and ? separately)

• Guess the form of the solution
• (e.g. using recursion trees, or expansion)
• Verify by induction (inductive step).
• Solve for O-constants n0 and c (base case of
  induction)

32
Proof by substitution

• Recurrence T(n) 2T(n/2) n.
• Guess T(n) O(n log n). (eg. by recurrence tree
  method)
• To prove, have to show T(n) c n log n for
  some c gt 0 and for all n gt n0
• Proof by induction assume it is true for T(n/2),
  prove that it is also true for T(n). This means
• Fact T(n) 2T(n/2) n
• Assumption T(n/2) cn/2 log (n/2)
• Need to Prove T(n) c n log (n)

33
Proof

• Fact T(n) 2T(n/2) n
• Assumption T(n/2) cn/2 log (n/2)
• Need to Prove T(n) c n log (n)
• Proof
• Substitute T(n/2) cn/2 log (n/2) into the
  recurrence, we get
• T(n) 2 T(n/2) n
• cn log (n/2) n
• c n log n - c n n
• c n log n (if we choose c 1).

34
What about base case?

• T(n) 2T(n/2) n O(n logn)
• Need to show that T(n) lt c n log n
• We have proved the inductive hypothesis, i.e., if
  the above relation is true for T(n/2), then it is
  also true for T(n)
• We need to prove the base case
• When n 1, c n log n 0. OOPS!
• Fortunately we can choose larger n

35
Proof by substitution

• Recurrence T(n) 2T(n/2) n.
• Guess T(n) O(n log n).
• To prove, have to show T(n) c n log n for
  some c gt 0 and for all n gt n0
• Proof by induction assume it is true for T(n/2),
  prove that it is also true for T(n). This means
• Fact
• Assumption
• Need to Prove T(n) c n log (n)

T(n) 2T(n/2) n
T(n/2) cn/2 log (n/2)
36
Proof

• Fact T(n) 2T(n/2) n
• Assumption T(n/2) cn/2 log (n/2)
• Need to Prove T(n) c n log (n)
• Proof
• Substitute T(n/2) cn/2 log (n/2) into the
  recurrence, we get
• T(n) 2 T(n/2) n
• cn log (n/2) n
• c n log n - c n n
• c n log n (if we choose c 1).

37
More recursion tree examples

• T(n) 3T(n/3) n
• T(n) T(n/3) T(2n/3) n
• T(n) 3T(n/4) n2

38
More substitution method examples (1)

• Prove that T(n) 3T(n/3) n O(nlogn)
• Need to prove that T(n) lt d n log n for some d,
  and sufficiently large n
• Assume above is true for T(n/3), i.e.
• T(n/3) lt dn/3 log (n/3)

39

• T(n) 3 T(n/3) n
• lt 3 dn/3 log (n/3) n
• dn log n dn log3 n
• lt dn log n when n dn log3 lt 0
• n dn log3 lt0 implies d gt 1/log3

40
More substitution method examples (2)

• Prove that T(n) T(n/3) T(2n/3) n O(nlogn)
• Need to prove that T(n) lt d n log n for some d,
  and sufficiently large n
• Assume above is true for T(n/3) and T(2n/3), i.e.
• T(n/3) lt dn/3 log (n/3)
• T(2n/3) lt 2dn/3 log (2n/3)

41

• T(n) T(n/3) T(2n/3) n
• lt dn/3 log(n/3) 2dn/3 log(2n/3) n
• dn log n n dn (log 3 2/3)
• dn log n n(1 dlog3 2d/3)
• lt dn log n when 1 d log3 2d/3 lt 0
• d log3 2d/3 gt 1 gt d gt 1 / (log3-2/3)

42
More substitution method examples (3)

• Prove that T(n) 3T(n/4) n2 O(n2)
• Need to prove that T(n) lt d n2 for some d, and
  sufficiently large n
• Assume above is true for T(n/4), i.e.
• T(n/4) lt d(n/4)2 dn2/16

43

• T(n) 3T(n/4) n2
• lt 3 d n2 / 16 n2
• (3d/16 1) n2
• lt dn2
• 3d/16 1 lt d implies that
• d gt 16/13

?
44
Making good guess

• T(n) 2T(n/2 17) n
• When n -gt infinity, n/2 17 are not too
  different from n/2
• Guess T(n) ?(n log n)

45
Subtleties

• Prove that T(n) T(?n/2?) T(?n/2?) 1 O(n)
• Need to prove that T(n) lt cn
• Assume it is true for T(?n/2?) T(?n/2?)
• T(n) lt c ?n/2? c?n/2? 1
• cn 1
• Is it correct?
• Can assume T(n) lt cn - 1

46
Avoiding pitfalls

• Guess T(n) 2T(n/2) n O(n)
• Need to prove that T(n) lt c n
• Assume T(n/2) lt cn/2
• T(n) lt 2 cn/2 n cn n O(n)
• Whats wrong?