MCA 301: Design and Analysis of Algorithms

1
MCA 301 Design and Analysis of Algorithms

• Instructor
• Neelima Gupta
• ngupta_at_cs.du.ac.in

2
Table Of Contents

• Solving Recurrences
• The Master Theorem

3
Recurrence Relations
4
Recurrences

• The expressionis a recurrence.
• Recurrence an equation that describes a function
  in terms of its value on smaller functions

5
Recurrence Examples
6
Solving Recurrences

• Substitution method
• Iteration method
• Master method

7
Solving Recurrences

• The substitution method (CLR 4.1)
• Making a good guess method
• Guess the form of the answer, then use induction
  to find the constants and show that solution
  works
• Examples
• T(n) 2T(n/2) ?(n) ? T(n) ?(n lg n)
• T(n) 2T(?n/2?) n ? ???

8
Solving Recurrences

• The substitution method (CLR 4.1)
• Making a good guess method
• Guess the form of the answer, then use induction
  to find the constants and show that solution
  works
• Examples
• T(n) 2T(n/2) ?(n) ? T(n) ?(n lg n)
• T(n) 2T(?n/2?) n ? T(n) ?(n lg n)
• T(n) 2T(?n/2? ) 17) n ? ???

9
Substitution method

• Guess the form of the solution .
• Use mathematical induction to find the constants
  and show that the solution works .
• The substitution method can be used to
  establish either upper or lower bounds on a
  recurrence.

10
An example (Substitution method )

• T(n) 2T(floor(n/2) ) n
• We guess that the solution is T(n)0(n lg
  n).
• i.e. to show that T(n) cn lg n , for some
  constant cgt 0 and n m.
• Assume that this bound holds for n/2. So
  , we get
• T(n) 2(c floor (n/2) lg(floor(n/2))) n
• cn lg(n/2) n
• cn lg n cn lg 2 n
• cn lg n cn n
• cn lg n
• where , the last step holds as
  long as c 1.

11

• Boundary conditions
• Suppose , T(1)1 is the sole boundary
  condition of the recurrence .
• then , for n1 , the bound T(n) c n lg n
  yields
• T(1) c lg10 , which is at odds with
  T(1)1.
• Thus ,the base case of our inductive proof
  fails to hold.
• To overcome this difficulty , we can take
  advantage of the asymptotic notation which only
  requires us to prove
• T(n)c n lg n for n m.
• The idea is to remove the difficult boundary
  condition T(1) 1 from consideration.
• Thus , we can replace T(1) by T(2) as the
  base cases in the inductive proof , letting m2.

12
Contd....

• From the recurrence , with T(1) 1, we get
• T(2)4
• We require T(2) c 2 lg 2
• It is clear that , any choice of c2 suffices
  for the base cases

13
Are we done?

• Have we proved the case for n 3.
• Have we proved that T(3) c 3 lg 3.
• No. Since floor(3/2) 1 and for n 1, it does
  not hold. So induction does not apply on n 3.
• From the recurrence, with T(1) 1, we get T(3)
  5.
• The inductive proof that T(n) c n lg n for
  some constant c1
• can now be completed by choosing c large enough
  that
• T(3)c 3 lg 3 also holds.
• It is clear that , any choice of c 2 is
  sufficient for this to hold.
• Thus we can conclude that T(n) c n lg n for any
  c 2 and n 2.

14
Wrong Application of induction
Given recurrence T(n) 2T(n/2) 1 Guess T(n)
O(n) Claim ? some constant c and n0 , such
that T(n) lt cn , ? n gt n0 . Proof
Suppose the claim is true for all values lt
n/2 then T(n)2T(n/2)1 (given) lt
2.c.(n/2) 1 (by induction hypothesis)
cn1 lt (c1) n , ? ngt1
15
Why is it Wrong?
Note that T(n/2)ltcn/2 gt T(n)lt(c1)n (this
statement is true but does not help us in
establishing a solution to the problem)
T(1)ltc (when n1) T(2)lt(c1).2 T(22)lt
(c2).22 . . . . T(2i)lt(ci).2I
So, T(n) T(2logn) (c log n)n
O (n log n)
16
What if we have extra lower order terms?

• So, does that mean that the claim we initially
  made that T(n)O(n) was wrong ?
• No.
• Recall
• T(n)2T(n/2)1 (given)
• lt 2.c.(n/2) 1 (by induction hypothesis)
• cn1
• Note that in the proof we have an extra lower
  order term in our inductive proof.

17
Given Recurrence T(n) 2T(n/2) 1 Guess T(n)
O(n) Claim ? some constant c and n0 , such
that T(n) lt cn - b , ? n gt n0 Proof Suppose
the claim is true for all values lt n/2 then
T(n)2T(n/2)1 lt2c(n/2)-b1
lt cn-2b1 lt cn-b(1-b)
lt cn-b , ? bgt1 Thus, T(n/2) lt cn/2
b gt T(n) lt cn b, ? bgt1 Hence, by
induction T(n) O(n), i.e. Our claim was true.
Hence proved.
18
Solving Recurrences using Recursion Tree
Method

• Here while solving recurrences, we divide the
  problem into
• subproblems of equal size.
• For e.g., T(n) a T(n/b) f(n) where a gt 1
  ,b gt 1 and f(n) is a given
• function .
• F(n) is the cost of splitting or combining
  the sub problems.

n
T n/b
T n/b
19
1) T(n) 2T(n/2) n The recursion tree
for this recurrence is n
n/2 n/2 n log2 n


n
T n/2
T n/2
n/22
n/22
n/22
n/22
1
1
1
1
1
1
1
20

• When we add the values across the levels of
  the recursion tree, we get a value of n for every
  level.
• We have n n n log n
  times
• n (1 1 1 log n
  times)
• n (log2 n)
• O (n log n)
• T(n) O (n log n)

21

• II.
• Given T(n) 2T(n/2) 1
• Solution The recursion tree for the above
  recurrence is
• 1 1 2
• log n 4

1

22

• Now we add up the costs over all levels of
  the recursion tree, to determine the cost for the
  entire tree
• We get series like
• 1 2 22 23 log n times
  which is a G.P.
• So, using the formula for sum of terms
  in a G.P.
• a ar ar2 ar3 ar n 1 a(
  r n 1 )
• r 1
• 1 (2log n 1)
• 2 1
• n 1
• O (n 1) (neglecting the lower order
  terms)
• O (n)

23

• III.
• Given T(n) T(n/3) T(2n/3) n
• Solution The recursion tree for the above
  recurrence is
• n
• n/3 2n/3
  n
• log3 n log3/2 n
• n
• 1 n/3i

n
n/3
2n/3
n 32
2 n 3 3
n . (3/2)2
1 2n 3 3
24

• When we add the values across the levels of
  the recursion tree , we get a value of n for
  every level.
• Since the shortest path from the root to
  the leaf is
• n ? n ? n ? n ? . 1
• 3 32 33
• we have 1 when n 1
• 3i
• gt n 3i
• Taking log3 on both the sides
• gt log3 n i
• Thus the height of the shorter tree is log3
  n
• T(n) gt n log3 n A

25

• Similarly, the longest path from root to
  the leaf is
• n ? 2 n ? 2 2 n ? 1
• 3 3
• So rightmost will be the longest
• when 2 k n 1
• 3
• or n 1
• (3/2)k
• gt k log3/2 n
• T(n) lt n log3/2 n B
• Since base does not matter in asymptotic
  notation , we guess
• from A and B
• T(n) O (n log2 n)

26
Solving Recurrences

• Another option is iteration method
• Expand the recurrence
• Work some algebra to express as a summation
• Evaluate the summation
• We will show some examples

27
Assignment 4

• Solve the following recurrence
• T(n) T(an) T(ßn) n,
• where 0 lt a ß lt 1
• Assume suitable initial conditions.

28
The Master Theorem

• Given a divide and conquer algorithm
• An algorithm that divides the problem of size n
  into a subproblems, each of size n/b
• Let the cost of each stage (i.e., the work to
  divide the problem combine solved subproblems)
  be described by the function f(n)
• Then, the Master Theorem gives us a cookbook for
  the algorithms running time

29
The Master Theorem

• if T(n) aT(n/b) f(n) then

30
Using The Master Method

• T(n) 9T(n/3) n
• a9, b3, f(n) n
• nlogb a nlog3 9 ?(n2)
• Since f(n) O(nlog3 9 - ?), where ?1, case 1
  applies
• Thus the solution is T(n) ?(n2)

31
More Examples of Masters Theorem

• T(n) 3T(n/5) n
• T(n) 2T(n/2) n
• T(n) 2T(n/2) 1
• T(n) T(n/2) n
• T(n) T(n/2) 1

32
When Masters Theorem cannot be applied

• T(n) 2T(n/2) n logn
• T(n) 2T(n/2) n/ logn

33
Up Next

• Proving the correctness of Algorithms