Running time of Recursive algorithm

1
Running time of Recursive algorithm
2
MergeSort
List mergeSort (List list) if (list
nill) return nill else if (list.next nill)
return list else secondList split
(list) return merge(mergeSort(list),
mergeSort(secondList))
3
MergeSort - merge
List merge (List list1, List list2) (1) if
(list1 null) return list2 // list1 is
empty (2) else if (list2 null) return list1
// list2 is empty (3) else if (list1.data lt
list2.data) // smaller element in list1 (4)
list1.next merge(list1.next,
list2) (5) return list1 else //
list2.data lt list1.data // smaller element in
list2 (6) list2.next merge(list1,
list2.next) (7) return list2
4
MergeSort - split
List split(List list) List secondList if
(list null) return null else secondLisy
list.next list.next secondList.next secondLis
t.next split(secondList.next) return
secondList
5
Running Time of Recursive FunctionsExample
Example 1 Factorial
Int fact(int n) if (n lt 1) return 1
// Base Case else return n
fact(n-1) // Inductive Step
T(1) O(1) n 1 T(n) O(1)
T(n-1) n gt 1
recurrence
6
Solving Recurrence Relation

• Many methods exists. Will show the following
  three
• The Iteration method
• Iterate the inductive rule until get T(n) in
  terms of n and the base case.
• Requires dealing with summations
• The substitution method
• Guess a solution
• Substitute your guess in the base and the
  inductive cases
• Prove correctness of the guess by mathematical
  induction
• The master method
• Can deal with recurrence relations of the
  following form T(n) aT(n/b) f(n)

7
I - Solving Recurrence by Iteration
A simple example Base T(1) a
Induction T(m) b T(m-1)
I. Replace m by n, n-1, n-2 .. Until get to
the base case 1. T(n) b T(n-1)
substitute m with n 2. T(n-1) b
T(n-2) substitute m with n-1 3. T(n-2)
b T(n-3) substitute m with n-2 ..
T(2) b T(1) substitute m
with 2 T(1) a (known)
8
Recurrence by Iteration (cont)
II. Substitute T(n-1), T(n-2) etc
successively until reach the base case and
substitute it. T(n) b T(n-1)
b b T(n-2) 2b T(n-2) 2b
b T(n-3) 3b T(n-3) 3b b
T(n-4) 4b T(n-4) T(n) (n-1)
b T(1) intuition 1 n -
(n-1) Substitute with the base case T(n)
(n-1) b a III. Evaluate associated big-O
expression T(n) bn - b a O(n)
9
MergeSort merge (cont)
1-7
2-7
if
Analyzing merge
else-if
3-7
1
else-if
2
6-7
else
block
BaseBase T(0) O(1), T(1)
O(1) Induction O(1) max(O(1), O(1)
T(n-1)) O(1) T(n-1), for
ngt1 Thus, T(n) c T(n-1) Like factorial.
Analysis will result in T(n) O(n)
7
4
5
6
10
II - Solving Recurrence by Substitution
Guess the form of the solution, then use
mathematical induction to find the constants of
the solution and to show that the guess is
correct.
Example T(1) a
T(n) 2T(n/2) n, for ngt1 guess
T(n) O(n lg n). (show
the intuition) Prove by induction T(n) lt c
n lg n, for an appropriate choice of
cgt0.
11
Example mergesort
List mergeSort (List list) List
secondList (1) if (list nill) return
nill (2) else if (list.next nill) return
list else (3) secondList split
(list) (4) return merge(mergeSort(list),
mergeSort(secondList))
12
MergeSort (cont)
Basis T(1) O(1) Induction 1. O(1)
for two tests (lines 1,2) 2. O(1)
(assignment) O(n) (call Split) (line
3) 3. T(n/2) (call MergeSort)
(line 4) 4. T(n/2) (call MergeSort)
(line 4) 5. O(n) (call Merge)
(line 4) 6. O(1) (return)
(line 4) T(n) O(n) 2T(n/2)

13
MergeSort (cont)
recurrence relation replace big-O notation
with hidden constants Base T(1)
a Induction T(n) 2T(n/2) bn for n a
power of 2 and n gt 1 Using the recursion trees,
we can see that by substitution, we get
bnbnbn... bn(depth of tree), where
depth of tree n/2i 1 when ilg n, thus can
guess (n lg n). Remains to prove correctness of
our guess (proof by induction).
14
Proving your Guess by Mathematical Induction
Proof I. Inductive step Assume the bound
holds for n/2, namely T(n/2) lt
c(n/2)lg(n/2) Prove the bound holds for n
T(n) 2T(n/2) n lt
2c(n/2)lg(n/2) n (inductive assumption)
lt cn lg(n/2) n cn lgn
- cn lg2 n cn lg n - cn n.
providing cgt1 lt cn lg n
15
Proving your guess by Mathematical Induction
(cont)
II - Base step Need to prove T(1) a lt
c1 lg 1 0 Not Possible. However, big-O
means for some n0 , thus can choose an
appropriate n for the boundary (base)
condition. In this case, choose n0 2
T(2) lt c 2 lg 2 2c, which is ok for c
T(2)/2.
16
III - The Master Method

• A cookbook for recurrences of the form
• T(n) aT(n/b) f(n), under the following
  conditions
• a gt 1,
• b gt 1 (constants),
• f(n) - asymptotically positive
• Represents the running time of an algorithm which
  solves a problem of size n, by splitting it to
  a problems of size n/b (divide and conquer)
• f(n) represents the cost of splitting the
  problem and combining the results.
• Example Mergesort T(n) 2 T(n/2) q (n)

(I)
17
The Master Method (cont)

• Given (I), then
• If f(n) O(n log a - e) for some constant
  e gt 0 T(n) q (n log a )
  (notation log base b, lg base 2)
• If f(n) q (n log a )
  T(n) q (n log
  a lg n) q (f(n) lg n)
• If f(n) W (n log a e) for some constant
  e gt 0, af(n/b) lt c f(n) for clt1, and large
  enough n T(n) q (f(n))