Recurrences

1
Recurrences
2

• Execution of a recursive program
• Deriving solving recurrence equations

3
Recursion

• What is the recursive definition of n! ?
• Program
• int fact(int n)
• if (nlt1) return 1
• else return nfact(n-1)
• // Note '' is done after returning from
  fact(n-1)

4
Recursive algorithms

• A recursive algorithm typically contains
  recursive calls to the same algorithm
• In order for the recursive algorithm to
  terminate, it must contain code for solving
  directly some base case(s)
• A direct solution does not include recursive
  calls.
• We use the following notation
• DirectSolutionSize is the size of the base case
• DirectSolutionCount is the count of the number of
  operations done by the direct solution

5
A Call Tree for fact(3)

• The Run Time Environment
• The following 2 slides show the program stack
  after the third call from fact(3)
• When a function is called an activation
  records('ar') is created and pushed on the
  program stack.
• The activation record stores copies of local
  variables, pointers to other ar in the stack
  and the return address.
• When a function returns the stack is popped.

returns 6
fact(3)
6

fact(2)
3
2

2
fact(1)
1
1
int fact(int n) if (nlt1) return 1else
return nfact(n-1)
6
ARActivation Record ep environmental pointer
free memory
N 1
function value 1
return address
AR(fact 1)
static link ep0
dynamic link EP
EP
N 2
function value ?
return address
AR(fact 2)
static link ep0
dynamic link ep2
ep2
N 3
function value ?
return address
AR(fact 3)
static link ep0
dynamic link ep1
ep1
Val
AR(Driver)
fact ltipF,ep0gt
ep0
Program stack
Snapshot of the environment
at end of third call to fact
7
(No Transcript)
8
Goal Analyzing recursive algorithms

• Until now we have only analyzed (derived a count)
  for non-recursive algorithms.
• In order to analyze recursive algorithms we must
  learn to
• Derive the recurrence equation from the code
• Solve recurrence equations.

9
Deriving a Recurrence Equation fora Recursive
Algorithm

• Our goal is to compute the count (Time) T(n) as a
  function of n, where n is the size of the problem
• We will first write a recurrence equation for
  T(n)
• For example T(n)T(n-1)1 and T(1)0
• Then we will solve the recurrence equation
• When we solve the recurrence equation above
  we will find that T(n)n, and any such recursive
  algorithm is linear in n.

10
Deriving a Recurrence Equation for a Recursive
Algorithm

• Determine the size of the problem. The count T
  is a function of this size
• 2. Determine DirectSolSize, such that for size?
  DirectSolSize the algorithm computes a direct
  solution, and the DirectSolCount(s).

11
Deriving a Recurrence Equation for a Recursive
Algorithm

• To determine GeneralCount
• Identify all the recursive calls done by a single
  call of the algorithm and their counts, T(n1),
  ,T(nk) and compute RecursiveCallSum
• 4. Determine the non recursive count t(size)
  done by a single call of the algorithm, i.e., the
  amount of work, excluding the recursive calls
  done by the algorithm

12
Deriving DirectSolutionCount for Factorial

• int fact(int n)
• if (nlt1) return 1
• else return nfact(n-1)
• 1. Size n
• 2.DirectSolSize is (nlt1)
• 3. DirectSolCount is ?(1)
• The algorithm does a small constant number of
  operations (comparing n to 1, and returning)

13
Deriving a GeneralCount for Factorial

• int fact(int n)
• if (nlt1) return 1
• // Note '' is done after returning from
  fact(n-1)else
• return n fact(n-1)
• 3. RecursiveCallSum T( n - 1)
• 4. t(n) ?(1) (if, , -, return)

Operations counted in t(n)
The only recursive call, requiring T(n - 1)
operations
14
Deriving a Recurrence Equation for Mystery

• In this example we are not concerned with what
  mystery computes. Our goal is to simply write the
  recurrence equation
• mystery (n)
• 1. if n 1
• 2. then return n
• 3. else return (5 mystery(n -1) - 6
  mystery(n -2) )

15
Deriving a DirectSolutionCount for Mystery

• 1. Size n
• 2. DirectSolSize is (nlt1)
• 3. DirectSolCount is ?(1)
• The algorithm does a small constant number of
  operations (comparing n to 1, and returning)

16
Deriving a GeneralCount for Mystery
A recursive call to Mystery requiring T(n - 1)

• mystery (n)
• 1. if n 1
• 2. then return n
• 3. else
• return (5 mystery(n -1) - 6 mystery(n
  - 2) )

A recursive call to Mystery requiring T(n - 2)
Operations counted in t(n)
17
Deriving a Recurrence Equation for Mystery

• 3. RecursiveCallSum T(n-1) T(n-2)
• 4. t (n) ?(1)
• The non recursive count includes the comparison
  nlt1, the 2 multiplications, the subtraction, and
  the return. So ?(1).

18
Deriving a Recurrence Equation for Multiply
Barometer operation

• Multiply(y, z)//binary multiplication
• 1. if (z 0) return 0
• 2. else if z is odd
• 3. then return (multiply(2y,ëz/2û) y)
• 4. else return (multiply(2y,ëz/2û))
• To derive the recurrence equation we need to
  determine the size
• of the problem. The variable which is decreased
  and provides the
• direct solution is z. So we can use z as the
  size. Another
• possibility is to use n which is the number of
  bits in z (nëlg zû1).
• Let addition be our barometer operation. We do 0
  additions for the
• direct solution. So DirectSolutionSize 0, and
  DirectSolutionCount0

19
Deriving a Recurrence Equation for Multiply

• 1. RecursiveCallSum T(z/2)
• t(z) 1 addition in the worst case. So
• Solving this equation we get
• T(z) ?(lg z) ?(n).
• 2. RecursiveCallSum T(n-1) since z/2 has n-1
  bits
• Solving this equation we get T(n) ?(n)

20
Deriving a recurrence equation for minimum

• Minimum(A,lt,rt) // minimum(A,1,length(A))
• 1. if rt - lt 1
• 2. then return (min(Alt, Art))
• 3. else m1 minimum(A,lt, ë(ltrt)/2û )
• 4. m2 minimum(A,ë(ltrt)/2û 1,rt)
• 5. return (min (m1, m2))
• This procedure computes the minimum of the left
  half of the array, the right half of the array,
  and the minimum of the two.
• Size rt-lt1 n
• DirectSolutionSize is n (rt - lt)1 112.
• DirectSolutionCount ?(1)

21
Deriving a recurrence equation for minimum

• Minimum(A,lt,rt) // minimum(A,1,length(A))
• 1. if rt - lt 1
• 2. then return (min(Alt, Art))
• 3. else m1 minimum(A,lt, ë(ltrt)/2û )
• 4. m2 minimum(A,ë(ltrt)/2û 1,rt)
• 5. return (min (m1, m2))

Two recursive calls with n/2 elements. So T(n/2)
T(n/2)
22
Computing xn

• We can decrease the number of times that x is
  multiplied by itself
• Example compute x8 we need only 3
  multiplications
• (x2xx, x4x2x2, and x8x4x4)
• compute x9, we need 4
  multiplications
• (x2xx, x4x2x2, x8x4x4 and x9
  x8x)
• A recursive procedure for computing xn should be
  ?(lg n)
• For the following algorithm we will derive the
  worst case recurrence equation.
• Note we are ignoring the fact that the numbers
  will quickly become too large to store in a long
  integer

23
Recursion - Power

• long power (long x, long n)
• if (n 0) return 1
• else
• long p power(x, n/2)
• if ((n 2) 0) return
  pp
• else return x pp
• We use the as our barometer operation.

24
Deriving a Recurrence Equation for Power

• Size n
• DirectSolutionSize is n0
• DirectSolutionCount is 0
• RecursiveCallSum is T(n/2)
• t(n) 2 multiplications in the worst case
• T(n)0 for n0
• T(n)T(n/2)2 for ngt0

25
Solving recurrence equations

• 5 techniques for solving recurrence equations
• Recursion tree
• Iteration method
• Induction (Guess and Test)
• Master Theorem (master method)
• Characteristic equations
• We discuss these methods with examples.

26
Deriving the count using the recursion tree method

• Recursion trees provide a convenient way to
  represent the unrolling of recursive algorithm
• It is not a formal proof but a good technique to
  compute the count.
• Once the tree is generated, each node contains
  its non recursive number of operations t(n) or
  DirectSolutionCount
• The count is derived by summing the non
  recursive number of operations of all the nodes
  in the tree
• For convenience we usually compute the sum for
  all nodes at each given depth, and then sum these
  sums over all depths.

27
Building the recursion tree

• The initial recursion tree has a single node
  containing two fields
• The recursive call, (for example Factorial(n))
  and
• the corresponding count T(n) .
• The tree is generated by
• unrolling the recursion of the node depth 0,
• then unrolling the recursion for the nodes at
  depth 1, etc.
• The recursion is unrolled as long as the size of
  the recursive call is greater than
  DirectSolutionSize

28
Building the recursion tree

• When the recursion is unrolled, each current
  leaf node is substituted by a subtree containing
  a root and a child for each recursive call done
  by the algorithm.
• The root of the subtree contains the recursive
  call, and the corresponding non recursive
  count.
• Each child node contains a recursive call, and
  its corresponding count.
• The unrolling continues, until the size in the
  recursive call is DirectSolutionSize
• Nodes with a call of DirectSolutionSize, are not
  unrolled, and their count is replaced by
  DirectSolutionCount

29
Example recursive factorial

• Initially, the recursive tree is a node
  containing the call to Factorial(n), and count
  T(n).
• When we unroll the computation this node is
  replaced with a subtree containing a root and one
  child
• The root of the subtree contains the call to
  Factorial(n) , and the non recursive count for
  this call t(n) ?(1).
• The child node contains the recursive call to
  Factorial(n-1), and the count for this call,
  T(n-1).

30
After the first unrolling
depth nd T(n)
0 1 ?(1)
1 1 T(n-1)
nd denotes the number of nodes at that depth
31
After the second unrolling
depth nd T(n)
0 1 ?(1)
1 1 ?(1)
2 1 T(n-2)
32
After the third unrolling
depth nd T(n)
0 1 ?(1)
1 1 ?(1)
2 1 ?(1)
3 1 T(n-3)
For Factorial DirectSolutionSize 1 and
DirectSolutionCount ?(1)
33
The recursion tree
depth nd T(n)
0 1 ?(1)
1 1 ?(1)
2 1 ?(1)
3 1 ?(1)
?(1)
n-1 1 T(1) ?(1)
The sum T(n)n ?(1) ?(n)
34
Divide and Conquer

• Basic idea divide a problem into smaller
  portions, solve the smaller portions and combine
  the results.
• Name some algorithms you already know that employ
  this technique.
• DC is a top down approach. Usually, we use
  recursion to implement DC algorithms.
• The following is an outline of a divide and
  conquer algorithm

35
Divide and Conquer

• procedure Solution(I)
• begin
• if size(I) ltsmallSize then calculate solution
• return(DirectSolution(I)) use direct solution
• else decompose, solve each and combine
• Decompose(I, I1,...,Ik)
• for i1 to k do
• S(i)Solution(Ii) solve a smaller problem
• return(Combine(S1,...,Sk)) combine solutions
• end Solution

36
Divide and Conquer

• Let size(I) n
• DirectSolutionCount DS(n)
• t(n) D(n) C(n) where
• D(n) count for dividing problem into
  subproblems
• C(n) count for combining solutions

37
Divide and Conquer

• Main advantages
• Code simple
• Algorithm efficient
• Implementation parallel computation

38
Binary search

• Assumption - the list Slowhigh is sorted, and
  x is the search key
• If the search key x is in the list, xSi, and
  the index i is returned.
• If x is not in the list a NoSuchKey is returned

39
Binary search

• The problem is divided into 3 subproblems
  xSmid, xÎ Slow,..,mid-1, xÎ Smid1,..,high
• The first case xSmid) is easily solved
• The other cases xÎ Slow,..,mid-1, or xÎ
  Smid1,..,high require a recursive call
• When the array is empty the search terminates
  with a non-index value

40

• BinarySearch(S, k, low, high)
• if low gt high then
• return NoSuchKey
• else
• mid ? floor ((lowhigh)/2)
• if (k Smid)
• return mid
• else if (k lt Smid) then
• return BinarySearch(S, k, low, mid-1)
• else
• return BinarySearch(S, k, mid1, high)

41
Worst case analysis

• A worst input (what is it?) causes the algorithm
  to keep searching until lowgthigh
• We count number of comparisons of a list element
  with x per recursive call.
• Assume 2k ? n lt 2k1 k ëlg nû

• T (n) - worst case number of comparisons for
  the call to BS(n)

42
Recursion tree for BinarySearch (BS)

• Initially, the recursive tree is a node
  containing the call to BS(n), and total amount of
  work in the worst case, T(n).
• When we unroll the computation this node is
  replaced with a subtree containing a root and one
  child
• The root of the subtree contains the call to
  BS(n) , and the nonrecursive work for this call
  t(n).
• The child node contains the recursive call to
  BS(n/2), and the total amount of work in the
  worst case for this call, T(n/2).

43
After first unrolling
depth nd T(n)
0 1 1
1 1 T(n/2)
44
After second unrolling
depth nd T(n)
0 1 1
1 1 1
2 1 T(n/4)
45
After third unrolling
depth nd T(n)
0 1 1
1 1 1
2 1 1
T(n/8)
For BinarySearch DirectSolutionSize 0, 1 and
DirectSolutionCount0 for 0, and 1 for 1
46
Terminating the unrolling

• Let 2k? n lt 2k1
• k? ??lg n?
• When a node has a call to BS(n/2k), (or to
  BS(n/2k1))
• The size of the list is DirectSolutionSize since
  ? ?? n/2k ? 1, (or ?? n/2k1 ? 0)
• In this case the unrolling terminates, and the
  node is a leaf containing DirectSolutionCount
  (DSC) 1, (or 0)

47
The recursion tree
depth nd T(n)
0 1 1
1 1 1
2 1 1
k 1 1
T(n)k1?lgn? 1
48
Merge Sort

• Input S of size n.
• Output a permutation of S, such that if i gt j
  then S i ? S j
• Divide If S has at least 2 elements divide into
  S1 and S2. S1 contains the the first ? n/2 ?
  elements of S. S2 has the last ? n/2? elements
  of S.
• Recurs Recursively sort S1 , and S2.
• Conquer Merge sorted S1 and S2 into S.

49
Merge Sort

• Input S1 if (n ? 2) 2 moveFirst( S, S1,
  (n1)/2))// divide3 moveSecond( S, S2, n/2)
  // divide4 Sort(S1) // recurs 5
  Sort(S2) // recurs6 Merge( S1, S2, S ) //
  conquer

50
Merge( S1 , S2, S ) Pseudocode

• Input S1 , S2, sorted in nondecreasing order
• Output S is a sorted sequence.1. while (S1 is
  Not Empty S2 is Not Empty)2. if (S1.first() ?
  S2.first() )3. remove first element of S1
  and move it to the end of S4. else
  remove first element of S2 and move it
  to the end of S5. move
  the remaining elements of the non-empty sequence
  (S1 or S2 ) to S

51
Deriving a recurrence equation for Merge Sort

• 1 if (n ? 2) 2 moveFirst( S, S1, (n1)/2))
  // divide3 moveSecond( S, S2, n/2)
  // divide4 Sort(S1) // recurs 5
  Sort(S2) // recurs6 Merge( S1, S2, S )
  // conquers
• DirectSolutionSize is nlt2
• DirectSolutionCount is ?(1)
• RecursiveCallSum is T( ?n/2? ) T( ?n/2? )
• The non recursive count t(n) ?(n)

52
Recurrence Equation (contd)

• The implementation of the moves costs Q(n) and
  the merge costs Q(n). So the total cost for
  dividing and merging is Q(n).
• The recurrence relation for the run time of Sort
  is T(n) T( ?n/2? ) T( ?n/2? ) Q(n) .
  T(0) T(1) Q(1)
• The solution is T(n) Q(nlg n)

53
Recursion tree for MergeSort (MS)

• Initially, the recursive tree is a node
  containing the call to MS(n), and total amount of
  work in the worst case, T(n).
• When we unroll the computation this node is
  replaced with a subtree
• The root of the subtree contains the call to
  MS(n) , and the nonrecursive work for this call
  t(n).
• The two children contain a recursive call to
  MS(n/2), and the total amount of work in the
  worst case for this call, T(n/2).

54
After first unrolling of mergeSort
depth nd T(n)
MS(n)
t(n)?(n)
0 1 ?(n)
2T(n/2)
55
After second unrolling
depth nd T(n)
0 1 ?(n)
1 2 ?(n)

4T(n/4)
56
After third unrolling
depth nd T(n)
0 1 ?(n)
1 2 ?(n)

2 4 ?(n)



8T(n/8)
57
Terminating the unrolling

• For simplicity let n 2k
• ?lg n k
• When a node has a call to MS(n/2k)
• The size of the list to merge sort is
  DirectSolutionSize since n/2k 1
• In this case the unrolling terminates, and the
  node is a leaf containing DirectSolutionCount
  ?(1)

58
The recursion tree
d nd T(n)
0 1 Q (n)
1 2 Q (n)
2 4 Q (n)
3 8 Q (n)
k 2k Q (n)
T(n) (k1) Q (n) Q( n lg n)
59
The sum for each depth
60
Induction method

• Estimate the form of the solution
• Prove by mathematical induction

For Example Computing Factorial Fact(n)
Fact(n-1)n Number of T(n) T(n-1) 1
(Recurrence equation) Proof Induction base
for n0 ? T(0) 0 Induction hypothesis for
arbitrary positive integer n ? T(n) n
(Guess) Induction step to show ? T(n1)
n1 Replace n by n1 in the recurrence equation,
we get T(n1) T(n) 1 n1 (Its
correct!) So this completes the proof that T(n)
is correct ? T(n) ?(n)
61
Induction (Guess and Test) for Binary Search
W(n) ?lg n? 1

• Proof by Induction
• Induction base n 1W(1) ?lg 1? 1
  0 1 1
• Induction hypothesis Assume for 1 ? k lt n,
  W(k) ?lg k? 1
• (Assume that above condition holds for n/2)
• Proof for n W(n) 1 W(?n/2?) 1 ?lg
  ?n/2? ? 1 (see next slide)

62
Proof Continued.
63
Iteration for binary search
64
Characteristic Equation (example)
Solve the recurrence
65
Characteristic Equation (one case)
The homogeneous linear recurrence equation with
constant coefficients Its characteristic
equation is If there are k distinct
solutions r1, r2, , rk, the only solution to the
recurrence is
66
The Master Method
Solving recurrence in the form T(n) aT(n/b)
f(n) (agt1, bgt1, f(n) is an asympototically
positive function.)
The time complexity T(n) is bounded in following
three cases Case 1
(If
) Case 2 (If
) Case 3
(If
and
af(n/b) lt cf(n) for clt1 and large n )
Example T(n) 4T(n/2)n ? a4, b2, Case
1 applies ? Solution T(n) ?(n2)

67
Master theorem (alternative form)
68
Master method examplesCase 1
T(n) 9T(n/3) Q(n). a 9, b 3 so nlogba
nlog39 Since Q(n) Q(nlog39 -1 ) we are
dealing with Case 1. Therefore T(n) Q (nlogba
)Q(nlog39) Q(n2).

• T(n) 7T(n/2) Q(n2). a 7, b 2 so nlogba
  nlog27
• Since Q(n2)/nlog27 Q(n2-lg 7) Q(n-0.8),
• we are dealing with Case 1.
• Therefore T(n) Q (nlogba ) Q(nlog27).

69
Master method examplesCase 2

• T(n) T(n/2) Q(1)a 1, b 2 so nlogba
  nlog21
• Since nlog21 n0 1, Q(1)/1 Q(lg0n), we are
  dealing with Case 2.
• T(n) Q (nlogba lgk1n) Q(lg n)

T(n) 2T(n/2) Q(n)a 2, b 2 so nlogba
nlog22 Since nlog22 n, Q(n)/n Q(1)
Q(lg0n), we are dealing with Case 2. T(n)
Q(nlogba lgk1n) Q(nlgn)
70
Master method examplesCase 3

• T(n) 4T(n/2) n3 a 4, b 2 so nlogba
  nlog24
• Since n3/nlg4 n, we are dealing with Case 3.
• We must find a c such that af(n/b) lt cf(n) for
  all n gtN
• 4(n/2)3 n3/2lt5/8n3 for all n³1
• T(n) Q(n3)
• The master method does not solve all recurrences
  of the given form.

71
Master method fails

• T(n) 4T(n/2) n2/lgn a 4, b 2 so nlogba
  nlog24 n2
• (n2/lgn)/n2 1/lgn and no case applies

The solution is T(n) Q(n2lglgn) and can be
verified by the substitution method.
72
A More General Recursion Tree T(n) a T(n/b)
f(n)
f(n)
f(n)
f(n/b)
f(n/b)
f(n/b)
af(n/b)
a2f(n/b2)
f(n/b2)
f(n/b2)
f(n/b2)
k
Q(1) Q(1) Q(1) ... Q(1) Q(1) Q(1) ...
Q(1) Q(1) Q(1)
akf(n/bk)
nbk, klogbn, f(n/bk )f(1) Q (1)alogbn
nlogba
73
Deriving a Recurrence Equation from Recursive
Algorithms

• Decide what n, the problem size, is.
• See what value of n is used as the base of the
  recursion. It will usually be a single value (ie
  n 1). However, it may be multiple values. Let
  the base value be n0 .
• Figure out what T(n0 ) is. You can usually use
  some constant c. Sometimes a specific number
  will be needed.
• The General T(n) is usually a sum of various
  choices of T(m ), the recursive calls, plus the
  sum of the other work done.
• T(n ) aT(f(n)) bT(g(n)) . . . c(n)

of subProblems
other work
size function
74
Recursion - Power

• long power (long x, long n)
• if (n 0) return 1
• else if ((n 2) 0) return power
  (x, n/2) power (x, n/2)
• else return x power (x, n/2) power
  (x, n/2)
• Worst case -- Count 2 multiplications.

75
T(n) 2 T(n/2) 2, n 2k T(0) 0 (n0)
--- Worst case

• Problem number number Size nodes
  of ops
• n 1 2
• n/2 2 2T(n/2)
• n 1 2
• n/2 2 22 4 n/4 4 4T(n/4)

T(n )
(2)
T(n / 2)
T(n / 2)
2
2
2
T(n / 4)
T(n / 4)
T(n / 4)
T(n / 4)
76
T(n) 2 T(n/2) 2, n 2k

• Problem number number Size nodes
  of ops
• n 1 2 n/2 2 224 n/4 4
  428 n/8 8 8T(n/8)

2
2
2
2
2
2
2
?
?
T(n/8)
T(n/8)
T(n/8)
T(n/8)
77
T(n) 2 T(n/2) 2, n 2k

• Problem number number Size nodes
  of ops
• n 1 2022 n/2 2 2?24 n/4 4
  4?28 n/8 8 8?216
• n/2k 2k 2k ?2 0 2k1 2k1?0

2
2
2
2
2
2
2
?
?
2
2
2
2
2
2
2
2
T(n/2k1)
T(n/2k1)
T(0)
T(0)

• SUMMING operations2(20 21 22 23 24
  2k) 2(2k1-1) 2(2n -1)