Recursion

1
Recursion
Thinking about Algorithms Abstractly
Multiplying Code Stack of Stack Frames Tree of
Stack Frames Friends and Strong Induction Towers
of Hanoi Check List Merge Quick Sort Simple
Recursion on Trees Generalizing the
Problem Things not to do Heap Sort Priority
Queues Trees Representing Equations Pretty
Print Parsing Recursive Images

• Jeff Edmonds
• York University

COSC 3101
Lecture 3
2
Grade School RevisitedHow To Multiply Two
Numbers
A Few Example Algorithms

• Rudich www.discretemath.com

3
Slides in this next section produced by

• Steven Rudich
• from Carnegie Mellon University

Individual Slides will be marked

• Rudich www.discretemath.com

4
Complex Numbers

• Remember how to multiply 2 complex numbers?
• (abi)(cdi) ac bd ad bc i
• Input a,b,c,d Output ac-bd, adbc
• If a real multiplication costs 1 and an addition
  cost a penny. What is the cheapest way to obtain
  the output from the input?
• Can you do better than 4.02?

5
Gauss 3.05 MethodInput a,b,c,d Output
ac-bd, adbc

• m1 ac
• m2 bd
• A1 m1 m2 ac-bd
• m3 (ab)(cd) ac ad bc bd
• A2 m3 m1 m2 adbc

6
Question

• The Gauss hack saves one multiplication out of
  four. It requires 25 less work.
• Could there be a context where performing 3
  multiplications for every 4 provides a more
  dramatic savings?

7
How to add 2 n-bit numbers.












8
How to add 2 n-bit numbers.












9
How to add 2 n-bit numbers.












10
How to add 2 n-bit numbers.












11
How to add 2 n-bit numbers.












12
How to add 2 n-bit numbers.













13
Time complexity of grade school addition
On any reasonable computer adding 3 bits can be
done in constant time.
T(n) The amount of time grade school addition
uses to add two n-bit numbers ?(n) linear
time.
14
f ?(n) means that f can be sandwiched between
two lines
time
of bits in numbers
f ?(g(n)) means within a multiplicative
constant ? n0, c1, c2, ? n ? n0, c1 g(n) ?
f(n) ? c1 g(n)
15
Please feel free to ask questions!

• Rudich www.discretemath.com

16
Is there a faster way to add?

• QUESTION Is there an algorithm to add two n-bit
  numbers whose time grows sub-linearly in n?

17
Any algorithm for addition must read all of the
input bits

• Suppose there is a mystery algorithm that does
  not examine each bit
• Give the algorithm a pair of numbers. There must
  be some unexamined bit position i in one of the
  numbers
• If the algorithm is not correct on the numbers,
  we found a bug
• If the algorithm is correct, flip the bit at
  position i and give the algorithm the new pair of
  numbers. It give the same answer as before so it
  must be wrong since the sum has changed

18
So any algorithm for addition must use time at
least linear in the size of the numbers. Grade
school addition is essentially as good as it can
be.
19
How to multiply 2 n-bit numbers.

X










20
I get it! The total time is bounded by cn2.
Can we do it faster?
21
How to multiply 2 n-bit numbers.Kindergarten
Algorithm
T(n) Time multiply
?(b) linear time.
Fast?
10000000000000000000 100000000000000000000
22
How to multiply 2 n-bit numbers.Kindergarten
Algorithm
two n-bit numbers
T(n) Time multiply
?(b)
?(2n).
Way slow!
10000000000000000000 100000000000000000000
Value b 100000000000000000000 bits n
log2(b) 60
23
Grade School Addition Linear timeGrade School
Multiplication Quadratic time Kindergarten
Multiplication Exponential time

• No matter how dramatic the difference in the
  constants the quadratic function will eventually
  dominate the linear function

• And all exponential functions are VERY fast
  growing

24
Neat! We have demonstrated that as things scale
multiplication is a harder problem than
addition.Mathematical confirmation of our
common sense.
25
Dont jump to conclusions!We have argued that
grade school multiplication uses more time than
grade school addition. This is a comparison of
the complexity of two algorithms. To argue that
multiplication is an inherently harder problem
than addition we would have to show that no
possible multiplication algorithm runs in linear
time.
26
Grade School Addition ?(n) timeGrade School
Multiplication ?(n2) time
Is there a clever algorithm to multiply two
numbers in linear time?
27
Despite years of research, no one knows! If you
resolve this question, York will give you a PhD!
28
Is there a faster way to multiply two numbers
than the way you learned in grade school?
29
Divide And Conquer(an approach to faster
algorithms)

• DIVIDE my instance to the problem into smaller
  instances to the same problem.
• Have a friend (recursively) solve them.Do not
  worry about it yourself.
• GLUE the answers together so as to obtain the
  answer to your larger instance.

30
Multiplication of 2 n-bit numbers

• X
• Y
• X a 2n/2 b Y c 2n/2 d
• XY ac 2n (adbc) 2n/2 bd

a
b
c
d
31
Multiplication of 2 n-bit numbers

• X
• Y
• XY ac 2n (adbc) 2n/2 bd

a
b
c
d
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d RETURN
MULT(a,c) 2n (MULT(a,d) MULT(b,c)) 2n/2
MULT(b,d)
32
Time required by MULT

• T(n) time taken by MULT on two n-bit
  numbers
• What is T(n)? What is its growth rate? Is it
  ?(n2)?

33
Recurrence Relation

• T(1) k for some constant k
• T(n) 4 T(n/2) k n k for some
  constants k and k

MULT(X,Y) If X Y 1 then RETURN XY
Break X into ab and Y into cd RETURN
MULT(a,c) 2n (MULT(a,d) MULT(b,c)) 2n/2
MULT(b,d)
34
Lets be concrete

• T(1) 1
• T(n) 4 T(n/2) n
• How do we unravel T(n) so that we can determine
  its growth rate?

35
Technique 1Guess and Verify

• Recurrence Relation
• T(1) 1 T(n) 4T(n/2) n
• Guess G(n) 2n2 n
• Verify

36
Technique 2 Decorate The Tree

• T(1) 1

T(1)

1

• T(n) n 4 T(n/2)

T(n)
T(n)
n
n


T(n/2)
T(n/2)
T(n/2)
T(n/2)
T(n/2)
T(n/2)
T(n/2)
T(n/2)
37
T(n)
n

T(n/2)
T(n/2)
T(n/2)
T(n/2)
38
T(n)
n

T(n/2)
T(n/2)
T(n/2)
39
T(n)
n

40
T(n)
n

n/2
n/2
n/2
n/2
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
11111111111111111111111111111111 . . . .
... . 111111111111111111111111111111111
41
0
1
2
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
i
log2n
42
1n
4(n/2)
16(n/4)
4i (n/2i)
4logn(n/2logn)
nlog4 x 1
alogn (2loga)logn 2(loga logn)
(2logn)loga nloga
Total ?
43
Geometric Sum
44
Geometric Increasing
?i0..n ri r0 r1 r2 . . . rn
Q(biggest term)
True whenever terms increase quickly
45
Adding Made Easy
46
Adding Made Easy
47
1n
4n/2
16n/4
4i n/2i
4lognn/2logn nlog4 1
Total ?(nlog4) ?(n2)
48
Divide and Conquer MULT ?(n2) time Grade School
Multiplication ?(n2) time
All that work for nothing!
Lets understand the math better.
49
Evaluating T(n) aT(n/b)f(n)
50
Evaluating T(n) aT(n/b)f(n)
51
Evaluating T(n) aT(n/b)f(n)
52
Evaluating T(n) aT(n/b)f(n)
53
Evaluating T(n) aT(n/b)f(n)
54
Evaluating T(n) aT(n/b)f(n)
55
Evaluating T(n) aT(n/b)f(n)
56
Evaluating T(n) aT(n/b)f(n)
57
Evaluating T(n) aT(n/b)f(n)
base case
58
Evaluating T(n) aT(n/b)f(n)
59
Evaluating T(n) aT(n/b)f(n)
bh n h log b log n h log n/log b
60
Evaluating T(n) aT(n/b)f(n)
61
Evaluating T(n) aT(n/b)f(n)
62
Evaluating T(n) aT(n/b)f(n)
63
Evaluating T(n) aT(n/b)f(n)
64
Evaluating T(n) aT(n/b)f(n)
65
Evaluating T(n) aT(n/b)f(n)
log n/log b
ah a n
log a/log b
66
Evaluating T(n) aT(n/b)f(n)
67
Evaluating T(n) aT(n/b)f(n)
Total Work T(n) ?i0..h aif(n/bi)
68
Evaluating T(n) aT(n/b)f(n)
?i0..h aif(n/bi)
69
Evaluating T(n) aT(n/b)f(n)
Dominated by Top Level or Base Cases
70
Evaluating T(n) aT(n/b)f(n)
71
Divide and Conquer MULT ?(n2) time Grade School
Multiplication ?(n2) time
All that work for nothing!
Lets try to multiply faster.
72
MULT revisited
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d RETURN
MULT(a,c) 2n (MULT(a,d) MULT(b,c)) 2n/2
MULT(b,d)

• MULT calls itself 4 times. Can you see a way to
  reduce the number of calls?

73
Gauss HackInput a,b,c,d Output ac, adbc,
bd

• A1 ac
• A3 bd
• m3 (ab)(cd)
• A2 m3 A1- A3 ad bc

2 additions and one multiplication
ac ad bc bd
74
Gaussified MULT - Karatsuba 1962)
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e2n (MULT(ab, cd)
e - f) 2n/2 f

• T(n) 3 T(n/2) n
• Actually T(n) 2 T(n/2) T(n/2 1) kn

75
T(n)
n

76
T(n)
n

T(n/2)
T(n/2)
T(n/2)
77
T(n)
n

T(n/2)
T(n/2)
78
T(n)
n

79
0
1
2
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
i
log2n
80
1n
3n/2
9n/4
3i n/2i
3lognn/2logn nlog31
81
Evaluating T(n) aT(n/b)f(n)
Dominated by Top Level or Base Cases
82
Evaluating T(n) aT(n/b) nc
3T(n/2) n
n c

• Time for top level
• Time for base cases
• Dominated? c 1 lt 1.58 log a/log b

83
Dramatic improvement for large n
Not just a 25 savings! ?(n2) vs ?(n1.58..)
84
Multiplication Algorithms
343333
85
The Goal is to UNDERSTAND Recursive Algorithms

• Fundamentally to their core

86
RepresentationUnderstand the relationship
between different representations of the same
information or idea
1
2
3

• Rudich www.discretemath.com

87
Different Representationsof Recursive Algorithms
Pros
Views
88
Code Representation of an Algorithm
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
Pros and Cons?
89
Code Representation of an Algorithm
Pros
Cons

• Runs on computers
• Precise and succinct
• Perception that being able to code is the only
  thing needed to get a job. (false)

• I am not a computer
• I need a higher level of intuition.
• Prone to bugs
• Language dependent

90
Different Representationsof Recursive Algorithms
Pros
Views
91
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
Stack Frame A particular execution of one
routine on one particular input instance.
92
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
93
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
94
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
95
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
96
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
97
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
98
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
99
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
100
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
101
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
102
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
103
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
104
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
105
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
106
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
107
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
108
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
109
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
110
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
111
Stack of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
An so on .
112
Stack of Stack Frames Representation of an
Algorithm
Pros
Cons

• Trace what actually occurs in the computer
• Concrete.
• It is what students attempt to describe when told
  not to give code.

• Described in words it is impossible to follow who
  is doing what.
• Does not explain why it works.
• Demonstrates for only one of many inputs.

113
Different Representationsof Recursive Algorithms
Pros
Views
114
Tree of Stack Frames
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
115
Stack of Stack Frames Representation of an
Algorithm
Pros
Cons

• View the entire computation.
• Good for computing the running time.

• Must describe entire tree.
• For each stack frame
• input instance
• computation
• solution returned.
• who calls who
• Structure of tree may be complex.

116
Different Representationsof Recursive Algorithms
Pros
Views
117
Friends Strong Induction
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
One Friend for each stack frame. Each worries
only about his job.
118
Friends Strong Induction
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
Worry about one step at a time. Imagine that you
are one specific friend.
119
Friends Strong Induction
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f

• Consider your input instance

120
Friends Strong Induction
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f

• Consider your input instance
• Allocate work
• Construct one or more sub-instances

X 3 Y 2 XY ?
X 3 Y 1 XY ?
X 6 Y 3 XY ?
121
Friends Strong Induction
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f

• Consider your input instance
• Allocate work
• Construct one or more sub-instances
• Assume by magic your friends give you the answer
  for these.

X 3 Y 2 XY 6
X 3 Y 1 XY 3
X 6 Y 3 XY 18
122
Friends Strong Induction
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f

• Consider your input instance
• Allocate work
• Construct one or more sub-instances
• Assume by magic your friends give you the answer
  for these.
• Use this help to solve your own instance.

X 3 Y 2 XY 6
X 3 Y 1 XY 3
X 6 Y 3 XY 18
123
Friends Strong Induction
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f

• Consider your input instance
• Allocate work
• Construct one or more sub-instances
• Assume by magic your friends give you the answer
  for these.
• Use this help to solve your own instance.
• Do not worry about anything else.
• Who your boss is
• How your friends solve their instance

X 3 Y 2 XY 6
X 3 Y 1 XY 3
X 6 Y 3 XY 18
124
Friends Strong Induction
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f

• This technique is often
• referred to as
• Divide and Conquer

X 3 Y 2 XY 6
X 3 Y 1 XY 3
X 6 Y 3 XY 18
125
Friends Strong Induction
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f

• Consider generic instances.

MULT(X,Y) Break X into a,b and Y into c,d e
MULT(a,c) and f MULT(b,d) RETURN e 10n
(MULT(ab, cd) e - f) 10n/2 f
MULT(X,Y) If X Y 1 then RETURN XY

• Remember!
• Do not worry about
• Who your boss is.
• How your friends solve their instance.

bd
(ab)(cd)
ac
126
Friends Strong Induction
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
This solves the problem for every possible
instance.
127
Friends Strong Induction

• Recursive Algorithm
• Assume you have an algorithm that works.
• Use it to write an algorithm that works.

128
Friends Strong Induction

• Recursive Algorithm
• Assume you have an algorithm that works.
• Use it to write an algorithm that works.

If I could get in, I could get the key. Then I
could unlock the door so that I can get in.
129
Friends Strong Induction

• Recursive Algorithm
• Assume you have an algorithm that works.
• Use it to write an algorithm that works.

To get into my house I must get the key from a
smaller house
130
Friends Strong Induction
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f

• Allocate work
• Construct one or more sub-instances

Each sub-instance must be a smaller instance to
the same problem.
X 3 Y 2 XY ?
X 3 Y 1 XY ?
X 6 Y 3 XY ?
131
Friends Strong Induction

• Recursive Algorithm
• Assume you have an algorithm that works.
• Use it to write an algorithm that works.

132
Friends Strong Induction
MULT(X,Y) If X Y 1 then RETURN XY
Break X into a,b and Y into c,d e MULT(a,c)
and f MULT(b,d) RETURN e 10n (MULT(ab,
cd) e - f) 10n/2 f
MULT(X,Y) If X Y 1 then RETURN XY
Use brute force to solve the base case instances.
133
Friends Strong Induction
The steps to proof the program is correct are
similar to those for an iterative algorithm.
134
Friends Strong Induction
Carefully write the specifications for the
problem.
135
Friends Strong Induction
Carefully write the specifications for the
problem.

• To be sure that we solve the problem for every
  legal instance.
• So that we know
• what we can give to a friend.

Set of legal instances (inputs)
ltpreCondgt

• So that we know
• what is expected of us.
• what we can expect from our friend.

Required output
ltpostCondgt
136
Friends Strong Induction
Focus on only one step. Do not trace out the
entire computation.

• Consider your input instance
• Remember that you know nothing other than what
  is given to you by the preconditions.
• Be sure each path in your code returns what is
  required by the postcondition.

137
Friends Strong Induction
Focus on only one step. Do not trace out the
entire computation.
Consider your input instance

• Do not worry about who your boss is
• or how your friends solve their instance.
• No global variable or effects

138
Friends Strong Induction
From your instance, construct one or more
subinstances.
139
Friends Strong Induction
If your instance meets the preconditions, then
each of your subinstances also meet the
preconditions.
140
Friends Strong Induction
Prove that the subinstances that you give to you
friends are in some way smaller'' than your
instance
141
Friends Strong Induction
Prove that the subinstances that you give to you
friends are in some way smaller'' than your
instance
And your instance has a finite size.
142
Friends Strong Induction
Assume by magic (strong induction) your friends
give you a correct solution for each of these.
143
Friends Strong Induction
Use their solutions for their subinstances to
help you find a solution to your own instance.
144
Friends Strong Induction
If your friend's solutions meet the
postconditions for their subinstances, then your
solution meets the postcondition for your
instance.
145
Friends Strong Induction

• If you want more from your friends,
• change the pre and/or postconditions
• Be sure to document it.
• Be sure you and all of your friends are
  using the same pre/postconditions.
• Be sure to handle these changes yourself.

146
Friends Strong Induction
If your instance is sufficiently small, solve it
yourself as a base case.
147
Friends Strong Induction
If your instance is sufficiently small, solve it
yourself as a base case.
Your solution for the base case meets the
postconditions.
148
Friends Strong Induction
149
Induction
Friends Strong Induction
Strong Induction
150
Friends Strong Induction

• Induction Hypothesis

The recursive algorithm works for every
instance of size n.''
The algorithm works for every instance of any
size.''
size i
151
Friends Strong Induction
Give and solve the recurrence relation for the
Running Time of this algorithm.
152
Friends Strong Induction
Done
153
Friends Strong Induction Representation of an
Algorithm
Pros
Cons

• Worry about one step at a time.
• An abstraction within which to develop, think
  about, and describe algorithms in such way that
  their correctness is transparent.

• Students resist it.
• Students expect too much from their friends.
• Each sub-instance must be
• a smaller instance
• to the same problem.
• Students expect too much from their boss.
• You only know your instance. You do not know your
  bosss instance.

154
Output
Try explaining what this algorithm does by
tracing it out.
I dare you!!!
It is much easier to TAKE ONE STEP AT A TIME
155
Output
X
n1
n2
Y
156
Output
X
n1
n2
Y
A ? B ? C
n3
157
Output
X
n1
n2
Y
A Y B X C
n3
158
Output
X
n1
n2
Y
AYBXC
n3
159
Output
X
n1
n2
Y
AYBXC
n3
A ? B ? C
n4
160
Output
X
n1
n2
Y
AYBXC
n3
A AYBXC B Y C
n4
161
Output
X
n1
n2
Y
AYBXC
n3
AAYBXCBYC
n4
162
Output
X
n1
n2
Y
AYBXC
n3
AAYBXCBYC
n4
A ? B ? C
n5
163
Output
X
n1
n2
Y
AYBXC
n3
AAYBXCBYC
n4
AAAYBXCBYCBAYBXCC
n5
164
Output
X
n1
n2
Y
AYBXC
n3
AAYBXCBYC
n4
AAAYBXCBYCBAYBXCC
n5
165
1
T(2)
Time T(1)
1
T(n-1) T(n-2) 3
T(n)
? 2T(n-1) 3
2Q(n)
defined?
166
Towers of Hanoi
And I am lazy.
167
Towers of Hanoi
At some point,the biggest disk moves. I will do
that job.
168
Towers of Hanoi
To do this, the other disks must be in the
middle.
169
Towers of Hanoi
How will these move? I will get a friend to do
it. And another to move these. I only move the
big disk.
170
Towers of Hanoi
171
Towers of Hanoi
Get a job at the Towers of Hanoi company at
level n1,2,3,4,..
Or get transferred in as the president you
decide what your vice president does.
172
Towers of Hanoi
Time T(1) 1, T(n) 2(2T(n-2))
4(2T(n-3))
1 2T(n-1)
2T(n-1)
4T(n-2)
8T(n-3)
2i T(n-i)
2n
173
Evaluating T(n) aT(n-b)f(n)
Work in Level
stack frames
Work in stack frame
Instance size
n
n-b
h ?
h n/b
base case 0 n-hb h n/b
174
Evaluating T(n) 1T(n-b)f(n)
Work in Level
stack frames
Work in stack frame
Instance size
h ?
?(f(n)) or ?(nf(n))
Total Work T(n) ?i0..h f(bi)
175
Evaluating T(n) aT(n/b)f(n)
176
Check Lists for Recursive Programs
This is the format of all recursive
programs. Dont deviate from this. Or else!
177
Check Lists for Recursive Programs
This is input instance is your mission. You must
return this output. (An x, y, z in every
computation path.) Focus on your mission.
178
Check Lists for Recursive Programs
The ltpreCondgt ltpostCondgt must document these
variables and what you must do.
179
Check Lists for Recursive Programs
To get help, you construct an instance for each
friend It must be valid and smaller. And pass
them to them.
180
Check Lists for Recursive Programs
Each friend returns a solution his
sub-instance. Trust him to do this. But dont
expect him to do or know anything else.
181
Check Lists for Recursive Programs
Combine their solutionsto construct a solution
to your instance. Return your solution.
182
Check Lists for Recursive Programs
If your solution is too small to get solved by
the general techniquesolve it yourself. Be sure
you handle every legal instance.
183
Check Lists for Recursive Programs
You rarely need to change the values of these
variablebeyond their initial setting (this is
not an iterative algorithm.) Your rarely need
additional input, output, or local variable. But
it you do document them.
184
Check Lists for Recursive Programs
Have NO global variables. If you change the value
of a local variable n,neither your friends nor
your boss get that value.
185
Sorting Problem Specification

• ltpreCondgt The input is a list of n values with
  the same value possibly repeated.
• ltpostCondgt The output is a list consisting of
  the same n values in non-decreasing order.

186
Recursive Sorts

• Given list of objects to be sorted
• Split the list into two sub-lists.
• Recursively have a friend sort the two sub-lists.
  
• Combine the two sorted sub-lists into one
  entirely sorted list.

187
Four Recursive Sorts
Size of Sub-lists
n-1,1
n/2,n/2
Minimal effort splitting Lots of effort
recombining Lots of effort splittingMinimal
effort recombining
188
Merge Sort
Divide and Conquer
189
Merge Sort
Split Set into Two
190
Merge Sort
Merge two sorted lists into one
191
(No Transcript)
192
Merge Sort Sort
Time T(n)
2T(n/2) Q(n)
Q(n log(n))
Total work at any level Q(n)
levels Q(log(n))
193
Evaluating T(n) aT(n/b)f(n)
2T(n/2) Q(n)
2iQ(n/2i) Q(n)
All levels basically the same.
Total Work T(n) ?i0..h aif(n/bi)
Q(n) log(n)
194
Quick Sort
Divide and Conquer
195
Quick Sort
Partition set into two using randomly chosen
pivot
196
Quick Sort
197
Quick Sort
198
(No Transcript)
199
Quick Sort
Let pivot be the first element in the list?
14
88
98
30
31
62
23
25
79
52
200
Quick Sort
14
If the list is already sorted, then the slit is
worst case unbalanced.
201
Quick Sort
Best Time
Worst Time
Expected Time
202
Quick Sort
T(n) 2T(n/2) Q(n) Q(n log(n))
Best Time
Worst Time
Q(n2)
Expected Time
203
Quick Sort
T(n) 2T(n/2) Q(n) Q(n log(n))
Best Time
T(n) T(0) T(n-1) Q(n)
Worst Time
Q(n2)
Expected Time
T(n) T(1/3n) T(2/3n) Q(n)
Q(n log(n))
204
Quick Sort
Expected Time
T(n) T(1/3n) T(2/3n) Q(n)
Q(n log(n))
Top work Q(n)
Q(n)
2nd level
Q(n)
3rd level
levels
Q(log(n))
205
Kth Element Problem Specification

• ltpreCondgt The input is a list of n values and
  an integer k.
• ltpostCondgt The kth smallest in the list.

k3
Output 25
206
Kth Element Problem Specification
Partition set into two using randomly chosen
pivot
207
Kth Element Problem Specification
88
98
52
62
79
r5 elements on left
k3
208
Kth Element Problem Specification
88
98
52
62
79
r5 elements on left
k8
209
(No Transcript)
210
Kth Element Problem Specification
T(n)
Best Time
1 T(n/2) Q(n)
Q(n)
Worst Time
n0 1
n n/2 n/4 n/8 1
Expected Time
211
Kth Element Problem Specification
T(n)
Best Time
1 T(n/2) Q(n)
Q(n)
T(n) 1 T(n-1) Q(n)
Worst Time
nT(n-1) n(n-1)T(n-2) n(n-1)(n-2)T(n-3
) n(n-1)(n-2)(n-3)1 Q(n2)
Expected Time
212
Kth Element Problem Specification
T(n)
Best Time
1 T(n/2) Q(n)
Q(n)
T(n) 1 T(n-1) Q(n)
Worst Time
Q(n2)
T(n) 1 T(2/3n) Q(n)
Expected Time
Q(n)
213
T(N) 2T(N/2) 1 ?(N) Size log(b)
log(N)
2?(n)
N7
214
T(N) 1T(N/2) 1 ?(log(N)) Size
log(b) log(N)
?(n)
N7
215
Recursion on Trees
216
Recursion on Trees
A binary tree is - the empty tree -
a node with a right and a left sub-tree.
217
Recursion on Trees
A binary tree is - the empty tree -
a node with a right and a left sub-tree.
node
tree
tree
218
Recursion on Trees
A binary tree is - the empty tree -
a node with a right and a left sub-tree.
tree because
219
Recursion on Trees
A binary tree is - the empty tree -
a node with a right and a left sub-tree.
node
tree
tree
220
Recursion on Trees
A binary tree is - the empty tree -
a node with a right and a left sub-tree.
tree because
221
Recursion on Trees
A binary tree is - the empty tree -
a node with a right and a left sub-tree.
node
tree
tree
222
Recursion on Trees
A binary tree is - the empty tree -
a node with a right and a left sub-tree.
tree because
empty
223
Recursion on Trees
number of nodes ?
224
Recursion on Trees
number of nodes
number on left number on right 1 6 5
1 12
5
225
Recursion on Trees
number of nodes
Base Case ?
Are we done?
226
Recursion on Trees
number of nodes
Base Case ?
?
227
Recursion on Trees
number of nodes
Base Case ?
Better base case!
228
Recursion on Trees
number of nodes
Base Case ?
Does this still need to be a base case?
229
Recursion on Trees
number of nodes
number on left number on right 1 0 0
1 1
0
0
No!
230
Recursion on Trees
Most programmers dont use the empty tree as a
base case. This causes a lot more work.
0
0
231
Recursion on Trees
232
Recursion on Trees
Designing Program/Test Cases
Try same code
Same code works!
Try same code
Same code works!
233
Recursion on Trees
Time T(n)
T(left) T(right) Q(1)
Q(n)
One stack frame for each node in the tree
234
Recursion on Trees
sum of nodes ?
235
Recursion on Trees
sum of nodes
sum on left sum on right value at root
23 25 3 51
25
236
Recursion on Trees
sum of nodes ?
Base Case ?
237
Recursion on Trees
238
Recursion on Trees
Designing Program/Test Cases
239
Recursion on Trees
max of nodes ?
240
Recursion on Trees
max of nodes
max( max on left, max on right, value at
root) max( 8, 9, 3) 9
9
241
Recursion on Trees
max of nodes ?
Base Case ?
242
Recursion on Trees
3428
Sum so far is 17
Sum so far is
0
NewSum OldSum nextElement
0 3
3
243
Recursion on Trees
3423
Product so far is 72
Product so far is
1
NewProd OldProd nextElement
1 3
3
244
Recursion on Trees
True and True and False and
Conclusion so far is True
Conclusion so far is
True
NewProd OldProd and nextElement
True and True
True
245
Recursion on Trees
Max(3,4,2,3,
Max so far is 4
-
Max so far is
NewMax max(OldMax, next Element)
max( - , 3 )
3
246
Recursion on Trees
247
Recursion on Trees
height of tree ?
248
Recursion on Trees
height of tree
max(height of left,height on right) 1
max(3,4) 1 5
4
249
Recursion on Trees
height of tree
Base Case ?
250
Recursion on Trees
height of tree?
2 nodes or 1 edge
251
Recursion on Trees
Work it out backwards
0 edge
252
Recursion on Trees
Work it out backwards
0 edge
max(height of left,height on right) 1
max(-1,-1) 1
-1
-1
253
Recursion on Trees
254
Recursion on Trees
number of leaves ?
255
Recursion on Trees
number of leaves
number on left number on right 3 2 5
2
256
Recursion on Trees
number of leaves
Base Case ?
257
Recursion on Trees
Designing Program/Test Cases
Needs to be another base case.
258
Recursion on Trees
259
Recursion on Trees
260
Recursion on Trees
261
Recursion on Trees
Drops the tree.
262
Define Problem Binary Search Tree

• ltpreCondgt
• Key 25
• A binary search tree.
• ltpostCondgt
• Find key in BST (if there).

263
Binary Search Tree

• Its left children Any node Its right
  children

38

25
51

17
31
42
63
4
21
28
35
40
49
55
71
264
Define Loop Invariant

• Maintain a sub-tree.
• If the key is contained in the original tree,
  then the key is contained in the sub-tree.

key 17
265
Define Step

• Cut sub-tree in half.
• Determine which half the key would be in.
• Keep that half.

key 17
If key lt root, then key is in left half.
If key gt root, then key is in right half.
If key root, then key is found
266
Recursion on Trees
267
Recursion on Trees
268
Recursion on Trees
Time T(n)
2T(n/2) Q(
n)
Q(n log(n))
If tree very unbalanced?
1T(n-1) ?(n)
T(n)
?(n2)
Computing max is too much work.
269
Recursion on Trees
Extra info from below
270
Recursion on Trees
Time T(n)
2T(n/2) Q(
n)
Q(n log(n))
Computing max is too much work.
271
Recursion on Trees
Extra info from above
38
? max
min ?
-,
272
Things to Remember Things to Do and Things
NOT TO DO
273
I am obsessed with the Friends - Strong
Induction View of Recursion.

• Trust your friends to solve sub-instances.
• The sub-instance given must be
• smaller and
• must be an instance to the same problem.
• Combine solution given by friend to construct
  your own solution for your instance.
• Focus on one step.
• Do not talk of their friends friends friends.
• Solve small instances on your own.

274
Typical Test Answer
Define pre post conditions
275
Typical Test Answer
Call recursively
on the correct types of input
276
Typical Test Answer
Call recursively
277
Typical Test Answer
Combine solutions given by friends to construct
your own solution.
278
Typical Test Answer
Return things of the correct types.
In every path through code
279
Typical Test Answer
Sub-instancesneed to be smaller.
Size is size of the tree
Wrong base case
280
Typical Test Answer
No Global Variables.
What is the value of n here?
Still zero.The friend has his own variable n.
281
Typical Test Answer
No Global Variables.
Maybe they mean to pass the new n in and out.
Please dont use theseto pass out valueswhen I
am marking.
282
Typical Test Answer
Looks like an iterative algorithm
Looks like an recursive algorithm
Which is it?
283
Heaps, Heap Sort, Priority Queues
284
Heap Definition

• Completely Balanced Binary Tree
• The value of each node
• ³ each of the node's children.
• Left or right child could be larger.

Maximum is at root.
Where can 9 go?
Where can 1 go?
Where can 8 go?
285
Heap Data Structure
Completely Balanced Binary TreeImplemented by an
Array
286
Make Heap
Get help from friends
287
Heapify
Where is the maximum?
Maximum needs to be at root.
?
288
Heapify
Find the maximum.
Repeat
289
Heapify
Running Time
290
(No Transcript)
291
(No Transcript)
292
Make Heap
Get help from friends
Heapify
Running time
T(n) 2T(n/2) log(n)
Q(n)
293
Another algorithm
?
294
(No Transcript)
295
(No Transcript)
296
(No Transcript)
297
(No Transcript)
298
Running Time
299
Priority Queues
300
Selection Sort
Selection
Max is easier to find if a heap.
301
Heap Sort
302
Heap Data Structure
303
Heap Sort
in a heap.
Put next value where it belongs.
304
Heap Sort
305
Heap Sort
?
?
?
?
?
?
?
Sorted
306
Heap Sort
Running Time
307
Recursion on Trees
Evaluate Equation Tree ?
308
Recursion on Trees
Evaluate Equation Tree
rootop( value on left, value on right )
rootop(12,7) 12 7 19
7
12
309
Recursion on Trees
Evaluate Equation Tree
Base Case ?
310
(No Transcript)
311
Derivatives
Output Its derivative df/dx.
Input a function f.
312
Derivatives
313
Derivatives
Output Its derivative df/dx.
Input a function f.
314
Derivatives
Output Its derivative df/dx.
Input a function f.
315
Derivatives
Output Its derivative df/dx.
Input a function f.
g
316
Simplify
Input a function f.
Output f simplified.
317
Simplify
318
Recursion on Trees
Printing a Tree
When I taught this one year, a student needed
just this algorithm for his job.
319
Recursion on Trees
Typing line by line?
320
Recursion on Trees
One stack frame prints
His input gives
321
Recursion on Trees
He gives his friends
322
(No Transcript)
323
(No Transcript)
324
(No Transcript)
325
(No Transcript)
326
Parsing
Input
s68((242)(512)98771231554)
Output
327
Parsing
328
Parsing
329
Parsing
Algorithm GetExp( s, i ) Input s is a string
of tokens i is a start
index Output p is a parsing of the longest valid
expression j is the end index
s68((242)(512)98771231554)
330
Parsing
Algorithm GetTerm( s, i ) Input s is a
string of tokens i is a start
index Output p is a parsing of the longest valid
term j is the end index
s68((242)(512)98771231554)
331
Parsing
Algorithm GetFact( s, i ) Input s is a
string of tokens i is a start
index Output p is a parsing of the longest valid
factor j is the end index
s68((242)(512)98771231554)
332
Algorithm GetExp( s, i )
s68((242)(512)98771231554)
333
Algorithm GetExp( s, i )
334
Algorithm GetTerm( s, i )
s68((242)(512)98771231554)
335
Algorithm GetTerm( s, i )
336
Parsing
Algorithm GetFact( s, i )
s68((242)(512)98771231554)
337
Parsing
Algorithm GetFact( s, i )
338
Algorithm GetFact( s, i )
s68((242)(512)98771231554)
339
Algorithm GetFact( s, i )
340
Parsing
Stackframes ? nodes in parse tree
341
Recursive Images
if n1
if n0, draw
n0
else draw
And recursively Draw here with n-1
342
Recursive Images
if n2
if n0, draw
n1
else draw
And recursively Draw here with n-1
343
Recursive Images
if n3
if n0, draw
n2
else draw
And recursively Draw here with n-1
344
Recursive Images
if n30
if n0, draw
else draw
And recursively Draw here with n-1
345
Recursive Images
if n0
346
Recursive Images
if n0
347
Recursive Images
Þ
(4/3)n
L(n) 4/3 L(n-1)
348
(No Transcript)
349
(No Transcript)
350
(No Transcript)
351
(No Transcript)
352
(No Transcript)
353
Ackermanns Function
How big is A(5,5)?
354
Ackermanns Function
355
Ackermanns Function
356
Ackermanns Function
3
357
Ackermanns Function
358
Ackermanns Function
359
Ackermanns Function
360
Ackermanns Function
361
Ackermanns Function
362
Please feel free to ask questions!
Please give me feedbackso that I can better
serve you.
Thanks for the feedback that you have given me
already.
363
End
364
Cut Out
365
Hard to write an iterative program to traverse
on a recursive structure tree
Need pointers from each node to its parent.
366
Hard to write a recursive program that implements
an iterative algorithm.
367
Writing a Recursive Program
Call recursively
on the correct types of input
368
Writing a Recursive Program
Call recursively
on sub-instanceswhich will giveuseful
information.
?
node in sub-tree
369
Writing a Recursive Program
Call recursively
on sub-instanceswhich will giveuseful
information.
node in sub-tree
3rd
370
Writing a Recursive Program
Call recursively
on sub-instanceswhich will giveuseful
information.
node in sub-tree
3rd
node in whole tree
371
Writing a Recursive Program
Call recursively
on sub-instanceswhich will giveuseful
information.
node in sub-tree
3rd
nl
node in whole tree
1
3rd
372
Writing a Recursive Program
But who will count nodes inleft sub-tree?
nl
nl
373
Writing a Recursive Program
But who will count nodes inleft sub-tree?
nl
nl
374
Writing a Recursive Program
Call recursively
returning the correct types of output
Save the results (or don't bother calling)
375
Writing a Recursive Program
Combine solutions given by friends to construct
your own solution.
376
Writing a Recursive Program
Sub-instancesneed to be smaller.
Size is nodes in the tree
377
Writing a Recursive Program
Sub-instancesneed to be smaller.
When the instance sufficiently smallsolve on
your own.
378
Writing a Recursive Program
Return things of the correct types.
379
Writing a Recursive Program
Running Time
2T(n/2) ?(1)
T(n)
?(n)
Faster?
Not possible because mustcount nodes in left
tree.
380
Writing a Recursive Program
Running Time
2T(n/2) ?(n)
T(n)
?(n logn)
If tree very unbalanced?
1T(n-1) ?(n)
T(n)
?(n2)
381
End Recursive Algorithms
Math Review

• Graphs Searching Algorithms