Lecture 6, Feb 9

1
Lecture 6, Feb 9
2
CSC373Algorithm Design and AnalysisAnnouncements

• Assignment 2, part 2 to be posted soon.

3
Divide and Conquer
4
How to multiply 2 n-bit numbers.

X










5
The total time is bounded by cn2.
6
Grade School Addition ?(n) timeGrade School
Multiplication ?(n2) time
Is there a clever algorithm to multiply two
numbers in linear time?
7
Is there a faster way to multiply two numbers
than the way you learned in grade school?
8
Divide And Conquer(an approach to faster
algorithms)

• DIVIDE a problem into smaller subproblems
• CONQUER them recursively
• GLUE the answers together so as to obtain the
  answer to the larger problem

9
Multiplication of 2 n-bit numbers

• X
• Y
• X a 2n/2 b Y c 2n/2 d
• XY ac 2n (adbc) 2n/2 bd

a
b
c
d
10
Multiplication of 2 n-bit numbers

• X
• Y
• XY ac 2n (adbc) 2n/2 bd

a
b
c
d
MULT(X,Y) If X Y 1 then RETURN XY
Break X into ab and Y into cd RETURN
MULT(a,c) 2n (MULT(a,d) MULT(b,c)) 2n/2
MULT(b,d)
11
Time required by MULT

• T(n) time taken by MULT on two n-bit
  numbers
• What is T(n)? What is its growth rate? Is it
  ?(n2)?

12
Recurrence Relation

• T(1) k for some constant k
• T(n) 4 T(n/2) k n k for some
  constants k and k

MULT(X,Y) If X Y 1 then RETURN XY
Break X into ab and Y into cd RETURN
MULT(a,c) 2n (MULT(a,d) MULT(b,c)) 2n/2
MULT(b,d)
13
Lets be concrete

• T(1) 1
• T(n) 4 T(n/2) n
• How do we unravel T(n) so that we can determine
  its growth rate?

14
Decorate The Tree

• T(1) 1

T(1)

1

• T(n) n 4 T(n/2)

• T(n) n 4 T(n/2)

T(n)
T(n)
n
n


T(n/2)
T(n/2)
T(n/2)
T(n/2)
T(n/2)
T(n/2)
T(n/2)
T(n/2)
15
T(n)
n

T(n/2)
T(n/2)
T(n/2)
T(n/2)
16
T(n)
n

T(n/2)
T(n/2)
T(n/2)
17
T(n)
n

18
T(n)
n

n/2
n/2
n/2
n/2
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
n/4
11111111111111111111111111111111 . . . . . .
111111111111111111111111111111111
19
0
1
2
i
20
1n
4n/2
16n/4
4i n/2i
4lognn/2logn nlog41
Total ?(nlog4) ?(n2)
21
Divide and Conquer MULT ?(n2) time Grade School
Multiplication ?(n2) time
All that work for nothing!
22
MULT revisited
MULT(X,Y) If X Y 1 then RETURN XY
Break X into ab and Y into cd RETURN
MULT(a,c) 2n (MULT(a,d) MULT(b,c)) 2n/2
MULT(b,d)

• MULT calls itself 4 times. Can you see a way to
  reduce the number of calls?

23
Gauss HackInput a,b,c,d Output ac, adbc,
bd

• A1 ac
• A3 bd
• m3 (ab)(cd) ac ad bc bd
• A2 m3 A1- A3 ad bc

24
Gaussified MULT(Karatsuba 1962)
MULT(X,Y) If X Y 1 then RETURN XY
Break X into ab and Y into cd e MULT(a,c)
and f MULT(b,d) RETURN e2n (MULT(ab, cd)
e - f) 2n/2 f

• T(n) 3 T(n/2) n
• Actually T(n) 2 T(n/2) T(n/2 1) kn

25
T(n)
n

26
T(n)
n

T(n/2)
T(n/2)
T(n/2)
27
T(n)
n

T(n/2)
T(n/2)
28
T(n)
n

29
0
1
2
n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4
n/4
i
30
1n
3n/2
9n/4
3i n/2i
3lognn/2logn nlog31
Total ?(nlog3) ?(n1.58..)
31
Dramatic improvement for large n
Not just a 25 savings! ?(n2) vs ?(n1.58..)
32
Multiplication Algorithms
33
Correctness proofs for divide and conquer
algorithms
34
Induction
Strong Induction
35
Strong Induction

• Induction Hypothesis

The recursive algorithm works for every
instance of size n.''
The algorithm works for every instance of any
size.''
size i
36
Math background - notations
37
Adding Made Easy
Four Classes of Functions
1
1
1
2
3
4
38

• If the terms f(i) grow sufficiently quickly, then
  the sum will be dominated by the largest term.
• Silly example
• 12345 1,000,000,000 1,000,000,000

39

• If f(n) c ban nd loge (n)
• c Î ?
• a Î ?
• b Î ?
• d Î ?
• e Î ?
• f(n) O, ? ?
• then ?i1..n f(i) ?(f(n)).

gt 0
gt 0
gt 1
(-,)
(-,)
³ 2O(n)
40
(No Transcript)
41
Adding Made Easy
Done
42

• If most of the terms f(i) have roughly the same
  value, then the sum is roughly the number of
  terms, n, times this value.
• Silly example
• 1,001 1,002 1,003 1,004 1,005
• 5 1,000

43

• If most of the terms f(i) have roughly the same
  value, then the sum is roughly the number of
  terms, n, times this value.
• Another silly example
• ?i1..n 1 n 1

44

• If most of the terms f(i) have roughly the same
  value, then the sum is roughly the number of
  terms, n, times this value.
• Is the statement true for this function?

?i1..n i 1 2 3 . . . n
45

• If most of the terms f(i) have roughly the same
  value, then the sum is roughly the number of
  terms, n, times this value.
• Is the statement true for this function?

?i1..n i 1 2 3 . . . n
The terms are not roughly the same.
46
?i1..n i 1 . . . n/2 . . . n

• But half the terms are roughly the same.

47
?i1..n i 1 . . . n/2 . . . n

• But half the terms are roughly the same
• and the sum is roughly the number
• terms, n, times this value

?i1..n i ?(n n)
48

• Is the statement true for this function?

?i1..n i2 12 22 32 . . . n2
Even though, the terms are not roughly the same.
49
?i1..n i 12 . . . (n/2)2 . . . n2
1/4 n2

• Again half the terms
• are roughly the same.

50
?i1..n i 2 12 . . . (n/2)2 . . . n2
1/4 n2

• Again half the terms
• are roughly the same.

?i1..n i2 ?(n n2)
51

• ?i1..n f(i)
• area under curve

52

• area of small square
• ?i1..n f(i)
• area under curve
• area of big square

53

• n/2 f(n/2)
• area of small square
• ?i1..n f(i)
• area under curve
• area of big square
• n f(n)

54

• ?(n f(n))
• n/2 f(n/2)
• area of small square
• ?i1..n f(i)
• area under curve
• area of big square
• n f(n)

55

• ?(n f(n))
• n/2 f(n/2)
• area of small square
• ?i1..n f(i)
• area under curve
• area of big square
• n f(n)

?

56
f(i) n2
?i1..n i2 12 22 32 . . . n2
?
57
f(i) n2
?i1..n i2 12 22 32 . . . n2
Þ ?(nf(n)) ?(n n2)

• The key property is

f(n/2) (n/2)2 1/4 n2 ?(n2) ?(f(n))
58
f(i) n?(1)
?i1..n f(i) ?

59
f(i) n?(1)
?i1..n ir 1r 2r 3r . . . nr

60
f(i) n?(1)
?i1..n ir 1r 2r 3r . . . nr
Þ ?(nf(n)) ?(n nr)

• The key property is

f(n/2) (n/2)r 1/2r nr ?(nr) ?(f(n))
61
f(i) n?(1)
?i1..n 2i 21 22 23 . . . 2n
62
f(i) n?(1)
?i1..n 2i 21 22 23 . . . 2n

• The key property is

f(n/2) 2(n/2) ?(2n) ?(f(n))
63
?i1..n i1000 1/1001 n1001
1/1001 n f(n)
Upper Extreme
All functions in between.
?i1..n 1 n 1 n f(n)
Middle Extreme
64
Adding Made Easy
Half done
65
Solving the recurrenceT(n) aT(n/b) f(n)
66
Decorate The Tree

• T(1) 1

T(1)

1

• T(n) a T(n/b) f(n)

T(n)
f(n)

a
T(n/b)
T(n/b)
T(n/b)
T(n/b)
67
T(n)
f(n)

a
T(n/b)
T(n/b)
T(n/b)
T(n/b)
68
T(n)
f(n)

a
T(n/b)
T(n/b)
T(n/b)
69
T(n)
f(n)

a
a
70
T(n)
f(n)

a
a
11111111111111111111111111111111 . . . . . .
111111111111111111111111111111111
71
Evaluating T(n) aT(n/b)f(n)
72
Evaluating T(n) aT(n/b)f(n)
73
Evaluating T(n) aT(n/b)f(n)
74
Evaluating T(n) aT(n/b)f(n)
75
Evaluating T(n) aT(n/b)f(n)
76
Evaluating T(n) aT(n/b)f(n)
77
Evaluating T(n) aT(n/b)f(n)
78
Evaluating T(n) aT(n/b)f(n)
base case
79
Evaluating T(n) aT(n/b)f(n)
80
Evaluating T(n) aT(n/b)f(n)
bh n h log b log n h log n/log b
81
Evaluating T(n) aT(n/b)f(n)
82
Evaluating T(n) aT(n/b)f(n)
83
Evaluating T(n) aT(n/b)f(n)
84
Evaluating T(n) aT(n/b)f(n)
85
Evaluating T(n) aT(n/b)f(n)
86
Evaluating T(n) aT(n/b)f(n)
log n/log b
ah a n
log a/log b
87
Evaluating T(n) aT(n/b)f(n)
88
Evaluating T(n) aT(n/b)f(n)
Total Work T(n) ?i0..h aif(n/bi)
89
Evaluating T(n) aT(n/b)f(n)
Most of the time dominated by the bigger of the
top level f(n) and the bottom level ?(n
).
log a/log b
90
Evaluating T(n) aT(n/b)f(n)
?i0..h aif(n/bi)
91
Evaluating T(n) aT(n/b)f(n)
Dominated by Top Level or Base Cases
92
Evaluating T(n) aT(n/b)f(n)

• Is the sum Geometric?
• Simplify by letting f(n) nc logkn
• T(n) ?i0..h aif(n/bi)
• ?i0..h ai(n/bi)c logk(n/bi)
• nc ?i0..h 2log a - c log b i
  logk(n/bi)

(n/..)c nc
ai 2log ai
(../bi)c 2-c log bi
nc ?i0..h 2-d i logk(n/bi)
93
Evaluating T(n) aT(n/b)f(n)

• Is the sum Geometric?
• Simplify by letting f(n) nc logkn
• T(n) ?i0..h aif(n/bi)
• ?i0..h ai(n/bi)c logk(n/bi)
• nc ?i0..h 2log a - c log b i
  logk(n/bi)

k-1 Þ logk(n/bi) 1/(logn - i logb) 1/i
(backwards)
nc ?i0..h 20 i logk(n/bi)
94
Evaluating T(n) aT(n/b)f(n)

• Is the sum Geometric?
• Simplify by letting f(n) nc logkn
• T(n) ?i0..h aif(n/bi)
• ?i0..h ai(n/bi)c logk(n/bi)
• nc ?i0..h 2log a - c log b i
  logk(n/bi)

nc ?i0..h 2d i logk(n/bi)
95
Evaluating T(n) 4T(n/2) n3/log5n
96
Evaluating T(n) 4T(n/2) n3/log5n

• Time for top level?

97
Evaluating T(n) 4T(n/2) n3/log5n

• Time for top level f(n) n3/log5n
• Time for base cases?

98
Evaluating T(n) 4T(n/2) n3/log5n

• Time for top level f(n) n3 /log5n
• Time for base cases

log ?/log ?
log a/log b
?(n )
?(n )
99
Evaluating T(n) 4T(n/2) n3/log5n

• Time for top level f(n) n3 /log5n
• Time for base cases

log 4/log 2
log a/log b
?(n ) ?
?(n )
100
Evaluating T(n) 4T(n/2) n3/log5n

• Time for top level f(n) n3/log5n
• Time for base cases
• Dominated? c ?

log 4/log 2
log a/log b
?(n ) ?(n2)
?(n )
log a/log b ?
101
Evaluating T(n) 4T(n/2) n3/log5n

• Time for top level f(n) n3/log5n
• Time for base cases
• Dominated? c 3 gt 2 log a/log b

log 4/log 2
log a/log b
?(n ) ?(n2)
?(n )
Hence, T(n) ?
102
Evaluating T(n) 4T(n/2) n3/log5n

• Time for top level f(n) n3/log5n
• Time for base cases
• Dominated? c 3 gt 2 log a/log b

log 4/log 2
log a/log b
?(n ) ?(n2)
?(n )
Hence, T(n) ?(top) ?(f(n)) ?(n3/log5n).
103
Evaluating T(n) 4T(n/2) 2n
bigger

• Time for top level f(n) 2n
• Time for base cases
• Dominated? c ? gt 2 log a/log b

log 4/log 2
log a/log b
?(n ) ?(n2)
?(n )
Hence, T(n) ?(top) ?(f(n)) ?(2n).
104
Evaluating T(n) 4T(n/2) n log5n

• Time for top level f(n) n log5n
• Time for base cases
• Dominated? c ? 2 log a/log b

log 4/log 2
log a/log b
?(n ) ?(n2)
?(n )
105
Evaluating T(n) 4T(n/2) log5n

• Time for top level f(n) log5n
• Time for base cases
• Dominated? c ? 2 log a/log b

log 4/log 2
log a/log b
?(n ) ?(n2)
?(n )
106
Evaluating T(n) 4T(n/2) n2

• Time for top level f(n) n2
• Time for base cases
• Dominated? c ? 2 log a/log b

log 4/log 2
log a/log b
?(n ) ?(n2)
?(n )
?(f(n) log(n) ) ?(n2 log(n)).
107
Evaluating T(n) 4T(n/2) n2 log5n

• Time for top level f(n) n2 log5n
• Time for base cases
• Dominated? c 2 2 log a/log b

log 4/log 2
log a/log b
?(n ) ?(n2)
?(n )
?(f(n) log(n) ) ?(n2 log6(n)).
108
Evaluating T(n) 4T(n/2) n2/log5n

• Time for top level f(n) n2/log5n
• Time for base cases
• Dominated? c 2 2 log a/log b

log 4/log 2
log a/log b
?(n ) ?(n2)
?(n )
k ?
109
Evaluating T(n) 4T(n/2) n2/log n

• Time for top level f(n) n2/log n
• Time for base cases
• Dominated? c 2 2 log a/log b

log 4/log 2
log a/log b
?(n ) ?(n2)
?(n )
k ?
?(nc loglog n) ?(n2 loglog n). Did not do
before.
110
Evaluating T2(n) 4 T2(n/2n1/2) n3

• Sufficiently close to T(n) 4T(n/2) n3
  ?(n3). T2(n) ?

?(n3).
111
Recursive Images
if n1
if n0, draw
n0
else draw
And recursively Draw here with n-1
112
Recursive Images
if n2
if n0, draw
n1
else draw
And recursively Draw here with n-1
113
Recursive Images
if n3
if n0, draw
n2
else draw
And recursively Draw here with n-1
114
Recursive Images
if n30
if n0, draw
else draw
And recursively Draw here with n-1
115
Recursive Images
if n0
116
Recursive Images
if n0
117
Recursive Images
Þ
(4/3)n
L(n) 4/3 L(n-1)