Divide-and-Conquer

1
Divide-and-Conquer

• Matrix multiplication and Strassens algorithm
• Median Problem
• In general finding the kth largest element of an
  unsorted list of numbers using comparisons

2
Matrix Multiplication

• How many operations are needed to multiply two 2
  by 2 matrices?

3
Traditional Approach

• r ae bg
• s af bh
• t ce dg
• u cf dh
• 8 multiplications and 4 additions

4
Extending to n by n matrices

• R AE BG
• S AF BH
• T CE DG
• U CF DH
• 8 multiplications of n/2 by n/2 matrices
• T(n) 8 T(n/2) Q(n2)
• T(n) Q(n3)

• Each letter represents an n/2 by n/2 matrix
• We can use the breakdown to form a divide and
  conquer algorithm

5
Strassens Approach

• r p5 p4 - p2 p6
• s p1 p2
• t p3 p4
• u p5 p1 p3 p7
• 7 multiplications
• 18 additions

• p1 a(f h)
• p2 (ab)h
• p3 (cd)e
• p4 d(g-e)
• p5 (ad)(e h)
• p6 (b-d)(gh)
• p7 (a-c)(ef)

6
Extending to n by n matrices

• 7 multiplications of n/2 by n/2 matrices
• 18 additions of n/2 by n/2 matrices
• T(n) 7 T(n/2) Q(n2)
• T(n) Q(nlg 7)

• Each letter represents an n/2 by n/2 matrix
• We can use the breakdown to form a divide and
  conquer algorithm

7
Observations

• Comparison n 70
• Direct multiplication 703 343,000
• Strassen 70lg 7 is approximately 150,000
• Crossover point typically around n 20
• Hopcroft and Kerr have shown 7 multiplications
  are necessary to multiply 2 by 2 matrices
• But we can do better with larger matrices
• Current best is O(n2.376) by Coppersmith and
  Winograd, but it is not practical
• Best lower bound is W(n2) (since there are n2
  entries)
• Matrix multiplication can be used in some graph
  algorithms as a fundamental step, so theoretical
  improvements in the efficiency of this operation
  can lead to theoretical improvements in those
  algorithms

8
Median Problem

• How quickly can we find the median (or in general
  the kth largest element) of an unsorted list of
  numbers?
• Two approaches
• Quicksort partition algorithm expected Q (n) time
  but W(n2) time in the worst-case
• Deterministic Q(n) time in the worst-case

9
Quicksort Approach

• int Select(int A, k, low, high)
• Choose a pivot item
• Compare all items to this pivot element
• If pivot is kth item, return pivot
• Else update low, high, k and recurse on partition
  that contains correct item

10
Probabilistic Analysis

• Assume each of n! permutations is equally likely
• What is probability ith largest item is compared
  to jth largest item?
• If k is contained in (i..j), then 2/(j-i1)
• If k lt i, then 2/(j-k1)
• If k gt j, then 2/(k-i1)

11
Cases where (i..j) do not contain k

• kgtjS(i1 to k-1)Sj i1 to k 2/(k-i1) Si1
  to k-1 (k-i) 2/(k-i1)
• Si1 to k-1 2i/(i1) replace k-i
  with i
• 2 Si1 to k-1 i/(i1)
• lt 2(k-1)
• kltiS(jk1 to n)Si k to j-1 2/(j-k1)
  Sjk1 to n (j-k) 2/(j-k1)
• Sj 1 to n-k 2j/(j1) replace
  j-k with j and change bounds
• 2 Sj1 to n-k j/(j1)
• lt 2(n-k)
• Total for both cases is lt 2n-2

12
Case where (i..j) contains k

• At most 1 interval of size 3 contains k
• ik-1, jk1
• At most 2 intervals of size 4 contain k
• ik-1, jk2 and ik-2, j k1
• In general, at most q-2 intervals of size q
  contain k
• Thus we get S(q3 to n) (q-2)/q lt n-2
• Summing together all cases we see the expected
  number of comparisons is less than 3n

13
Best case, Worst-case

• Best case running time?
• What happens in the worst-case?
• Pivot element chosen is always what?
• This leads to comparing all possible pairs
• This leads to Q(n2) comparisons

14
Deterministic O(n) approach

• Need to guarantee a good pivot element while
  doing O(n) work to find the pivot element
• int Select(int A, k, low, high)
• Choosing pivot element
• Divide into groups of 5
• For each group of 5, find that groups median
• Find the median of the medians
• Compare remaining items directly to median to
  identify its current exact placement
• Update low, high, k and recurse on partition that
  contains correct item

15
Guarantees on the pivot element

• Median of medians is guaranteed to be smaller
  than all the red colored items
• Why?
• How many red items are there?
• ½ (n/5) 3 3n/10
• Likewise, median of medians is guaranteed to be
  larger than the 3n/10 blue colored items
• Thus median of medians is in the range 3n/10 to
  7n/10

16
Analysis of running time

• int Select(int A, k, low, high)
• Choosing pivot element
• For each group of 5, find that groups median
• Find the median of the medians
• Compare remaining items directly to median
• Recurse on correct partition

• Analysis
• Choosing pivot element
• c1 n/5
• c1 for median of 5
• Recurse on problem of size n/5
• n comparisons
• Recurse on problem of size 7n/10
• T(n) T(7n/10) T(n/5) O(n)

17
Solving recurrence relation

• T(n) T(7n/10) T(n/5) O(n)
• Key observation 7/10 1/5 9/10 lt 1
• Prove T(n) lt cn for some constant n by induction
  on n
• T(n) 7cn/10 cn/5 dn
• 9cn/10 dn
• Need 9cn/10 dn lt cn
• Thus c/10 gt d ? c gt 10d