Divide-and-Conquer

1
Divide-and-Conquer

• Matrix multiplication and Strassens algorithm

2
Matrix Multiplication

• How many operations are needed to multiply two 2
  by 2 matrices?

3
Traditional Approach

• r ae bg
• s af bh
• t ce dg
• u cf dh
• 8 multiplications and 4 additions

4
Extending to n by n matrices

• R AE BG
• S AF BH
• T CE DG
• U CF DH
• 8 multiplications of n/2 by n/2 matrices
• T(n) 8 T(n/2) Q(n2)
• T(n) Q(n3)

• Each letter represents an n/2 by n/2 matrix
• We can use the breakdown to form a divide and
  conquer algorithm

5
Example




1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

• R AE BG
• S AF BH
• T CE DG
• U CF DH

• What are A, B, , H?
• What happens at this level of the DC
  multiplication?

6
Strassens Approach

• r p5 p4 - p2 p6
• s p1 p2
• t p3 p4
• u p5 p1 p3 p7
• 7 multiplications
• 18 additions

• p1 a(f h)
• p2 (ab)h
• p3 (cd)e
• p4 d(g-e)
• p5 (ad)(e h)
• p6 (b-d)(gh)
• p7 (a-c)(ef)

7
Extending to n by n matrices

• 7 multiplications of n/2 by n/2 matrices
• 18 additions of n/2 by n/2 matrices
• T(n) 7 T(n/2) Q(n2)
• T(n) Q(nlg 7)

• Each letter represents an n/2 by n/2 matrix
• We can use the breakdown to form a divide and
  conquer algorithm

8
Example






1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

• p1 a(f h)
• p2 (ab)h
• p3 (cd)e
• p4 d(g-e)
• p5 (ad)(e h)
• p6 (b-d)(gh)
• p7 (a-c)(ef)
• r p5 p4 - p2 p6
• s p1 p2
• t p3 p4
• u p5 p1 p3 p7

• What are A, B, , H?
• What happens at this level of the DC
  multiplication?

9
Observations

• Comparison n 70
• Direct multiplication 703 343,000
• Strassen 70lg 7 is approximately 150,000
• Crossover point typically around n 20
• Hopcroft and Kerr have shown 7 multiplications
  are necessary to multiply 2 by 2 matrices
• But we can do better with larger matrices
• Current best is O(n2.376) by Coppersmith and
  Winograd, but it is not practical
• Best lower bound is W(n2) (since there are n2
  entries)
• Matrix multiplication can be used in some graph
  algorithms as a fundamental step, so theoretical
  improvements in the efficiency of this operation
  can lead to theoretical improvements in those
  algorithms