CS 312: Algorithm Analysis

1
CS 312 Algorithm Analysis

• Lecture 11 Median and Matrix Multiplication

2
Thought

• "Patience is tied very closely to faith in our
  Heavenly Father. Actually, when we are unduly
  impatient, we are suggesting that we know what is
  best -- better than does God. Or, at least, we
  are asserting that our timetable is better than
  His.
• -- Elder Neal A. Maxwell, Patience, Ensign,
  Oct. 1980, p. 28.

3
From Dr. Chuck Knutson Today

• http//money.cnn.com/galleries/2007/news/0702/gall
  ery.jobs_in_demand/6.html
• It's a brief story about 10 jobs in really high
  demand with low supply and high salaries,
• and numbers 7 and 8 are SQL Database Admins, and
  Java and .NET programmers.
• This is the sort of stuff that we should be
  showing to our undergrads to help them stay
  committed, and to non-majors to help persuade
  them to give CS a shot.
• The dollar ranges are 75-85K in small markets,
  100K in large markets.

4
Follow-up from Dr. Kent Seamons

• And a related URL shows the top 10 best jobs.
• http//money.cnn.com/magazines/moneymag/bestjobs/i
  ndex.html
• 1 is software engineer.
• 2 is college professor.
• Hmmm, what do some of our students know that we
  don't -)

5
Announcements

• HW 9 Due Today
• Questions?
• Project 1
• Questions?
• Early Day is Today
• Due Date is Monday (2/5)

6
Objectives

• Lecture 11 Median Matrix Multiplication
• Design a (expected) linear-time algorithm to
  select the k-th smallest element in a list
• Find the Median
• Apply divide and conquer to matrix multiplication
• Analyze using the Master Theorem

• Lecture 10 Mergesort, Quicksort, RQuicksort
• Review Mergesort (quickly!)
• Apply the Master Theorem to solve recurrences
• Revisit Quicksort
• Theoretical analysis
• Worst case
• Average case
• Empirical analysis
• Compare quick and merge sort
• Compare randomized Quicksort

7
The Selection Problem

• Input
• S1..n, an array of numbers
• An integer k, 1ltkltn
• Output
• The k-th smallest element of S
• i.e., the element in the k-th position if S were
  sorted in non-decreasing order

8
Median Problem

• Can find the median by
• Sorting
• Then extracting the \floor(n/2)-th element
• What was wrong with this?
• How do you find the median using selection?

9
Divide and Conquer Solution

• S 2, 36, 5, 21, 8, 13, 11, 20, 5, 4, 1
• Let v5
• Compute 3-way split
• SL 2, 4, 1
• Sv 5, 5
• SR 36, 21, 8, 13, 11, 20

10
Divide and Conquer Solution

• Shrink the number of elements on each recursive
  step
• From S
• To max (SL, SR)
• Goal
• Choose v quickly
• Ideally so that SL, SR are about ½ S
• What would the running time be?

11
Randomized DC

• Pick v randomly from S
• Efficiency Analysis
• Worst-case bad luck picks end points each time
• Result?
• Best-case good luck picks the k-th item the
  first time
• Result?
• Average case?

12
Average Case Hints
13
Median

• What are the implications?

14
Matrix Multiplication

• Who cares?
• In this setting, , , and x are NOT elementary
  operations.
• What about commutativity?
• Naïve algorithm
• O(n3) worst case
• Better solution
• obtained in a manner similar to the Karatsuba
  Divide and Conquer algorithm for scalar
  multiplication

15
Naïve Algorithm
5 6 7 8
1 2 3 4
(1x5 2x7) (1x6 2x8) (3x5 4x7) (3x6 4x8)

x
19 22 43 50

O(n3)
16
Strassens Algorithm
e f g h
m2m3 m1m2m5m6 m1m2m4-m7
m1m2m4m5
a b c d

x
m1 (c d - a) x (h f e) m2 (a x e) m3
(b x g) m4 (a - c) x (h - f) m5 (c d) x (f
- e) m6 (b - c a - d) x h m7 d x (e h f
- g)
17
Strassens on our Example
5 6 7 8
m2m3 m1m2m5m6 m1m2m4-m7
m1m2m4m5
1 2 3 4

x
5 14 4257(-32) 425(-4)-0
425(-4)7
19 22 43 50


m1 (34-1) x (8-65) 6 x 7 42 m2 (1 x
5) 5 m3 (2 x 7) 14 m4 (1-3) x (8-6)
-2 x 2 -4 m5 (34) x (6-5) 7 x 1
7 m6 (2-3 1-4) x 8 -4 x 8
-32 m7 4 x (58-6-7) 4x0 0
18
Recursive Application
19
Recursive Application
20
Recursive Application
A B
E F
G H
C D
Divide each matrix into fourths, and apply
Strassens algorithm. Whats the efficiency?
21
Again Strassens Algorithm
A B
E F
M2M3 M1M2M5M6 M1M2M4-M7
M1M2M4M5

x
C D
G H
M1 (C D - A) x (H F E) M2 (A x E) M3
(B x G) M4 (A - C) x (H - F) M5 (C D) x (F
- E) M6 (B - C A - D) x H M7 D x (E H F
- G)
Recall Matrix addition (and subtraction) is
O(n2)
22
Efficiency Analysis
a number of subinstances n original instance
size n/b size of subinstances d polynomial
order of g(n) where g(n) cost of doing the
divide
23
Efficiency Analysis
a 7 n original instance size (width of left
operand) n/b n/2 d 2 (Cost of doing the
division and recombination) Therefore T(nlg7),
or T(n2.81)
24
Is it correct?
What does it mean for Strassens to be
correct? How can/will you prove it in general?
25
Correctness
e f g h
m2m3 m1m2m5m6 m1m2m4-m7
m1m2m4m5
a b c d

x
m1 (c d - a) x (h f e) m2 (a x e) m3
(b x g) m4 (a - c) x (h - f) m5 (c d) x (f
- e) m6 (b - c a - d) x h m7 d x (e h f
- g)
e.g., Lower left entry c d x e gT c x e
d x g ? m1 m2 m4 m7 (c d a) x (h
f e) a x e (a c) x (h f) d x (e h
f g) c x h c x f c x e d x h d x f
d x e a x h a x f a x e a x e a x h
a x f c x h c x f d x e d x h d x f
d x g c x e d x g Do likewise for all four
entries in the result
26
Decimal Wars

• Cant get fewer than 7 multiplications on a 2x2
  matrix
• No fewer than 21 on 3x3
• Pan 70x70 in 143,640 multiplications
• Strassen O(n2.81)
• 1979 O(n2.521813)
• 1980 O(n2.521801)
• 1986 O(n2.376)

27
Simplification vs. D/C
28
Assignment

• Homework None
• Read Section 2.6 on the FFT!