Theory of Algorithms: Divide and Conquer

1

• Theory of AlgorithmsDivide and Conquer

2
Objectives

• To introduce the divide-and-conquer mind set
• To show a variety of divide-and-conquer
  solutions
• Merge Sort
• Quick Sort
• Convex Hull by Divide-and-Conquer
• Strassens Matrix Multiplication
• To discuss the strengths and weaknesses of a
  divide-and-conquer strategy

3
Divide-and-Conquer

• Best known algorithm design strategy
• Divide instance of problem into two or more
  smaller instances
• Solve smaller instances recursively
• Obtain solution to original (larger) instance by
  combining these solutions
• Recurrence Templates apply
• Recurrences are of the form
• T(n) aT(n/b) f (n), a ? 1, b ? 2
• Silly Example (Addition)
• a0 . an-1 (a0 an/2-1) (an/2
  an-1)

4
Divide-and-Conquer Illustrated
A PROBLEM OF SIZE n
SUBPROBLEM 2 OF SIZE n/2
SUBPROBLEM 1 OF SIZE n/2
A SOLUTION TO SUBPROBLEM 1
A SOLUTION TO SUBPROBLEM 2
A SOLUTION TO THE ORIGINAL PROBLEM
5
Mergesort

• Algorithm
• Split A1..n in half and put copy of each half
  into arrays B1.. n/2 and C1.. n/2
• Recursively MergeSort arrays B and C
• Merge sorted arrays B and C into array A
• Merging
• REPEAT until no elements remain in one of B or C
• Compare 1st elements in the rest of B and C
• Copy smaller into A, incrementing index of
  corresponding array
• Once all elements in one of B or C are processed,
  copy the remaining unprocessed elements from the
  other array into A

6
Mergesort Example
7 2 1 6 4
7 2 1
6 4
7 2
1
6
4
7
2
4 6
2 7
1 2 7
1 2 4 6 7
7
Efficiency of Mergesort

• Recurrence
• C(n) 2 C(n/2) Cmerge(n) for n gt 1, C(1) 0
• Cmerge(n) n - 1 in the worst case
• All cases have same efficiency ?(n log n)
• Number of comparisons is close to theoretical
  minimum for comparison-based sorting
• log n! n lg n - 1.44 n
• Space requirement ?( n ) (NOT in-place)
• Can be implemented without recursion (bottom-up)

8
Quicksort

• Select a pivot (partitioning element)
• Rearrange the list into two sublists
• All elements positioned before the pivot are
  the pivot
• Those positioned after the pivot are gt the pivot
• Requires a pivoting algorithm
• Exchange the pivot with the last element in the
  first sublist
• The pivot is now in its final position
• QuickSort the two sublists

p
Aip
Aigtp
9
The Partition Algorithm
10
Quicksort Example
5 3 1 9 8 2 7
2 3 1
8 9 7
i j
i j
i j
5 3 1 9 8 2 7
2 1 3
8 7 9
i j
i j
i j
5 3 1 2 8 9 7
2 1 3
8 7 9
i j
j i
j i
5 3 1 2 8 9 7
1 2 3
7 8 9
j i
2 3 1 5 8 9 7
Recursive Call Quicksort (7) Quicksort (9)
Recursive Call Quicksort (1) Quicksort (3)
Recursive Call Quicksort (2 3 1) Quicksort (8 9
7)
11
Worst Case Efficiency of Quicksort

• In the worst case all splits are completely
  skewed
• For instance, an already sorted list!
• One subarray is empty, other reduced by only one
• Make n1 comparisons
• Exchange pivot with itself
• Quicksort left Ø, right A1..n-1
• Cworst (n1) n 3 (n1)(n2)/2 - 3
  ?(n2)

p
Aip
Aigtp
12
General Efficiency of Quicksort

• Efficiency Cases
• Best split in the middle ?( n log n)
• Worst sorted array! ?( n2)
• Average random arrays ?( n log n)
• Improvements (in combination 20-25 faster)
• Better pivot selection median of three
  partitioning avoids worst case in sorted files
• Switch to Insertion Sort on small subfiles
• Elimination of recursion
• Considered the method of choice for internal
  sorting for large files (n 10000)

13
Objectives

• To introduce the divide-and-conquer mind set
• To show a variety of divide-and-conquer
  solutions
• Merge Sort
• Quick Sort
• Convex Hull by Divide-and-Conquer
• Strassens Matrix Multiplication
• To discuss the strengths and weaknesses of a
  divide-and-conquer strategy

14
QuickHull Algorithm

• Sort points by increasing x-coordinate values
• Identify leftmost and rightmost extreme points P1
  and P2 (part of hull)
• Compute upper hull
• Find point Pmax that is farthest away from line
  P1P2
• Quickhull the points to the left of line P1Pmax
• Quickhull the points to the left of line PmaxP2
• Similarly compute lower hull

Pmax
L
P2
L
P1
15
Finding the Furthest Point

• Given three points in the plane p1 , p2 , p3
• Area of Triangle ? p1 p2 p3 1/2 ? D ?
• D
• ? D ? x1 y2 x3 y1 x2 y3 - x3 y2 - x2 y1 -
  x1 y3
• Properties of ? D ?
• Positive iff p3 is to the left of p1p2
• Correlates with distance of p3 from p1p2

x1 y1 1
x2 y2 1
x3 y3 1
16
Efficiency of Quickhull

• Finding point farthest away from line P1P2 is
  linear in the number of points
• This gives same efficiency as quicksort
• Worst case ?(n2)
• Average case ?(n log n)
• If an initial sort is required, this can be
  accomplished in ?( n log n) no increase in
  asymptotic efficiency class
• Alternative Divide-and-Conquer Convex Hull
• Grahams scan and DCHull
• Also ?( n log n) but with lower coefficients

17
Strassens Matrix Multiplication

• Strassen observed 1969 that the product of two
  matrices can be computed as follows
• Op Count
• Each of M1, , M7 requires 1 mult and 1 or 2
  add/sub
• Total 7 mul and 18 add/sub
• Compared with brute force which requires 8 mults
  and 4 add/sub. But is asymptotic behaviour is NB
• n/2 ? n/2 submatrices are computed recursively by
  the same method

C00 C01
C10 C11
A00 A01
A10 A11
B00 B01
B10 B11


M1 M4 -M5M7 M3 M5
M2 M4 M1 M3 - M2 M6

18
Efficiency of Strassens Algorithm

• If n is not a power of 2, matrices can be padded
  with zeros
• Number of multiplications
• M(n) 7 M(n/2) for n gt 1, M(1) 1
• Set n 2k, apply backward substitution
• M(2k) 7M(2k-1) 7 7 M(2k-2) 72 M(2k-2)
  7k
• M(n) 7log n nlog 7 n2.807
• Number of additions A(n) ? ?(nlog 7)
• Other algorithms closer to the lower limit of n2
  multiplications, but are even more complicated

19
Strengths and Weaknesses of Divide-and-Conquer

• Strengths
• Generally improves on Brute Force by one base
  efficiency class
• Easy to analyse using the Recurrence Templates
• Weaknesses
• Often requires recursion, which introduces
  overheads
• Can be inapplicable and inferior to simpler
  algorithmic solutions