Divide and Conquer

1
Divide and Conquer
2
Three Steps of The Divide and Conquer Approach

• The most well known algorithm design strategy
• Divide the problem into two or more smaller
  subproblems.
• Conquer the subproblems by solving them
  recursively.
• Combine the solutions to the subproblems into the
  solutions for the original problem.

3
A Typical Divide and Conquer Case
a problem of size n
subproblem 2 of size n/2
subproblem 1 of size n/2
a solution to subproblem 1
a solution to subproblem 2
a solution to the original problem
4
An Example Calculating a0 a1 an-1

• Efficiency (for n 2k)
• A(n) 2A(n/2) 1, n gt1
• A(1) 0

5
General Divide and Conquer recurrence

• The Master Theorem
• T(n) aT(n/b) f (n), where f (n) ? T(nk)
• a lt bk T(n) ? T(nk)
• a bk T(n) ? T(nk lg n )
• a gt bk T(n) ? T(nlog b a)

6
Divide and Conquer Examples

• Sorting mergesort and quicksort
• Tree traversals
• Binary search
• Matrix multiplication-Strassens algorithm

7
Mergesort

• Algorithm
• The base case n 1, the problem is naturally
  solved.
• The general case
• Divide Divide array A0..n-1 in two and make
  copies of each half in arrays B0.. n/2 - 1 and
  C0.. n/2 - 1
• Conquer Sort arrays B and C recursively using
  merge sort
• Combine Merge sorted arrays B and C into array A
  as follows (Example 1 3 7, 2 4 5 9 )
• Repeat the following until no elements remain in
  one of the arrays
• compare the first elements in the remaining
  unprocessed portions of the arrays
• copy the smaller of the two into A, while
  incrementing the index indicating the unprocessed
  portion of that array
• Once one of the arrays is exhausted, copy the
  remaining unprocessed elements from the other
  array into A.

8
The Mergesort Algorithm

• ALGORITHM Mergesort(A0..n-1)
• //Sorts array A0..n-1 by recursive mergesort
• //Input An array A0..n-1 of orderable elements
• //Output Array A0..n-1 sorted in nondecreasing
  order
• if n gt 1
• //divide
• copy A0.. n/2 1 to B0.. n/2 1
• copy A n/2 .. n 1 to C0.. n/2 1
• //conquer
• Mergesort(B0.. n/2 1)
• Mergesort(C0.. n/2 1)
• //combine
• Merge(B, C, A)

9
The Merge Algorithm

• ALGORITHM Merge(B0..p-1, C0..q-1,
  A0..pq-1)
• //Merges two sorted arrays into one sorted array
• //Input Arrays B0..p-1 and C0..q-1 both
  sorted
• //Output Sorted Array A0..pq-1 of the
  elements of B and C
• i ? 0 j ? 0 k ? 0
• while i lt p and j lt q do //while both B and C are
  not exhausted
• if Bi lt Cj //put the smaller element into
  A
• Ak ?Bi i ? i 1
• else Ak ?Cj j ? j 1
• k ? k 1
• if i p //if list B is exhausted first
• copy Cj..q-1 to Ak..pq-1
• else //list C is exhausted first
• copy Bi..p-1 to Ak..pq-1

10
Mergesort Examples

• 8 3 2 9 7 1 5 4
• 7 2 1 6 4

Efficiency?
Worst case the smaller element comes from
alternating arrays.
11
The Divide, Conquer and Combine Steps in Quicksort

• Divide Partition array Al..r into 2 subarrays,
  Al..s-1 and As1..r such that each element of
  the first array is As and each element of the
  second array is As. (computing the index of s
  is part of partition.)
• Implication As will be in its final position
  in the sorted array.
• Conquer Sort the two subarrays Al..s-1 and
  As1..r by recursive calls to quicksort
• Combine No work is needed, because As is
  already in its correct place after the partition
  is done, and the two subarrays have been sorted.

12
Quicksort

• Select a pivot w.r.t. whose value we are going
  to divide the sublist. (e.g., p Al)
• Rearrange the list so that it starts with the
  pivot followed by a sublist (a sublist whose
  elements are all smaller than or equal to the
  pivot) and a sublist (a sublist whose elements
  are all greater than or equal to the pivot ) (See
  algorithm Partition in section 4.2)
• Exchange the pivot with the last element in the
  first sublist(i.e., sublist) the pivot is now
  in its final position
• Sort the two sublists recursively using
  quicksort.

13
The Quicksort Algorithm

• ALGORITHM Quicksort(Al..r)
• //Sorts a subarray by quicksort
• //Input A subarray Al..r of A0..n-1,defined
  by its left and right indices l and r
• //Output The subarray Al..r sorted in
  nondecreasing order
• if l lt r
• s ? Partition (Al..r) // s is a split position
• Quicksort(Al..s-1)
• Quicksort(As1..r

14
The Partition Algorithm

• ALGORITHM Partition (Al ..r)
• //Partitions a subarray by using its first
  element as a pivot
• //Input A subarray Al..r of A0..n-1, defined
  by its left and right indices l and r (l lt r)
• //Output A partition of Al..r, with the split
  position returned as this functions value
• P ?Al
• i ?l j ? r 1
• Repeat
• repeat i ? i 1 until Aigtp //left-right scan
• repeat j ?j 1 until Aj lt p//right-left scan
• if (i lt j) //need to continue with the
  scan
• swap(Ai, aj)
• until i gt j //no need to scan
• swap(Al, Aj)
• return j

15
Quicksort Example

• 15 22 13 27 12 10 20 25

16
Efficiency of Quicksort

• Based on whether the partitioning is balanced.
• Best case split in the middle T( n log n)
• C(n) 2C(n/2) T(n) //2 subproblems of size
  n/2 each
• Worst case sorted array! T( n2)
• C(n) C(n-1) n1 //2 subproblems of size 0 and
  n-1 respectively
• Average case random arrays T( n log n)

17
Binary Search an Iterative Algorithm

• ALGORITHM BinarySearch(A0..n-1, K)
• //Implements nonrecursive binary search
• //Input An array A0..n-1 sorted in ascending
  order and
• // a search key K
• //Output An index of the arrays element that is
  equal to K
• // or 1 if there is no such element
• l ? 0, r ? n 1
• while l ? r do //l and r crosses over? cant
  find K.
• m ??(l r) / 2?
• if K Am return m //the key is found
• else
• if K lt Am r? m 1 //the key is on the left
  half of the array
• else l ? m 1 // the key is on the right half
  of the array
• return -1

18
Binary Search a Recursive Algorithm

• ALGORITHM BinarySearchRecur(A0..n-1, l, r, K)
• if l gt r //base case 1 l and r cross over?
  cant find K
• return 1
• else
• m ? ?(l r) / 2?
• if K Am //base case 2 K is found
• return m
• else //general case divide the problem.
• if K lt Am //the key is on the left half of
  the array
• return BinarySearchRecur(A0..n-1, l, m-1, K)
  
• else //the key is on the left half of the
  array
• return BinarySearchRecur(A0..n-1, m1, r, K)
  

19
Binary Search -- Efficiency

• What is the recurrence relation?
• Efficiency

C(n) C(?n / 2?) 2
C(n) ? T( log n)
20
Binary Tree Traversals

• Definitions
• A binary tree T is defined as a finite set of
  nodes that is either empty or consists of a root
  and two disjoint binary trees TL and TR called,
  respectively, the left and right subtree of the
  root.
• Examples.
• The height of a tree is defined as the length of
  the longest path from the root to a leaf.
• Problem find the height of a binary tree.

21
Find the Height of a Binary Tree

• ALGORITHM Height(T)
• //Computes recursively the height of a binary
  tree
• //Input A binary tree T
• //Output The height of T
• if T ?
• return 1
• else
• return maxHeight(TL), Height(TR) 1
• Base case?
• Efficiency?

C(n) n x 2n 1
22
Exercises

• The following algorithm seeks to compute the
  number of leaves in a binary tree. Is it correct?
• ALGORITHM LeafCounter(T)
• //Computes recursively the number of leaves in a
  binary tree
• //Input A binary tree T
• //Output The number of leaves in T
• if T ?
• return 0
• else
• return LeafCounter(TL) LeafCounter(TR)

23
Binary Tree Traversals preorder, inorder, and
postorder traversal

• Binary tee traversal visit all nodes of a binary
  tree recursively.
• Write a recursive preorder binary tree traversal
  algorithm.

Algorithm Preorder(T) //Implement the preorder
traversal of a binary tree //Input Binary tree T
(with labeled vertices) //Output Node labels
listed in preorder if T ? write label of Ts
root Preorder(TL) Preorder(TR)
How many recursive calls ?
24
Multiplication of Large Integers (1)

• Multiplication of two n-digit integers.
• Multiplication of a m-digit integer and a n-digit
  integer ( where n gt m) can be modeled as the
  multiplication of 2 n-digit integers (by padding
  n m 0s before the first digit of the m-digit
  integer)

25
Multiplication of Large Integers (2)

• Do we have to make n2 digit multiplications?
• n 2 we only need 3 digit multiplications.
• What about n gt 2?
• Assume n is an even number. (What about the
  situations where n is an odd number?)
• Recursively compute the product of two integers,
  with the integer size ( of digits) reduced by
  half each time.
• Efficiency of the recursive algorithm
• Recurrence Relation

26
Strassens Matrix Multiplication

• Brute-force algorithm
• c00 c01 a00 a01
  b00 b01
• 
  
• c10 c11 a10 a11
  b10 b11
• a00 b00 a01 b10 a00 b01 a01
  b11
• a10 b00 a11 b10
  a10 b01 a11 b11

8 multiplications
Efficiency class ? (n3)
4 additions
27
Strassens Matrix Multiplication

• Strassens Algorithm (two 2x2 matrices)
• c00 c01 a00 a01
  b00 b01
• 
  
• c10 c11 a10 a11
  b10 b11
• m1 m4 - m5 m7 m3 m5
• m2 m4
  m1 m3 - m2 m6
• m1 (a00 a11) (b00 b11)
• m2 (a10 a11) b00
• m3 a00 (b01 - b11)
• m4 a11 (b10 - b00)
• m5 (a00 a01) b11
• m6 (a10 - a00) (b00 b01)
• m7 (a01 - a11) (b10 b11)
  

7 multiplications
18 additions
28
Strassens Matrix Multiplication

• Two nxn matrices, where n is a power of two (What
  about the situations where n is not a power of
  two?)
• C00 C01 A00 A01
  B00 B01
• 
  
• C10 C11 A10 A11
  B10 B11
• M1 M4 - M5 M7
  M3 M5
• M2 M4
  M1 M3 - M2
  M6

29
Submatrices

• M1 (A00 A11) (B00 B11)
• M2 (A10 A11) B00
• M3 A00 (B01 - B11)
• M4 A11 (B10 - B00)
• M5 (A00 A01) B11
• M6 (A10 - A00) (B00 B01)
• M7 (A01 - A11) (B10 B11)

30
Efficiency of Strassens Algorithm

• If n is not a power of 2, matrices can be padded
  with rows and columns with zeros
• Number of multiplications
• Number of additions
• Other algorithms have improved this result, but
  are even more complex