CS 450 Design

1
CS 450 Design Analysis of AlgorithmsNotes 5

• Fall 2009Sarah Mocas

2
Heap Sort

• Outline Chapter 6
• Heaps
• HeapSort
• Priority Qs

3
Heap Sort

• Heaps
• A binary tree
• Nodes correspond to (are) elements in the array
• Tree is completely filled except perhaps the
  lowest level
• Obeys Heap Condition key in each node is larger
  (or equal to) child keys

4
Heap Sort 2 Types of Heaps

• A max-heap has the property that for every node i
  other than the root
• AParent(i) ? Ai
• A min-heap has the property that for every node i
  other than the root
• AParent(i) ? Ai
• The height of a node is the number of edges on
  the longest simple downward path from that node
  to a leaf

5
A Heap
16
14
10
7
9
8
3
2
1
4
6
Heapsort Internal Accessors

• Left(i)
• return 2i
• Right(i)
• return 2i1

• Parent(i)
• return?i/2?

7
A Heap As An Array
Height of a node is the number of edges on
longest downward path. Height of a heap is lgn.
8
Heap Sort Main Procedures

• MAX-HEAPIFY used to maintain the max-heap
  property
• BUILD-MAX-HEAP produces a max-heap from an
  unordered array
• HEAPSORT sorts an array in place

9
Max-Heapify

• MAX-HEAPIFY(A,i) Left(i) and Right(i) heaps
• l ? Left(i)
• r ? Right(i)
• if l ? heap-sizeA and Al gt Ai
• then largest ? l
• else largest ? i
• If r ? heap-sizeA and Ar gt Alargest
• then largest ? r
• if largest ? i
• then exchange Ai ? Alargest
• MAX-HEAPIFY(A,largest)

10
Heapsort

• Running time of MAX-HEAPIFY is just
• T(n) ? T(2n/3) T(1)
• Where T(1) describes the time to fix up the
  relationship among the elements Ai, ALeft(i),
  ARight(i),
• T(2n/3) arises because the procedure must call
  MAX-HEAPIFY for the subtree rooted at one of the
  children of node i and childrens subtrees each
  have size at most 2n/3. The worst case occurs
  when the last row is exactly ½ full.
• By the master theorem, T(n) O(lg n)

a1, b3/2, f(n) T(1) so Case 2 f(n)
T(nlogba) ? T(n) T(nlogba lg n) T(n0lgn)
T(lgn)
11
Building a Heap

• We use MAX-HEAPIFY in a bottom-up way to convert
  an array A1..n, where n lengthA, into a
  max-heap.
• Note that A(?n/2? 1)..n are all leaves
• BUILD-MAX-HEAP goes through the non-leave nodes
  of the tree and runs MAX-HEAPIFY on each one.

12
Build-Max-Heap

• BUILD-MAX-HEAP(A)
• heap-sizeA ? lengthA
• for i ? lengthA/2 downto 1
• do MAX-HEAPIFY(A,i)

13
BUILD-MAX-HEAP loop invariants

• Initialization prior to the first iteration of
  the loop, i ?n/2? . Each node ?n/2? 1, ?n/2?
  2, ,n is a leaf and at the root of a trivial
  max-heap.
• Maintenance the children of node i are numbered
  higher than i. They are both roots of max-heaps.
  
• Termination At termination, i 0. By the loop
  invariant, each node 1,2,,n is the root of a
  max-heap.

14
A
1
15
Heapsort BUILD-MAX-HEAP

• A loose bound on BUILD-MAX-HEAP is argued as
  follows Each call to MAX-HEAPIFY costs O(lg n)
  time and there are just O(n) of these calls.
  Thus, the running time is O(n lg n)
• A tighter bound is derived in the text but
  essentially has the result that we can build a
  max-heap from an unordered array in linear time
  O(n).

16
Heap Sort

• HEAPSORT(A)
• BUILD-MAX-HEAP(A)
• for i ? lengthA downto 2
• do exchange A1 ? Ai
  heap-sizeA ? heap-sizeA -1
• MAX-HEAPIFY(A,1)

17
The Heapsort algorithm

• Use BUILD-MAX-HEAP to build a max-heap on the
  input array A1n, where n is just lengthA.
• Since the maximum element in the array is stored
  at the root A1 it is easy to put it in its
  correct final position by just exchanging with
  An.
• Now discard node n from the tree.

18
The Heapsort algorithm(cont)

• Now we use MAX-HEAPIFY to build a heap 1 smaller
  than before and keep doing this process until we
  are down to 2.
• This takes O(n lg n) time since the call to
  BUILD-MAX-HEAP takes O(n) and each of the n-1
  calls to MAX-HEAPIFY take O(lg n)

19
Heapsort
20
Heapsort
8
10
10
10
21
Heapsort
(k)
22
Heap Sort Sub-Procedures

• MAX-HEAPIFY O(lg n)BUILD-MAX-HEAP O(n)
• HEAPSORT O(n lg n)

23
Heapsort Applicationpriority queue

• A priority queue is a data structure for
  maintaining a partial-ordered set S of elements
• A max-priority queue supports the following
• MAXIMUM(S) returns the element of S with the
  largest key.
• EXTRACT-MAX(S) removes and returns the element of
  S with the largest key.
• INSERT(S,x) inserts the element x in set S. We
  could write as S ? S ? x.
• INCREASE-KEY(S,x,k) increases the value of
  element xs value to k, k ? current key value.

24
Heapsort Applicationpriority queue

• A common application is scheduling jobs on a
  shared computer. The queue keeps track of jobs
  to performed and their relative priorities.
• A min-priority queue is the mirror image of the
  max-priority queue.

25
Priority Queue Operations

• MAXIMUM(A)
• return A1
• EXTRACT-MAX(A)
• if heap-sizeA lt 1 then error heap underflow
  
• max ? A1
• A1 ? Aheap-sizeA
• heap-sizeA ? heap-sizeA 1
• MAX-HEAPIFY(A,1)
• return max

26
Priority Queue Operations

• INCREASE-KEY(A,i,key)
• If key lt Ai then error new key is lt current
  key
• Ai ? key
• while i gt 1 and AParent(i) lt Ai
• do exchange Ai ? AParent(i) i ?
  Parent(i)
• INSERT(A, key)heap-sizeA ? heap-sizeA
  1Aheap-sizeA ? - infinityINCREASE-KEY( A,
  heap-sizeA, key )

27
HEAP-INCREASE-KEY
16
16
3
2
3
2
14
10
(b)
14
10
(a)
5
4
6
5
4
6
7
3
8
9
7
3
8
9
10
9
10
9
1
2
15
1
2
4
i
i
Pri15
16
16
3
2
i
3
2
15
10
(d)
14
10
(c)
5
4
6
5
4
6
i
7
14
3
9
7
3
9
15
10
9
10
9
1
2
8
2
1
8
28
Priority Queue Operations

• Maximum O(1)
• Extract-Max O(lg n)
• Increase-Key O(lg n)
• Insert O(lg n)

29
Exercises

• Give an O(n lg k)-time algorithm to merge k
  sorted lists into one sorted lists where n is the
  total number of elements in all lists.

n/k
n/k
n
n/k
30
Incremental Heap Construction

• Consider
• INCR-BUILD-MAX-HEAP(A)
• heap-sizeA ? 1for i ? 2 to lengthA
• do MAX-HEAP-INSERT(A, A i )
• What is the upper bound runtime for this?

31
Merging Heaps

• All you can do is concatenate the arrays and run
  BUILD-MAX-HEAP at a cost of O(n).
• Other heap structures such as Binomial Heaps and
  Fibonacci Heaps can do merge faster.

32
D-ary Heaps (d children)
16
4
8

14

• Parent(i)
• return ?(i-2) / d 1?
• kth-child (i)
• return d(i-1) k 1What is height?
• What about run-time of Extract-Max?

33
Lecture 6 Key Points

• HeapSort runs asymptotically as fast as MergeSort
  but works in-place.
• Heaps are easily stored in arrays
• Good structure for priority queues
• Merging is done in O(n) time, but other heap
  structures can do it faster.
• Fun reference Survey of known priority queue
  structures by Stefan Xenos