Heaps, HeapSort,

1
Heaps, HeapSort, Priority Queues

• Briana B. Morrison
• Adapted from Alan Eugenio,
• William J. Collins, Michael Main

2
Topics

• Heaps
• Implementation
• Insertion
• Deletion
• Applications
• Priority Queue
• HeapSort

3
Heaps

• A heap is a certain kind of complete binary tree.

4
Heaps
Root

• A heap is a certain kind of complete binary tree.

When a complete binary tree is built, its first
node must be the root.
5
Heaps

• Complete binary tree.

Left child of the root
The second node is always the left child of the
root.
6
Heaps

• Complete binary tree.

Right child of the root
The third node is always the right child of the
root.
7
Heaps

• Complete binary tree.

The next nodes always fill the next level from
left-to-right.
8
Heaps

• Complete binary tree.

The next nodes always fill the next level from
left-to-right.
9
Heaps

• Complete binary tree.

10
Heaps
45

• A heap is a certain kind of complete binary tree.

23
35
4
22
21
27
19
Each node in a heap contains a key that can be
compared to other nodes' keys.
11
Heaps
45

• A heap is a certain kind of complete binary tree.

23
35
4
22
21
27
19
- Max heap requires gt Min heap requires lt
The "heap property" requires that each node's key
is gt the keys of its children
12
What is a heap? (7.3.1)

• A heap is a binary tree storing keys at its
  internal nodes and satisfying the following
  properties
• Heap-Order for every internal node v other than
  the root,key(v) ? key(parent(v))
• Complete Binary Tree let h be the height of the
  heap
• for i 0, , h - 1, there are 2i nodes of depth
  i
• at depth h - 1, the internal nodes are to the
  left of the external nodes

• The last node of a heap is the rightmost internal
  node of depth h - 1

2
6
5
7
9
last node
13
Maximum and Minimum Heaps
14
(No Transcript)
15
Height of a Heap

• Theorem A heap storing n keys has height O(log
  n)
• Proof (we apply the complete binary tree
  property)
• Let h be the height of a heap storing n keys
• Since there are 2i keys at depth i 0, , h - 2
  and at least one key at depth h - 1, we have n ?
  1 2 4 2h-2 1
• Thus, n ? 2h-1 , i.e., h ? log n 1

keys
depth
1
0
2
1
2h-2
h-2
h-1
1
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
Complete Binary Tree for a Vector
20
(No Transcript)
21
Insertion Into A Heap
22
Adding a Node to a Heap
45

• Put the new node in the next available spot.
• Push the new node upward, swapping with its
  parent until the new node reaches an acceptable
  location.

23
35
4
22
21
27
19
42
23
Adding a Node to a Heap
45

• Put the new node in the next available spot.
• Push the new node upward, swapping with its
  parent until the new node reaches an acceptable
  location.

23
35
4
22
21
42
19
27
24
Adding a Node to a Heap
45

• Put the new node in the next available spot.
• Push the new node upward, swapping with its
  parent until the new node reaches an acceptable
  location.

23
42
4
22
21
35
19
27
25
Adding a Node to a Heap
45

• The parent has a key that is gt new node, or
• The node reaches the root.
• The process of pushing the new node upward is
  called trickle up, or
  reheapification upward.

23
42
4
22
21
35
19
27
26
Insertion into a Heap (7.3.2)

• Method insertItem of the priority queue ADT
  corresponds to the insertion of a key k to the
  heap
• The insertion algorithm consists of three steps
• Find the insertion node z (the new last node)
• Store k at z and expand z into an internal node
• Restore the heap-order property (discussed next)

z
insertion node
2
6
5
z
7
9
1
27
Upheap

• After the insertion of a new key k, the
  heap-order property may be violated
• Algorithm upheap restores the heap-order property
  by swapping k along an upward path from the
  insertion node
• Upheap terminates when the key k reaches the root
  or a node whose parent has a key smaller than or
  equal to k
• Since a heap has height O(log n), upheap runs in
  O(log n) time

2
1
1
5
2
5
z
z
7
9
6
7
9
6
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
Example of Heap Before and After Insertion of 50
38
Reorder the tree after Insertion
39
(No Transcript)
40
Deletion From A Heap
The Top Item is Always Deleted.Known as a pop
operation.
41
Popping from the Heap
45

• Move the last node onto the root.

23
42
4
22
21
35
19
27
42
Popping from the Heap
27

• Move the last node onto the root.

23
42
4
22
21
35
19
43
Popping from the Heap
27

• Move the last node onto the root.
• Push the out-of-place node downward, swapping
  with its larger child until the new node reaches
  an acceptable location.

23
42
4
22
21
35
19
44
Popping from the Heap
42

• Move the last node onto the root.
• Push the out-of-place node downward, swapping
  with its larger child until the new node reaches
  an acceptable location.

23
27
4
22
21
35
19
45
Popping from the Heap
42

• Move the last node onto the root.
• Push the out-of-place node downward, swapping
  with its larger child until the new node reaches
  an acceptable location.

23
35
4
22
21
27
19
46
Popping from the Heap
42

• The children all have keys lt the out-of-place
  node, or
• The node reaches the leaf.
• The process of pushing the new node downward is
  called trickle down, or
  reheapification downward.

23
35
4
22
21
27
19
47
Removal from a Heap (7.3.2)

• Method removeMin of the priority queue ADT
  corresponds to the removal of the root key from
  the heap
• The removal algorithm consists of three steps
• Replace the root key with the key of the last
  node w
• Compress w and its children into a leaf
• Restore the heap-order property (discussed next)

w
last node
7
6
5
w
9
48
Downheap

• After replacing the root key with the key k of
  the last node, the heap-order property may be
  violated
• Algorithm downheap restores the heap-order
  property by swapping key k along a downward path
  from the root
• Upheap terminates when key k reaches a leaf or a
  node whose children have keys greater than or
  equal to k
• Since a heap has height O(log n), downheap runs
  in O(log n) time

7
6
5
w
9
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
(No Transcript)
58
Example of Pop
59
Example of Pop
60
Application 1

• Priority Queues

61
A PRIORITY QUEUE IS A CONTAINER IN WHICH ACCESS
OR DELETION IS OF THE HIGHEST-PRIORITY
ITEM, ACCORDING TO SOME WAY OF ASSIGNING
PRIORITIES TO ITEMS.
62
Priority Queue Implementation

• We have looked at several ways to implement a
  priority queue
• Unsorted container
• Sorted container
• Vector of queues
• It also could be implemented using a heap
• How would it be implemented
• How do the priority queue functions relate to
  heap functions?
• What is the performance?

63
Heaps and Priority Queues

• We can use a heap to implement a priority queue
• Insertion into the priority queue is an insertion
  into the heap
• Removal from the priority queue is a pop from the
  heap

(2, Sue)
(6, Mark)
(5, Pat)
(9, Jeff)
(7, Anna)
64
Heaps the STL

• The STL provides 4 algorithms that pertain to
  heaps
• make_heap
• pop_heap
• push_heap
• sort_heap

65

• make_heap (first, last) constructs a heap from
  values in a given container (heapify)
• push_heap (first, last) assumes value to be
  inserted has been pushed onto the end of the
  container does trickle up (assumes you already
  have a valid heap)
• pop_heap (first, last) given a valid heap,
  swaps the first value with last value and
  reconstructs a valid heap (trickle down)
• sort_heap (first, last) sorts the elements in a
  valid heap

66
(No Transcript)
67
(No Transcript)
68
(No Transcript)
69
(No Transcript)
70
Heap-Sort (7.3.4)

• Consider a priority queue with n items
  implemented by means of a heap
• the space used is O(n)
• methods push and pop take O(log n) time
• methods size, empty, top take time O(1) time

• Using a heap-based priority queue, we can sort a
  sequence of n elements in O(n log n) time
• The resulting algorithm is called heap-sort
• Heap-sort is much faster than quadratic sorting
  algorithms, such as insertion-sort and
  selection-sort

71
Application 2
72
Application 1 HeapSort

• Implementing HeapSort requires 2 steps
• Heapify the data (convert data into a heap)
• Keep popping from the Heap until all are sorted.
  Puts top value of heap into the end of the
  container.

73

• Now we have to heapify the structure.
• We begin with the first non-leaf node (from the
  end of the tree).
• How do we calculate this?
• Parent of last node!

74
Example of Heapifying a Vector
Starts at 1st non-leaf node. trickles down if
needed. Then proceed to next item (i 1) in
container.
75
Example of Heapifying a Vector (Cont)
76
Example of Implementing heap sort
int arr 50, 20, 75, 35, 25 vectorltintgt
h(arr, 5)
Original Tree
50
20
75
35
25
Now Heapify make_heap(h.begin(), h.end())
77
Example of Implementing heap sort
h 75, 35, 50, 20, 25 Result.
78
Example of Implementing heap sort (Cont.)
The vector is h 50, 35, 25, 20, 75
The vector is h 35, 20, 25, 50, 75
79
What the Vector Looks Like
The vector is h 20, 25, 35, 50, 75 SORTED!!
The vector is h 25, 20, 35, 50, 75
80
Class Example

• Perform HeapSort (make_heap followed by
  sort_heap) on the following vector
• V R, C, D, Q, L, U, A, Z, E,
  M, G, S

81
Merging Two Heaps

• We are given two two heaps and a key k
• We create a new heap with the root node storing k
  and with the two heaps as subtrees
• We perform downheap to restore the heap-order
  property

7
3
2
5
8
6
4
2
3
4
5
8
6
7
82
Bottom-up Heap Construction (7.3.5)

• We can construct a heap storing n given keys in
  using a bottom-up construction with log n phases
• In phase i, pairs of heaps with 2i -1 keys are
  merged into heaps with 2i1-1 keys

2i1-1
83
Example
15
16
12
4
7
6
20
23
25
5
11
27
15
16
12
4
7
6
20
23
84
Example (contd.)
25
5
11
27
15
16
12
4
9
6
20
23
15
4
6
23
25
16
12
5
9
11
20
27
85
Example (contd.)
7
8
15
4
6
23
25
16
12
5
9
11
20
27
4
6
15
5
8
23
25
16
12
7
9
11
20
27
86
Example (end)
10
4
6
15
5
8
23
25
16
12
7
9
11
20
27
4
5
6
15
7
8
23
25
16
12
10
9
11
20
27
87
(No Transcript)
88
(No Transcript)
89
General vector that we want to sort
90

• Now we have to heapify the structure.
• Where do we begin?
• With the first non-leaf node (from the end of the
  tree).
• How do we calculate this?
• Parent of last node!

91
(No Transcript)
92
Subtree at 46 is already a heapnow onto position
v2
93
Subtree at 17 is not a heapmust trickle down
THIS BECOMES
94
Subtree at 78 is now a heapnow onto position v2
95
Subtree at 82 is not a heapmust trickle down
THIS BECOMES
96
Subtree at 92 is now a heapnow onto position v0
97
Tree at 55 is not a heapmust trickle (all the
way) down
THIS BECOMES
98
(No Transcript)
99
Now the Sorting

• Pop one element at a time from the heap, swapping
  with the last element in the container until the
  values are sorted

100
Position v0 (value 92) is swapped with last
element (17) Now we pop_back( ) last element, and
trickle down
101
From here
102
Position v0 (value 82) is swapped with last
element (17) Now we pop_back( ) last element, and
trickle down
103
Position v0 (value 82) is swapped with last
element (17) Now we pop_back( ) last element, and
trickle down
104
The rest of the popping is left to you to
practice.
105
HeapSort Performance

• To do the heapify process is O(n) (linear)
  because you must look at every value in the heap
• To do all the deletes is then O(log2 n)
• Overall performance is
• O(n log2 n)

106
(No Transcript)
107
Summary Slide 1
- Heap - an array-based tree that has heap
order - maximum heap if vi is a parent,
then vi ? v2i1 and vi ? v2i2 (a
parent is ? its children) - root, v0, is
the maximum value in the vector - minimum
heap the parent is ? its children. - v0 is
the minimum value - Insertion place the new
value at the back of the heap and filtering
it up the tree.
107
108
Summary Slide 2
- Heap (Cont) - Deletion exchanging its
value with the back of the heap and then
filtering the new root down the tree, which now
has one less element. - Insert and delete
running time O(log2 n) - heapifying apply
the filter-down operation to the interior
nodes, from the last interior node in the tree
down to the root - running time O(n) -
The O(n log2 n) heapsort algorithm heapifies a
vector and erases repeatedly from the heap,
locating each deleted value in its final
position.