CSE 326: Data Structures Priority Queues (Heaps)

1
CSE 326 Data Structures Priority Queues (Heaps)

• Lecture 10 Wednesday, Jan 28, 2003

2
New Operation Merge

• Merge(H1,H2) Merge two heaps H1 and H2 of size
  O(N).
• E.g. Combine queues from two different sources to
  run on one CPU.
• Can do O(N) Insert operations O(N log N) time
• Better Copy H2 at the end of H1 (assuming array
  implementation) and use Floyds Method for
  BuildHeap.
• Running Time O(N)
• Can we do even better? (i.e. Merge in O(log N)
  time?)

3
Binomial Queues
insert heap ? value ? heap findMin heap ?
value deleteMin heap ? heap merge heap ? heap
? heap

• All in O(log n) time
• Recursive Definition of Binomial Tree Bk of
  height k
• B0 single root node
• Bk Attach Bk-1 to root of another Bk-1
• Idea a binomial heap H is a forest of binomial
  trees H B0 B1 B2 ... Bkwhere each Bi may
  be present, or may be empty

4
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
5
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
6
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
7
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
8
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
9
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
10
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
Recall a binomial heap may have any subset of
these trees
11
Why Binomial?

• Why are these trees called binomial?
• Hint how many nodes at depth d?

B0 B1 B2 B3
12
Why Binomial?

• Why are these trees called binomial?
• Hint how many nodes at depth d?
• Number of nodes at different depths d for Bk
• 1, 1 1, 1 2 1, 1 3 3 1,
• Binomial coefficients of (a b)k
  k!/((k-d)!d!)

B0 B1 B2 B3
13
Binomial Queue Properties

• Suppose you are given a binomial queue of N nodes
• There is a unique set of binomial trees for N
  nodes
• What is the maximum number of trees that can be
  in an N-node queue?
• 1 node ? 1 tree B0 2 nodes ? 1 tree B1 3 nodes
  ? 2 trees B0 and B1 7 nodes ? 3 trees B0, B1 and
  B2
• Trees B0, B1, , Bk can store up to 20 21
  2k 2k1 1 nodes N.
• Maximum is when all trees are used. So, solve
  for (k1).
• Number of trees is ? log(N1) O(log N)

14
Definition of Binomial Queues
Binomial Queue forest of heap-ordered
binomial trees
B0 B2 B0 B1 B3
1
-1
21
5
3
3
7
2
1
9
6
11
5
8
7
Binomial queue H1 5 elements 101 base 2 ? B2 B0
Binomial queue H2 11 elements 1011 base 2 ? B3
B1 B0
6
15
findMin()

• In each Bi, the minimum key is at the root
• So scan sequentially B1, B2, ..., Bk, compute the
  smallest of their keys
• Time O(log n) (why ?)

B0 B1 B2 B3
16
Binomial Queues Merge

• Main Idea Merge two binomial queues by merging
  individual binomial trees
• Since Bk1 is just two Bks attached together,
  merging trees is easy
• Steps for creating new queue by merging
• Start with Bk for smallest k in either queue.
• If only one Bk, add Bk to new queue and go to
  next k.
• Merge two Bks to get new Bk1 by making larger
  root the child of smaller root. Go to step 2 with
  k k 1.

17
Example Binomial Queue Merge
H1 H2
1
-1
5
3
21
3
9
7
2
1
6
11
5
7
8
6
18
Example Binomial Queue Merge
H1 H2
1
-1
5
3
3
9
7
2
1
6
21
11
5
7
8
6
19
Example Binomial Queue Merge
H1 H2
1
-1
5
3
9
7
2
1
6
3
11
5
7
8
21
6
20
Example Binomial Queue Merge
H1 H2
1
-1
3
5
7
2
1
3
11
5
9
6
8
21
6
7
21
Example Binomial Queue Merge
H1 H2
-1
1
3
2
1
5
7
3
11
5
8
9
6
6
21
7
22
Example Binomial Queue Merge
H1 H2
-1
1
3
2
1
5
7
3
11
5
8
9
6
6
21
7
23
Binomial Queues Merge and Insert

• What is the run time for Merge of two O(N)
  queues?
• How would you insert a new item into the queue?

24
Binomial Queues Merge and Insert

• What is the run time for Merge of two O(N)
  queues?
• O(number of trees) O(log N)
• How would you insert a new item into the queue?
• Create a single node queue B0 with new item and
  merge with existing queue
• Again, O(log N) time
• Example Insert 1, 2, 3, ,7 into an empty
  binomial queue

25
Insert 1,2,,7
1
26
Insert 1,2,,7
1
2
27
Insert 1,2,,7
3
1
2
28
Insert 1,2,,7
3
1
2
4
29
Insert 1,2,,7
1
2
3
4
30
Insert 1,2,,7
1
5
2
3
4
31
Insert 1,2,,7
1
5
2
3
6
4
32
Insert 1,2,,7
1
5
7
2
3
6
4
33
Binomial Queues DeleteMin

• Steps
• Find tree Bk with the smallest root
• Remove Bk from the queue
• Delete root of Bk (return this value) You now
  have a new queue made up of the forest B0, B1, ,
  Bk-1
• Merge this queue with remainder of the original
  (from step 2)
• Run time analysis Step 1 is O(log N), step 2 and
  3 are O(1), and step 4 is O(log N). Total time
  O(log N)
• Example Insert 1, 2, , 7 into empty queue and
  DeleteMin

34
Insert 1,2,,7
1
5
7
2
3
6
4
35
DeleteMin
5
7
2
3
6
4
36
Merge
5
2
3
6
4
7
37
Merge
5
2
3
6
4
7
38
Merge
5
2
6
3
7
4
39
Merge
5
2
6
3
7
DONE!
4
40
Implementation of Binomial Queues

• Need to be able to scan through all trees, and
  given two binomial queues find trees that are
  same size
• Use array of pointers to root nodes, sorted by
  size
• Since is only of length log(N), dont have to
  worry about cost of copying this array
• At each node, keep track of the size of the (sub)
  tree rooted at that node
• Want to merge by just setting pointers
• Need pointer-based implementation of heaps
• DeleteMin requires fast access to all subtrees of
  root
• Use First-Child/Next-Sibling representation of
  trees

41
Efficient BuildHeap for Binomial Queues

• Insert one at a time - O(n log n)
• Better algorithm
• Start with each element as a singleton tree
• Merge trees of size 1
• Merge trees of size 2
• Merge trees of size 4
• Complexity

42
Other Mergeable Priority Queues Leftist and
Skew Heaps

• Leftist Heaps Binary heap-ordered trees with
  left subtrees always longer than right subtrees
• Main idea Recursively work on right path for
  Merge/Insert/DeleteMin
• Right path is always short ? has O(log N) nodes
• Merge, Insert, DeleteMin all have O(log N)
  running time (see text)
• Skew Heaps Self-adjusting version of leftist
  heaps (a la splay trees)
• Do not actually keep track of path lengths
• Adjust tree by swapping children during each
  merge
• O(log N) amortized time per operation for a
  sequence of M operations
• See Weiss for details

43
Leftist Heaps

• An alternative heap structure that also enables
  fast merges
• Based on binary trees rather than k-ary trees

44
Idea Hang a New Tree
2
1


11
5
4
9
12
13
10
6
10
14
?
Now, just percolate down!
1
2
4
9
11
5
14
12
13
10
6
10
45
Idea Hang a New Tree
2
1


11
5
4
9
12
13
10
6
10
14
1
Now, just percolate down!
?
2
4
9
11
5
14
12
13
10
6
10
46
Idea Hang a New Tree
2
1


11
5
4
9
12
13
10
6
10
14
1
Now, just percolate down!
4
2
?
9
11
5
14
12
13
10
6
10
47
Idea Hang a New Tree
2
1


11
5
4
9
12
13
10
6
10
14
1
Note we just gave up the nice structural
property on binary heaps!
Now, just percolate down!
4
2
9
11
5
14
12
13
10
6
10
48
Problem?
1
2


4
12
15

• Need some other kind of balance condition

15
18
?
1
2
4
12
15
15
18
49
Leftist Heaps

• Idea make it so that all the work you have to do
  in maintaining a heap is in one small part
• Leftist heap
• almost all nodes are on the left
• all the merging work is on the right

50
Random DefinitionNull Path Length
the null path length (npl) of a node is the
number of nodes between it and a null in the tree

• npl(null) -1
• npl(leaf) 0
• npl(single-child node) 0

2
1
1
0
0
0
1
another way of looking at it npl is the height
of complete subtree rooted at this node
0
0
0
51
Leftist Heap Properties

• Heap-order property
• parents priority value is ? to childrens
  priority values
• result minimum element is at the root
• Leftist property
• null path length of left subtree is ? npl of
  right subtree
• result tree is at least as heavy on the left
  as the right

Are leftist trees complete? Balanced?
52
Leftist tree examples
NOT leftist
leftist
leftist
2
2
0
1
1
1
1
0
0
0
1
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
0
every subtree of a leftist tree is leftist,
comrade!
0
53
Right Path in a Leftist Tree is Short
2

• If the right path has length
• at least r, the tree has at
• least 2r - 1 nodes
• Proof by induction
• Basis r 1. Tree has at least one node 21 - 1
  1
• Inductive step assume true for r lt r. The
  right subtree has a right path of at least r - 1
  nodes, so it has at least 2r - 1 - 1 nodes. The
  left subtree must also have a right path of at
  least r - 1 (otherwise, there is a null path of r
  - 3, less than the right subtree). Again, the
  left has 2r - 1 - 1 nodes. All told then, there
  are at least
• 2r - 1 - 1 2r - 1 - 1 1 2r - 1
• So, a leftist tree with at least n nodes has a
  right path of at most log n nodes

1
1
0
0
0
1
0
0
0
54
Merging Two Leftist Heaps

• merge(T1,T2) returns one leftist heap containing
  all elements of the two (distinct) leftist heaps
  T1 and T2

merge
T1
a
a
recursive merge
L1
R1
L1
R1
a lt b
T2
b
b
L2
R2
L2
R2
55
Merge Continued
a
a
npl(R) gt npl(L1)
L1
R
R
L1
R Merge(R1, T2)
constant work at each merge recursively traverse
RIGHT right path of each tree total work O(log
n)
56
Operations on Leftist Heaps

• merge with two trees of total size n O(log n)
• insert with heap size n O(log n)
• pretend node is a size 1 leftist heap
• insert by merging original heap with one node
  heap
• deleteMin with heap size n O(log n)
• remove and return root
• merge left and right subtrees

merge
merge
57
Example
merge
?
1
3
5
merge
0
0
0
7
1
12
10
?
5
5
0
1
merge
14
0
0
0
3
12
10
10
0
12
0
0
0
8
7
0
8
8
0
14
0
8
0
12
58
Sewing Up the Example
?
?
2
3
3
3
0
0
0
?
7
1
1
7
7
5
5
5
0
0
0
0
0
0
0
0
14
14
8
14
10
0
10
8
10
8
0
0
12
0
12
12
Done?
59
Finally
2
2
3
3
0
0
1
1
7
7
5
5
0
0
0
0
0
0
8
14
14
8
10
10
0
0
12
12
60
Skew Heaps

• Problems with leftist heaps
• extra storage for npl
• two pass merge (with stack!)
• extra complexity/logic to maintain and check npl
• Solution skew heaps
• blind adjusting version of leftist heaps
• amortized time for merge, insert, and deleteMin
  is O(log n)
• worst case time for all three is O(n)
• merge always switches children when fixing right
  path
• iterative method has only one pass

What do skew heaps remind us of?
61
Merging Two Skew Heaps
merge
T1
a
a
merge
L1
R1
L1
R1
a lt b
T2
b
b
Notice the old switcheroo!
L2
R2
L2
R2
62
Example
merge
3
3
5
merge
7
7
5
12
10
5
merge
14
14
10
3
12
12
10
8
8
7
8
3
14
7
5
14
10
8
12
63
Skew Heap Code

• void merge(heap1, heap2)
• case
• heap1 NULL return heap2
• heap2 NULL return heap1
• heap1.findMin() lt heap2.findMin()
• temp heap1.right
• heap1.right heap1.left
• heap1.left merge(heap2, temp)
• return heap1
• otherwise
• return merge(heap2, heap1)

64
Comparing Heaps

• Binary Heaps
• d-Heaps
• Binomial Queues

• Leftist Heaps
• Skew Heaps

65
Summary of Heap ADT Analysis

• Consider a heap of N nodes
• Space needed O(N)
• Actually, O(MaxSize) where MaxSize is the size of
  the array
• Pointer-based implementation pointers for
  children and parent
• Total space 3N 1 (3 pointers per node 1 for
  size)
• FindMin O(1) time DeleteMin and Insert O(log
  N) time
• BuildHeap from N inputs What is the run time?
• N Insert operations O(N log N)
• O(N) Treat input array as a heap and fix it
  using percolate down
• Thanks, Floyd!
• Mergable Heaps Binomial Queues, Leftist Heaps,
  Skew Heaps