Analysis of Algorithms CS 477/677

1
Analysis of AlgorithmsCS 477/677

• Instructor Monica Nicolescu

2
The Heap Data Structure

• Def A heap is a nearly complete binary tree with
  the following two properties
• Structural property all levels are full, except
  possibly the last one, which is filled from left
  to right
• Order (heap) property for any node x
• Parent(x) x

8
It doesnt matter that 4 in level 1 is smaller
than 5 in level 2
7
4
5
2
Heap
3
Array Representation of Heaps

• A heap can be stored as an array A.
• Root of tree is A1
• Left child of Ai A2i
• Right child of Ai A2i 1
• Parent of Ai A ?i/2?
• HeapsizeA lengthA
• The elements in the subarray A(?n/2?1) .. n
  are leaves
• The root is the maximum element of the heap

A heap is a binary tree that is filled in order
4
Maintaining the Heap Property

• Suppose a node is smaller than a child
• Left and Right subtrees of i are max-heaps
• Invariant
• the heap condition is violated only at that node
• To eliminate the violation
• Exchange with larger child
• Move down the tree
• Continue until node is not smaller than children

5
Building a Heap

• Convert an array A1 n into a max-heap (n
  lengthA)
• The elements in the subarray A(?n/2?1) .. n
  are leaves
• Apply MAX-HEAPIFY on elements between 1 and ?n/2?

A
4 1 3 2 16 9 10 14 8 7
6
Heapsort

• Goal
• Sort an array using heap representations
• Idea
• Build a max-heap from the array
• Swap the root (the maximum element) with the last
  element in the array
• Discard this last node by decreasing the heap
  size
• Call MAX-HEAPIFY on the new root
• Repeat this process until only one node remains

7
Alg HEAPSORT(A)

• BUILD-MAX-HEAP(A)
• for i ? lengthA downto 2
• do exchange A1 ? Ai
• MAX-HEAPIFY(A, 1, i - 1)
• Running time O(nlgn)

8
Example A7, 4, 3, 1, 2
9
HEAP-MAXIMUM

• Goal
• Return the largest element of the heap
• Alg HEAP-MAXIMUM(A)
• return A1

Running time O(1)
Heap A
Heap-Maximum(A) returns 7
10
HEAP-EXTRACT-MAX

• Goal
• Extract the largest element of the heap (i.e.,
  return the max value and also remove that element
  from the heap
• Idea
• Exchange the root element with the last
• Decrease the size of the heap by 1 element
• Call MAX-HEAPIFY on the new root, on a heap of
  size n-1

Heap A
11
HEAP-EXTRACT-MAX

• Alg HEAP-EXTRACT-MAX(A, n)
• if n lt 1
• then error heap underflow
• max ? A1
• A1 ? An
• MAX-HEAPIFY(A, 1, n-1) remakes
  heap
• return max

Running time O(lgn)
12
Example HEAP-EXTRACT-MAX
max 16
Heap size decreased with 1
Call MAX-HEAPIFY(A, 1, n-1)
13
HEAP-INCREASE-KEY

• Goal
• Increases the key of an element i in the heap
• Idea
• Increment the key of Ai to its new value
• If the max-heap property does not hold anymore
  traverse a path toward the root to find the
  proper place for the newly increased key

Key i ? 15
14
HEAP-INCREASE-KEY

• Alg HEAP-INCREASE-KEY(A, i, key)
• if key lt Ai
• then error new key is smaller than current
  key
• Ai ? key
• while i gt 1 and APARENT(i) lt Ai
• do exchange Ai ? APARENT(i)
• i ? PARENT(i)
• Running time O(lgn)

Key i ? 15
15
Example HEAP-INCREASE-KEY
16
MAX-HEAP-INSERT

• Goal
• Inserts a new element into a max-heap
• Idea
• Expand the max-heap with a new element whose key
  is -?
• Calls HEAP-INCREASE-KEY to set the key of the new
  node to its correct value and maintain the
  max-heap property

16
14
10
8
7
9
3
2
4
1
17
MAX-HEAP-INSERT

• Alg MAX-HEAP-INSERT(A, key, n)
• heap-sizeA ? n 1
• An 1 ? -?
• HEAP-INCREASE-KEY(A, n 1, key)

16
14
10
8
7
9
3
2
4
1
Running time O(lgn)
18
Example MAX-HEAP-INSERT
19
Summary

• We can perform the following operations on heaps
• MAX-HEAPIFY O(lgn)
• BUILD-MAX-HEAP O(n)
• HEAP-SORT O(nlgn)
• MAX-HEAP-INSERT O(lgn)
• HEAP-EXTRACT-MAX O(lgn)
• HEAP-INCREASE-KEY O(lgn)
• HEAP-MAXIMUM O(1)

20
The Search Problem

• Find items with keys matching a given search key
• Applications
• Keeping track of customer account information at
  a bank
• Search through records to check balances and
  perform transactions
• Keep track of reservations on flights
• Search to find empty seats, cancel/modify
  reservations
• Search engine
• Looks for all documents containing a given word

21
Symbol Tables (Dictionaries)

• Dictionary data structure that supports two
  basic operations insert a new item and return an
  item with a given key
• Queries return information about the set
• Search (S, k)
• Minimum (S), Maximum (S)
• Successor (S, x), Predecessor (S, x)
• Modifying operations change the set
• Insert (S, k)
• Delete (S, k)

22
Implementations of Symbol Tables

• Key-indexed-array
• Key values are distinct, small numbers
• Store the items in an array, indexed by keys
• Ordered/unordered arrays
• Ordered/unordered linked lists

Insert
Search
key-indexed array
1
1
ordered array
N
N
ordered list
N
N
unordered array
N
1
unordered list
N
1
23
Binary Search Trees

• Support many dynamic set operations
• SEARCH, MINIMUM, MAXIMUM, PREDECESSOR, SUCCESSOR,
  INSERT
• Running time of basic operations on binary search
  trees
• On average ?(lgn)
• The expected height of the tree is lgn
• In the worst case ?(n)
• The tree is a linear chain of n nodes

24
Binary Search Trees

• Tree representation
• A linked data structure in which each node is an
  object
• Node representation
• Key field
• Satellite data
• Left pointer to left child
• Right pointer to right child
• p pointer to parent (p root T NIL)
• Satisfies the binary-search-tree property

25
Binary Search Tree Example

• Binary search tree property
• If y is in left subtree of x,
• then key y key x
• If y is in right subtree of x,
• then key y key x

26
Traversing a Binary Search Tree

• Inorder tree walk
• Prints the keys of a binary tree in sorted order
• Root is printed between the values of its left
  and right subtrees left, root, right
• Preorder tree walk
• root printed first root, left, right
• Postorder tree walk left, right, root
• root printed last

Inorder 2 3 5 5 7 9
Preorder 5 3 2 5 7 9
Postorder 2 5 3 9 7 5
27
Traversing a Binary Search Tree

• Alg INORDER-TREE-WALK(x)
• if x ? NIL
• then INORDER-TREE-WALK ( left x )
• print key x
• INORDER-TREE-WALK ( right x )
• E.g.
• Running time
• ?(n), where n is the size of the tree rooted at x

Output 2 3 5 5 7 9
28
Searching for a Key

• Given a pointer to the root of a tree and a key
  k
• Return a pointer to a node with key k
• if one exists
• Otherwise return NIL
• Idea
• Starting at the root trace down a path by
  comparing k with the key of the current node
• If the keys are equal we have found the key
• If k lt keyx search in the left subtree of x
• If k gt keyx search in the right subtree of x

29
Searching for a Key

• Alg TREE-SEARCH(x, k)
• if x NIL or k key x
• then return x
• if k lt key x
• then return TREE-SEARCH(left x, k )
• else return TREE-SEARCH(right x, k )

Running Time O (h), h the height of the tree
30
Example TREE-SEARCH

• Search for key 13
• 15 ? 6 ? 7 ? 13

31
Iterative Tree Search

• Alg ITERATIVE-TREE-SEARCH(x, k)
• while x ? NIL and k ? key x
• do if k lt key x
• then x ? left x
• else x ? right x
• return x

32
Finding the Minimum in a Binary Search Tree

• Goal find the minimum value in a BST
• Following left child pointers from the root,
  until a NIL is encountered
• Alg TREE-MINIMUM(x)
• while left x ? NIL
• do x ? left x
• return x
• Running time O(h), h height of tree

Minimum 2
33
Finding the Maximum in a Binary Search Tree

• Goal find the maximum value in a BST
• Following right child pointers from the root,
  until a NIL is encountered
• Alg TREE-MAXIMUM(x)
• while right x ? NIL
• do x ? right x
• return x
• Running time O(h), h height of tree

Maximum 20
34
Successor

• Def successor (x ) y, such that key y is the
  smallest key gt key x
• E.g. successor (15)
• successor (13)
• successor (9)
• Case 1 right (x) is non empty
• successor (x ) the minimum in right (x)
• Case 2 right (x) is empty
• go up the tree until the current node is a left
  child successor (x ) is the parent of the
  current node
• if you cannot go further (and you reached the
  root) x is the largest element

17
15
13
35
Finding the Successor

• Alg TREE-SUCCESSOR(x)
• if right x ? NIL
• then return TREE-MINIMUM(right x)
• y ? px
• while y ? NIL and x right y
• do x ? y
• y ? py
• return y
• Running time O (h), h height of the tree

y
x
36
Predecessor

• Def predecessor (x ) y, such that key y is
  the biggest key lt key x
• E.g. predecessor (15)
• predecessor (9)
• predecessor (13)
• Case 1 left (x) is non empty
• predecessor (x ) the maximum in left (x)
• Case 2 left (x) is empty
• go up the tree until the current node is a right
  child predecessor (x ) is the parent of the
  current node
• if you cannot go further (and you reached the
  root) x is the smallest element

13
7
9
37
Insertion

• Goal
• Insert value v into a binary search tree
• Idea
• If key x lt v move to the right child of x,
• else move to the left child of x
• When x is NIL, we found the correct position
• If v lt key y insert the new node as ys left
  child
• else insert it as ys right child
• Begining at the root, go down the tree and
  maintain
• Pointer x traces the downward path (current
  node)
• Pointer y parent of x (trailing pointer )

Insert value 13
38
Example TREE-INSERT
x, yNIL
Insert 13
x NIL y 15
39
Alg TREE-INSERT(T, z)

1. y ? NIL
2. x ? root T
3. while x ? NIL
4. do y ? x
5. if key z lt key x
6. then x ? left x
7. else x ? right x
8. pz ? y
9. if y NIL
10. then root T ? z Tree T
    was empty
11. else if key z lt key y
12. then left y ? z
13. else right y ? z

Running time O(h)
40
Readings

• Chapter 6
• Chapter 12