CS 450 Design

1
CS 450 Design Analysis of AlgorithmsNotes
10

• Fall 2009Sarah Mocas

2
Optimized Trees

• Overview Self-balancing binary search tree
• Red Black Trees
• Splay Trees

3
Optimized Trees - Overview

• A self-balancing binary search tree is any BST
  that keeps its height small, O(lg n), even with
  arbitrary insertions and deletions
• Self-balancing binary trees perform
  transformations on the tree (such as tree
  rotations) at key times

Trade off overhead to balance in order to get
fast execution of later operations (search)
4
Optimized Trees - Overview

• Popular data structures implementing
  self-balancing binary search tree
• AVL tree first 1962 and slower than red-black
  trees for insertion and removal but faster
  look-ups
• Red-black tree - 1972
• Scapegoat tree - 1993
• Splay tree - self optimizing, frequently accessed
  nodes will move nearer to the root
• Treap
• B-tree is a generalization of a binary search
  tree (extra children), still self-balancing

5
Optimized Trees - Overview

• B-trees is a tree data structure that allows
  searches, insertions, deletions in logarithmic
  amortized time
• the average running time per operation over a
  worst-case sequence of operations is O(lg n)

6
Optimized Trees - Overview

• Generalization of a binary search tree
• more than two paths from a single node
• Better performance when node access times far
  exceed access times within nodes
• when nodes are in secondary storage (on disk
  drives)

7
Optimized Trees - Overview

• Unlike self-balancing binary search trees, the
  B-tree is optimized for systems that read and
  write large blocks of data
• most commonly used in databases and filesystem
• do not need re-balancing as frequently as other
  self-balancing search trees, but may waste space

Trade off is the space in a node to keep track of
children verses the time to do rotates
8
Optimized Trees
Optimized TreesPart I - Red Black Trees
9
Red-Black Trees

• We want a method of ensuring tree shape does not
  become degenerate.
• Red-Black trees are one of many schemes to keep
  the trees balanced in some sense to guarantee
  O(lg n) time in the worst case.

10
Red-Black Trees II

• A red-black tree is a binary search tree with one
  extra attributeeach node has the color
  attribute, which may be red or black.
• We constrain the way that nodes can be colored in
  any path from the root to any leaf in order to
  ensure that no such path is more than twice as
  long as any other.

11
Red-Black Trees III

• Every node of a red-black tree now has the fields
  color, key, left, right, and p.
• Nil pointers will be regarded as pointers to
  external nodes (leaves) of the BST and then the
  normal, key-bearing nodes are internal nodes.

12
Properties of Red-Black Trees

• A BST is a Red-Black tree if it satisfies the
  following Red-Black properties
• Every node is either red or black
• The root is black
• Every leaf (NIL) is black
• If a node is red, both of its children are black
• For each node, all paths from the node to
  descendant leaves contain the same number of
  black nodes

13
Example of a red-black tree
26
41
17
47
21
14
30
23
19
16
38
10
28
20
15
12
39
7
35
3
Leaves are black NIL children that are ignored
14
Example of a red-black tree
26
41
17
47
21
14
30
23
19
16
38
10
28
20
15
12
39
7
35
3
15
Black Height
b
bh2, h4
r
b
bh2, h3
b
b
bh1, h2
r
r
bh1, h1
bh0, h0
bh(x) blacks on path to leaf, excluding x
16
Red Black Tree Properties

• A RB tree with n internal nodes has height h ?
  2 lg(n1)
• Height, number of nodes, and degree of balance
  are all related properties
• We can show the height bound by showing how the
  number of nodes grows with height

17
Height, Balance, Nodes

• Height 3
• Internal nodes 3
• Internal nodes grow linearly with Height

• Height 3
• Internal nodes 7
• Internal nodes grow exponentially with Height

18
RB Tree Height

• Consider a subtree rooted at x
• If we can show that the number of nodes in the
  subtree grows exponentially with the height,
  were all set.
• Will show that a subtree rooted at x has at least
  2bh(x) 1 internal nodes

19
Proof of Black Height
b

• Will show that a subtree rooted at x has at least
  2bh(x) 1 internal nodes

bh0, h0

• Prove by induction on height of x
• Begin with height 0
• Must be a leaf
• Must have black height 0
• internal nodes 0 20 1 0 (true)

20
Proof of Black Height
bhbh(x), hn
bh1 or bh2

• Now consider a root node, x, with positive height
• Children have black height bh(x) or bh(x) - 1

?
bh1
r?
b?

• So, the fewest number of internal nodes that each
  child could have is (2bh(x)-1 1)
• Thus the tree in total must have at least
  2(2bh(x)-1 1) nodes which is just 2bh(x) 1

21
Proof of Black Height

• We know that
• n 2bh(x) 1
• n1 2bh(x)
• lg(n1) bh(x)
• bh(x) lg(n1)
• Since h(x)/2 bh(x)
• h(x)/2 bh(x) lg(n1)
• h(x) 2lg(n1)

• If a RB tree has at least 2bh(x) 1 internal
  nodes, it must be reasonably balanced.
• Why?
• bh(x) h(x) 2bh(x)
• So, h(x)/2 bh(x)

22
Problem of inserting into RB tree
7
9
5
12
23
Problem of inserting into RB tree
7
9
5
12
8
Added (8) as red
24

• Every node is either red or black
• The root is black
• Every leaf (NIL) is black
• If a node is red, both of its children are black
• For each node, all paths contain the same number
  of black nodes

7
9
5
12
8
11
11 cannot be b or r!
25
Problem of inserting into RB tree
7
9
5
12
b
8
b
b
b
b
11
Can save by re-coloring
b
b
26
Problem of inserting into RB tree
7
Now, insert 10 Recoloring cannot save us! Must
change structure Goal O(lg n) time
9
5
12
8
b
b
b
b
b
11
?
b
10
b
b
27
Red-Black Tree Rotations

• The search-tree operations TREE-INSERT and
  TREE-DELETE, when run on a red-black tree with n
  keys, take O(lg n) time
• We must keep appropriate RB tree properties may
  need to
• change the colors of some nodes
• change the pointer structures
• This is called rotation

28
Red-Black Tree Rotations (cont)

• There are two kinds of rotations, namely left
  rotations and right rotations.
• To do a left rotation on a node x, we assume that
  that its right child is not nilT.
• x may be any node in the tree whose right child y
  is not nil.
• The left rotation pivots around the link from x
  to y. It makes y the new root of the subtree,
  with x as ys left child and ys left child as
  xs right child.

29
Red-Black Tree Rotations
Left-Rotate(T,x)
y
x
Right-Rotate(T,y)
?
x
?
y
?
?
?
?
30
Problem of inserting into RB tree
7
What is a rotate to fix this?
9
5
12
8
b
b
b
b
b
11
?
b
10
b
b
31
Problem of inserting into RB tree
7
Assume that 10 is red What is a rotate to fix
this?
9
5
12
8
b
b
b
b
b
11
?
b
10
b
b
32
Problem of inserting into RB tree
7
Recolor the parent of 10 What is a rotate to fix
this?
9
5
12
8
b
b
b
b
b
11
?
b
10
b
b
33
Problem of inserting into RB tree
7
9
5
11
8
b
b
Fixed? Basic idea - but the steps of the
algorithm are a little different
b
b
12
10
b
b
b
b
34
RB Tress Rotations

• Rotations must be selected based on
• Operation being performed
• Local structure of tree
• They must maintain the red-black property
• Allow O(lg n) bound on all operations

35
Insertion

• Similar to Insert in BST, but
• Set left and right child to be NIL leaves
• Color new node RED
• Perform a RB-INSERT-FIXUP on the new node

36
Problem of inserting into RB tree
11
2
14
7
1
15
8
5
4
37
Problem of inserting into RB tree
11
2
14
7
1
15
8
5
Fix Me
4
38
Problem of inserting into RB tree
11
2
14
7
1
15
Fix Me
8
5
4
39
Problem of inserting into RB tree
11
Fix Me
7
14
8
2
15
1
5
4
40
Problem of inserting into RB tree
7
11
2
1
14
8
5
15
4
41
RB-INSERT-FIXUP( T, z )

• An iterative procedure
• Begins with added node
• On each iteration
• z is RED
• If pz is the root, pz is BLACK
• If theres a violation of RB properties, it must
  be
• z is the root (and its red)
• z and pz are both red

42
Case 1 Zs Uncle is Red
11
11
2
2
14
14
Z
7
7
15
15
1
1
8
5
8
5
4
4
Z

• Wont matter if z is left or right child
• Just recolor parent, grandparent and uncle
• Move Z to Zs grandparent

43
Case 2 Zs Uncle is Black and Z is Right Child
11
11
2
14
14
7
Z
Z
7
15
15
1
2
8
1
8
5
5
4
4

• Performing rotation wont affect black height
• Move Z to Zs pre-rotated parent

44
Case 3 Zs Uncle is Black and Z is Left Child
Z
11
7
14
7
Z
11
2
15
2
8
1
14
5
8
1
5
15
4
4

• Performing rotation wont affect black height
• Move Z to Zs pre-rotated parent

45
Time for RB Tree Insert

• The work for RB-INSERT is similar to BST (finds
  the correct leaf) but then calls RB-INSERT-FIXUP
  once
• Time O(lg n) time for RB-INSERT-FIXUP
• RB-INSERT-FIXUP just moves from the bottom of the
  tree to the top
• Time is O(height of the tree)
• Time O(lg n)
• Total time is O(lg n) O(lg n) so just O(lg n)

46
Delete from a RB Tree

• In a BST- deleting a node with two non-leaf
  children
• find either the max element in left subtree or
  the min element in its right subtree, and move
  its value into the node being deleted
• RB-DELETE(T, z) is similar but calls
  RB-DELETE-FIXUP
• RB-DELETE(T, z) time is O(lg n)

47
Recall ordinary BST Delete

• If node to be deleted is a leaf, just delete it.
• If node to be deleted has just one child, replace
  it with that child (splice)
• If node to be deleted has two children, replace
  the value in the node by its in-order
  predecessor/successors value then delete the
  in-order predecessor/successor (a recursive step)

48
Bottom-Up Deletion

• Do ordinary BST deletion. Eventually a case 1
  or case 2 deletion will be done (leaf or just
  one child)
• If deleted node U is a leaf, think of deletion as
  replacing U with the NULL pointer (black)
• If U had one child V think of deletion as
  replacing U with V
• What can go wrong??

49
Which RB Property may be violated after deletion?

• If U is Red?
• Not a problem no RB properties violated
• 2. If U is Black?
• If U is not the root, deleting it will change
  the black-height along some path

50
Example of a red-black tree
26
41
17
47
21
14
30
23
19
16
38
10
28
20
15
12
39
7
35
11
3
Try deleting 14 by moving 12
51
Fixing the problem

• Think of V (node replacing the deleted node) as
  having an extra unit of blackness. This extra
  blackness must be absorbed into the tree (by a
  red node), or propagated up to the root and out
  of the tree
• There are four cases our examples and rules
  assume that V is a left child. There are
  symmetric cases for V as a right child

52
Terminology

• The node just deleted was U
• The node that replaces it is V, which has an
  extra unit of blackness
• The parent of V is P
• The sibling of V is S

Black Node
Red or Black and dont care
Red Node
53
Bottom-Up DeletionCase 1

• Vs sibling, S, is Red
• Rotate S around P and recolor S P
• NOT a terminal case One of the other cases will
  now apply
• All other cases apply when S is Black

54
Case 1 Diagram - Vs sibling, S, is Red
S
P
Rotate S around P
P
S
V
V
S
Recolor S P
P
V
55
Bottom-Up DeletionCase 2

• Vs sibling S is Black and has two Black
  children.
• Recolor S to be Red
• P absorbs Vs extra blackness
• If P is Red, were done (it absorbed the
  blackness gets colored black)
• If P is Black, it now has extra blackness and
  problem has been propagated up the tree

56
Case 2 diagram - Vs sibling S is Black with two
Black children
Recolor S P absorbs blackness
P
P
S
S
V
V
Either extra Black absorbed by P (if it was red)
or P now has extra blackness (if it was black)
57
Bottom-Up DeletionCase 3

• S is Black, Ss right child is Black and Ss left
  child is Red
• Rotate Ss left child around S
• Swap color of S and Ss left child

58
Case 3 Diagrams - S is Black, Ss right child is
Black and Ss left child is Red
P
P
S
V
V
P
S
Rotate Ss red child left around S
V
S
Swap colors of S and Ss original left child
59
Bottom-Up DeletionCase 4

• S is Black
• Ss right child is RED (Left child either color)
• Rotate S around P
• Swap colors of S and P, and color Ss right
  child Black
• This is the terminal case were done

60
Case 4 diagrams - S is Black and Ss right child
is RED
S
P
Rotate S around P
P
S
V
V
S
P
Swap colors of S PColor Ss right child Black
V
61
Top-Down Deletion

• An alternative to the recursive bottom-up
  deletion is top-down deletion.
• This method is iterative. It moves down the tree
  only, fixing things as it goes.

62
Delete 90
65
50
80
10
70
90
60
52
8
62
Perform the following deletions, in the order
specified Delete 90, Delete 80, Delete 50,
Delete 52
63
Delete 90
65
50
80
10
70
90
60
nil
8
52
62

• Vs sibling S (70) was Black and had two Black
  children.
• Recolor S to be Red
• P absorbs Vs extra blackness
• If P is Red, were done (it absorbed the
  blackness)

64
Delete 80
65
50
80
10
70
60
8
52
62
65
Delete 80
65
50
70
10
60
8
52
62

• Need predecessor pf 80

66
Delete 50
65
50
65
70
10
60
52
8
52
62
70
10
60

• Successor of 50 is 52
• Since 52 was red - done

8
nil
62
67
Delete 52
65
52
65
70
10
60
60
8
62
70
10
62

• Successor of 52 is 60
• Since 52 was red - done

8
68
Uses of RBTs

• Completely Fair Scheduler used in current Linux
  kernels uses red-black trees
• Valuable in functional programming, where they
  are one of the most common persistent data
  structures used to construct associative arrays
  and sets which can retain previous versions after
  mutations

69
Optimized TreesPart II - Augmented Trees
70
Order-Statistic Tree

• Want to find the element with the ith smallest
  key
• Obvious approach would be to use a sorted list /
  array
• Insertion / deletion has poor performance (array)
• Search has poor performance (list)

71
How Can We Use a Tree?

• Add a size field that contains the number of
  elements (internal nodes) in the rooted subtree
• Leaf.size 0

• size(x) 1 size(leftx) size(rightx)

72
Optimized TreesPart - Splay Trees
73
Optimizing Trees

• What if the keys I want most of the time are
  stuck in the bottom of my tree?
• Use rotations to move data to best place

74
Splay Trees

• Introduce new operation splay
• splay(x) moves a node to the root using rotations
• Idea is to raise up frequently used nodes

75
Splay Tree Rotations

• zig rotate about root
• zig-zig rotate about grandparent, rotate
  about parent (same direction)
• zig-zag rotate about parent rotate about
  grandparent (different directions)

76
A Zig on x
z
x
x
y
z
y
77
A Zig-Zig on x
z
x
y
y
?
?
x
z
?
?
?
?
?
?
78
A Zig-Zag on x
z
x
y
?
y
z
x
?
?
?
?
?
?
?
79
A Zig-Zag on x
z
x
y
z
?
?
?
y
?
?
x
x
?
y
z
?
?
?
?
?
?
80
Splay(x)

• Is X the root? if yes, stop
• Is X a child of the root? If yes, zig
• Is X a left-left child or a
• right-right child? If yes, zig-zig
• Is X a left-right child or a
• right-left child? If yes, zig-zag

81
Splay Tree

• A Binary Search Tree where a node is splayed
  after being accessed (via search or update)
• Deepest internal node is splayed
• splay costs O(depth of tree) worst case, this is
  still O(n)

82
Deciding which node to splay

• Search
• splay key if found
• splay last internal node otherwise
• Insert
• splay new node
• Delete
• splay parent of the node that was removed

83
Splay Tree Highlights

• Amortized costs are O(lg n)
• the average running time per operation over a
  worst-case sequence of operations is O(lg n)
• If some data is used frequently, the adaptation
  can be a big benefit
• average costs much less than O(lg n)

84
Tree Highlights

• Binary Search trees, effective data structure
  when searching is frequent
• Suffers from degenerative height
• Red Black trees ensure balancing
• May still leave frequently used data at the
  bottom
• Splay trees optimize for the use, but may become
  unbalanced
• some data may be costly to find, manipulate