Search Trees RedBlack and Other Dynamically BalancedTrees

1
Search Trees Red-Black and Other Dynamically
BalancedTrees

• ECE573 Data Structures and Algorithms
• Electrical and Computer Engineering Dept.
• Rutgers University
• http//www.cs.rutgers.edu/vchinni/dsa/

2
Searching - Re-visited

• Binary tree O(log n) if it stays balanced
• Simple binary tree good for static collections
• Low (preferably zero) frequency of
  insertions/deletions
• but my collection keeps changing!
• Its dynamic
• Need to keep the tree balanced
• First, examine some basic tree operations
• Useful in several ways!

3
Tree Traversal

• Traversal visiting every node of a tree
• Three basic alternatives
• Pre-order
• Root
• Left sub-tree
• Right sub-tree

x A x B C x D E F
L
R
L
L
R
4
Tree Traversal

• Traversal visiting every node of a tree
• Three basic alternatives
• In-order
• Left sub-tree
• Root
• Right sub-tree

µ


³


²

A x B C x D x E F

L
L
R
5
Tree Traversal

• Traversal visiting every node of a tree
• Three basic alternatives
• Post-order
• Left sub-tree
• Right sub-tree
• Root

µ

³


²
A B C D E x x F x




L
L
R
6
Tree Traversal

• Post-order
• Left sub-tree
• Right sub-tree
• Root
• Reverse-Polish
• Normal algebraic form
• which traversal?

µ

³


²



(A (((BC)(DEx) x) F )x )

(A x(((BC)(DxE))F))
7
Trees - Searching

• Binary search tree
• Produces a sorted list by in-order traversal
• In order A D E G H K L M N O P T V

8
Trees - Searching

• Binary search tree
• Preserving the order
• Observe that this transformation preserves
  thesearch tree

9
Trees - Searching

• Binary search tree
• Preserving the order
• Observe that this transformation preserves
  thesearch tree
• Weve performed a rotation of the sub-tree about
  the T and O nodes

10
Trees - Rotations

• Binary search tree
• Rotations can be either left- or right-rotations
• For both trees the inorder traversal is
• A x B y C

11
Trees - Rotations

• Binary search tree
• Rotations can be either left- or right-rotations
• Note that in this rotation, it was necessary to
  moveB from the right child of x to the left
  child of y

12
Trees - Red-Black Trees

• A Red-Black Tree
• Binary search tree
• Each node is coloured red or black
• An ordinary binary search tree with node
  colouringsto make a red-black tree

13
Trees - Red-Black Trees

• A Red-Black Tree
• Every node is RED or BLACK
• Every leaf is BLACK

When you examinerb-tree code, you will see
sentinel nodes (black) added as the leaves. They
contain no data.
Sentinel nodes (black)
14
Trees - Red-Black Trees

• A Red-Black Tree
• Every node is RED or BLACK
• Every leaf is BLACK
• If a node is RED, then both children are BLACK

This implies that no path may have two
adjacent RED nodes. (But any number of
BLACK nodes may be adjacent.)
15
Trees - Red-Black Trees

• A Red-Black Tree
• Every node is RED or BLACK
• Every leaf is BLACK
• If a node is RED, then both children are BLACK
• Every path from a node to a leaf contains the
  same number of BLACK nodes

From the root, there are 3 BLACK nodes on every
path
16
Trees - Red-Black Trees

• A Red-Black Tree
• Every node is RED or BLACK
• Every leaf is BLACK
• If a node is RED, then both children are BLACK
• Every path from a node to a leaf contains the
  same number of BLACK nodes

The length of this path is the black height of
the tree
17
Trees - Red-Black Trees

• Lemma
• A RB-Tree with n nodes has
• height 2 log(n1)
• Proof .. See Cormen
• Essentially, height 2 black height
• Search time
• O( log n )

18
Trees - Red-Black Trees

• Data structure
• As well see, nodes in red-black trees need to
  know their parents,
• so we need this data structure

struct t_red_black_node enum red, black
colour void item struct
t_red_black_node left,
right, parent
Same as a binary tree with these
two attributes added
19
Trees - Insertion

• Insertion of a new node
• Requires a re-balance of the tree

rb_insert( Tree T, node x ) / Insert in
the tree in the usual way / tree_insert( T,
x ) / Now restore the red-black property
/ x-gtcolour red while ( (x !
T-gtroot) (x-gtparent-gtcolour red) )
if ( x-gtparent x-gtparent-gtparent-gtleft )
/ If x's parent is a left, y is x's right
'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red )
/ case 1 - change the colours /
x-gtparent-gtcolour black
y-gtcolour black
x-gtparent-gtparent-gtcolour red
/ Move x up the tree / x
x-gtparent-gtparent
Insert node 4 Mark it red
Label the current node x
20
Trees - Insertion
rb_insert( Tree T, node x ) / Insert in
the tree in the usual way / tree_insert( T,
x ) / Now restore the red-black property
/ x-gtcolour red while ( (x !
T-gtroot) (x-gtparent-gtcolour red) )
if ( x-gtparent x-gtparent-gtparent-gtleft )
/ If x's parent is a left, y is x's right
'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red )
/ case 1 - change the colours /
x-gtparent-gtcolour black
y-gtcolour black
x-gtparent-gtparent-gtcolour red
/ Move x up the tree / x
x-gtparent-gtparent
While we havent reached the root and xs parent
is red
x-gtparent
21
Trees - Insertion
rb_insert( Tree T, node x ) / Insert in
the tree in the usual way / tree_insert( T,
x ) / Now restore the red-black property /
x-gtcolour red while ( (x ! T-gtroot)
(x-gtparent-gtcolour red) ) if (
x-gtparent x-gtparent-gtparent-gtleft )
/ If x's parent is a left, y is x's right
'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red )
/ case 1 - change the colours /
x-gtparent-gtcolour black
y-gtcolour black
x-gtparent-gtparent-gtcolour red
/ Move x up the tree / x
x-gtparent-gtparent
If x is to the left of its granparent
x-gtparent-gtparent
x-gtparent
22
Trees - Insertion
/ Now restore the red-black property /
x-gtcolour red while ( (x ! T-gtroot)
(x-gtparent-gtcolour red) ) if (
x-gtparent x-gtparent-gtparent-gtleft )
/ If x's parent is a left, y is x's right
'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red )
/ case 1 - change the colours /
x-gtparent-gtcolour black
y-gtcolour black
x-gtparent-gtparent-gtcolour red
/ Move x up the tree / x
x-gtparent-gtparent
y is xs right uncle
x-gtparent-gtparent
x-gtparent
right uncle
23
Trees - Insertion
while ((x ! T-gtroot) (x-gtparent-gtcolour
red) ) if ( x-gtparent
x-gtparent-gtparent-gtleft ) /If x's
parent is a left, y is x's right'uncle/
y x-gtparent-gtparent-gtright if (
y-gtcolour red ) / case 1 -
change the colours /
x-gtparent-gtcolour black
y-gtcolour black
x-gtparent-gtparent-gtcolour red
/ Move x up the tree / x
x-gtparent-gtparent
If the uncle is red, change the colours of y, the
grand-parent and the parent
x-gtparent-gtparent
x-gtparent
right uncle
24
Trees - Insertion
while ((x ! T-gtroot) (x-gtparent-gtcolour
red) ) if ( x-gtparent
x-gtparent-gtparent-gtleft ) / If x's
parent is a left, y is x's right 'uncle' /
y x-gtparent-gtparent-gtright if (
y-gtcolour red ) / case 1 -
change the colours /
x-gtparent-gtcolour black
y-gtcolour black
x-gtparent-gtparent-gtcolour red
/ Move x up the tree / x
x-gtparent-gtparent
25
Trees - Insertion
while ( (x ! T-gtroot) (x-gtparent-gtcolour
red) ) if ( x-gtparent x-gtparent-gtparent-gtle
ft ) / If x's parent is a left, y is x's
right 'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red ) / case 1
- change the colours / x-gtparent-gtcolour
black y-gtcolour black
x-gtparent-gtparent-gtcolour red / Move
x up the tree / x x-gtparent-gtparent
xs parent is a left again, mark xs uncle but
the uncle is black this time
New x
26
Trees - Insertion
while ( (x ! T-gtroot) (x-gtparent-gtcolour
red) ) if ( x-gtparent x-gtparent-gtparent-gtle
ft ) / If x's parent is a left, y is x's
right 'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red ) / case 1
- change the colours / x-gtparent-gtcolour
black y-gtcolour black
x-gtparent-gtparent-gtcolour red / Move
x up the tree / x x-gtparent-gtparent
else / y is a black node /
if ( x x-gtparent-gtright ) / and
x is to the right / / case 2 - move
x up and rotate / x x-gtparent
left_rotate( T, x )
.. but the uncle is black this time and x is to
the right of its parent
27
Trees - Insertion
while ( (x ! T-gtroot) (x-gtparent-gtcolour
red) ) if ( x-gtparent x-gtparent-gtparent-gtle
ft ) / If x's parent is a left, y is x's
right 'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red ) / case 1
- change the colours / x-gtparent-gtcolour
black y-gtcolour black
x-gtparent-gtparent-gtcolour red / Move
x up the tree / x x-gtparent-gtparent
else / y is a black node /
if ( x x-gtparent-gtright ) / and
x is to the right / / case 2 - move
x up and rotate / x x-gtparent
left_rotate( T, x )
.. So move x up and rotate about x as root ...
28
Trees - Insertion
while ( (x ! T-gtroot) (x-gtparent-gtcolour
red) ) if ( x-gtparent x-gtparent-gtparent-gtle
ft ) / If x's parent is a left, y is x's
right 'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red ) / case 1
- change the colours / x-gtparent-gtcolour
black y-gtcolour black
x-gtparent-gtparent-gtcolour red / Move
x up the tree / x x-gtparent-gtparent
else / y is a black node /
if ( x x-gtparent-gtright ) / and
x is to the right / / case 2 - move
x up and rotate / x x-gtparent
left_rotate( T, x )
29
Trees - Insertion
while ( (x ! T-gtroot) (x-gtparent-gtcolour
red) ) if ( x-gtparent x-gtparent-gtparent-gtle
ft ) / If x's parent is a left, y is x's
right 'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red ) / case 1
- change the colours / x-gtparent-gtcolour
black y-gtcolour black
x-gtparent-gtparent-gtcolour red / Move
x up the tree / x x-gtparent-gtparent
else / y is a black node /
if ( x x-gtparent-gtright ) / and
x is to the right / / case 2 - move
x up and rotate / x x-gtparent
left_rotate( T, x )
.. but xs parent is still red ...
30
Trees - Insertion
while ( (x ! T-gtroot) (x-gtparent-gtcolour
red) ) if ( x-gtparent x-gtparent-gtparent-gtle
ft ) / If x's parent is a left, y is x's
right 'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red ) / case 1 -
change the colours / x-gtparent-gtcolour
black y-gtcolour black
x-gtparent-gtparent-gtcolour red / Move
x up the tree / x x-gtparent-gtparent
else / y is a black node / if ( x
x-gtparent-gtright ) / and x is to
the right / / case 2 - move x up
and rotate / x x-gtparent
left_rotate( T, x )
.. The uncle is black ..
uncle
.. and x is to the left of its parent
31
Trees - Insertion
while ( (x ! T-gtroot) (x-gtparent-gtcolour
red) ) if ( x-gtparent x-gtparent-gtparent-gtle
ft ) / If x's parent is a left, y is x's
right 'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red ) / case 1 -
change the colours / x-gtparent-gtcolour
black y-gtcolour black
x-gtparent-gtparent-gtcolour red / Move
x up the tree / x x-gtparent-gtparent
else / y is a black node / if (
x x-gtparent-gtright ) / and x is
to the right / / case 2 - move x up
and rotate / x x-gtparent
left_rotate( T, x ) else / case 3 /
x-gtparent-gtcolour black
x-gtparent-gtparent-gtcolour red
right_rotate( T, x-gtparent-gtparent )
.. So we have the final case ..
32
Trees - Insertion
while ( (x ! T-gtroot) (x-gtparent-gtcolour
red) ) if ( x-gtparent x-gtparent-gtparent-gtle
ft ) / If x's parent is a left, y is x's
right 'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red ) / case 1 -
change the colours / x-gtparent-gtcolour
black y-gtcolour black
x-gtparent-gtparent-gtcolour red / Move
x up the tree / x x-gtparent-gtparent
else / y is a black node / if (
x x-gtparent-gtright ) / and x is
to the right / / case 2 - move x up
and rotate / x x-gtparent
left_rotate( T, x ) else / case 3 /
x-gtparent-gtcolour black
x-gtparent-gtparent-gtcolour red
right_rotate( T, x-gtparent-gtparent )
.. Change colours and rotate ..
33
Trees - Insertion
while ( (x ! T-gtroot) (x-gtparent-gtcolour
red) ) if ( x-gtparent x-gtparent-gtparent-gtle
ft ) / If x's parent is a left, y is x's
right 'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red ) / case 1 -
change the colours / x-gtparent-gtcolour
black y-gtcolour black
x-gtparent-gtparent-gtcolour red / Move
x up the tree / x x-gtparent-gtparent
else / y is a black node / if (
x x-gtparent-gtright ) / and x is
to the right / / case 2 - move x up
and rotate / x x-gtparent
left_rotate( T, x ) else / case 3 /
x-gtparent-gtcolour black
x-gtparent-gtparent-gtcolour red
right_rotate( T, x-gtparent-gtparent )
34
Trees - Insertion
while ( (x ! T-gtroot) (x-gtparent-gtcolour
red) ) if ( x-gtparent x-gtparent-gtparent-gtle
ft ) / If x's parent is a left, y is x's
right 'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red ) / case 1 -
change the colours / x-gtparent-gtcolour
black y-gtcolour black
x-gtparent-gtparent-gtcolour red / Move
x up the tree / x x-gtparent-gtparent
else / y is a black node / if (
x x-gtparent-gtright ) / and x is
to the right / / case 2 - move x up
and rotate / x x-gtparent
left_rotate( T, x ) else / case 3 /
x-gtparent-gtcolour black
x-gtparent-gtparent-gtcolour red
right_rotate( T, x-gtparent-gtparent )
This is now a red-black tree .. So were finished!
35
Trees - Insertion
while ( (x ! T-gtroot) (x-gtparent-gtcolour
red) ) if ( x-gtparent x-gtparent-gtparent-gtle
ft ) / If x's parent is a left, y is x's
right 'uncle' / y x-gtparent-gtparent-gtright
if ( y-gtcolour red ) / case 1 -
change the colours / x-gtparent-gtcolour
black y-gtcolour black
x-gtparent-gtparent-gtcolour red / Move
x up the tree / x x-gtparent-gtparent
else / y is a black node / if (
x x-gtparent-gtright ) / and x is
to the right / / case 2 - move x up
and rotate / x x-gtparent
left_rotate( T, x ) else / case 3 /
x-gtparent-gtcolour black
x-gtparent-gtparent-gtcolour red
right_rotate( T, x-gtparent-gtparent )
else ....
Theres an equivalent set of cases when the
parent is to the right of the grandparent!
36
Red-black trees - Analysis

• Addition
• Insertion Comparisons O(log n)
• Fix-up
• At every stage,x moves up the tree at least one
  level O(log n)
• Overall O(log n)
• Deletion
• Also O(log n)
• More complex
• ... but gives O(log n) behaviour in dynamic cases

37
Red Black Trees - What you need to know?

• Code?
• This is not a course for masochists!
• You can find it in a text-book
• You need to know
• The algorithm exists
• What its called
• When to use it
• ie what problem does it solve?
• Its complexity
• Basically how it works
• Where to find an implementation
• How to transform it to your application

38
Key Points

• Red-Black Trees
• A complex algorithm
• Gives O(log n) addition, deletion
  and search
• Software engineers
• Know about algorithms
• Know when to use them
• Know performance
• Know where to find an implementation
• Are too clever to re-invent wheels ...
• They re-use code
• So they have more time for
• Sailing, eating, drinking, ...
• Anything else I should put here
  

39
AVL and other balanced trees

• AVL Trees
  
• First balanced tree algorithm
• Discoverers Adelson-Velskii and Landis
• Properties
• Binary tree
• Height of left and right-subtrees differ by at
  most 1
• Subtrees are AVL trees

AVL Tree
AVL Tree
40
AVL trees - Height

• Theorem
• An AVL tree of height h has at least Fh31 nodes
• Proof
• Let Sh be the size of the smallest AVL tree of
  height h
• Clearly, S0 1 and S1 2
• Also, Sh Sh-1 Sh-2 1
• A minimum height tree must be composed of min
  height trees differing in height by at most 1
• By induction ..
• Sh Fh31

41
AVL trees - Height

• Now, for large i, Fi1 / Fi f,
• where f ½(1 Ö5)
• or Fi c (½(1
  Ö5))i
• Sh Fh3 1 O( bh )
• n ³ Sh , so n is W( bh )
• and h logb n or h is O(log
  n)

42
AVL trees - Height

• Now, for large i, Fi1 / Fi f,
• where f ½(1 Ö5)
• or Fi c (½(1
  Ö5))i
• Sh Fh3 1 O( bh )
• n ³ Sh , so n is W( bh )
• and h logb n or h is O(log
  n)
• In this case, we can show
• h 1.44 logb (n2) - 1.328

h is no worse than 44 higher than the optimum
43
AVL Trees - Rebalancing

• Insertion leads to non-AVL tree
• 4 cases
• 1 and 4 are mirror images
• 2 and 3 are mirror images

1
2
3
4
44
AVL Trees - Rebalancing

• Case 1 solved by rotation
• Case 4 is the mirror image rotation

45
AVL Trees - Rebalancing

• Case 2 needs a double rotation
• Case 3 is the mirror image rotation

46
AVL Trees - Data Structures

• AVL trees can be implemented with a flag to
  indicate the balance state
• Insertion
• Insert a new node (as any binary tree)
• Work up the tree re-balancing as necessary to
  restorethe AVL property

typedef enum LeftHeavy, Balanced, RightHeavy
BalanceFactor struct
AVL_node BalanceFactor bf void item
struct AVL_node left, right
47
Dynamic Trees - Red-Black or AVL

• Insertion
• AVL two passes through the tree
• Down to insert the node
• Up to re-balance
• Red-Black two passes through the tree
• Down to insert the node
• Up to re-balance
• but Red-Black is more popular??

48
Dynamic Trees - A cautionary tale

• Insertion
• If you read Cormen et al,
• Theres no reason to prefer a red-black tree
• However, in Weiss text
• M A Weiss, Algorithms, Data Structures and
  Problem Solving with C, Addison-Wesley, 1996
• you find that you can balance a red-black tree
  in one pass!
• Making red-black more efficient than AVLif coded
  properly!!!

49
Dynamic Trees - A cautionary tale

• Insertion
• If you read Cormen et al,
• Theres no reason to prefer a red-black tree
• However, in Weiss text
• M A Weiss, Algorithms, Data Structures and
  Problem Solving with C, Addison-Wesley, 1996
• you find that you can balance a red-black tree
  in one pass!
• Making red-black more efficient than AVLif coded
  properly!!!

Moral You need to read the literature!
50
Dynamic Trees - A cautionary tale

• Insertion in one pass
• As you proceed down the tree,if you find a node
  with two red children,make it red and the
  children black
• This doesnt alter the number of black nodes in
  any path
• If the parent of this node was red,a rotation is
  needed ...
• May need to be a single or a double rotation

51
Trees - Insertion

• Adding 4 ...

Discover two red children here
Swap colours around
52
Trees - Insertion

• Adding 4 ...

Red sequence, violates red-black property
Rotate
53
Trees - Insertion

• Adding 4 ...

Rotate
Add the 4
54
Balanced Trees - Yet more variants

• Basically the same ideas
• 2-3 Trees
• 2-3-4 Trees
• Special cases of m-way trees ... coming!
• Variable number of children per node
• A more complex implementation
• 2-3-4 trees
• Map to red-black trees
• Possibly useful to understand red-black trees

55
Key Points

• AVL Trees
• First dynamically balanced tree
• Height within 44 of optimum
• Rebalanced with rotations
• O(log n)
• Less efficient than properly coded red-black
  trees
• 2-3, 2-3-4 trees
• m-way trees - Yet more variations
• 2-3-4 trees map to red-black trees

56
m-way trees

• Only two children per node?
• Reduce the depth of the tree to O(logmn)with
  m-way trees
• m children, m-1 keys per node
• m 10 106 keys in 6 levels vs 20 for a
  binary tree
• but ........

57
m-way trees

• But you have to search through the m keys in
  each node!
• Reduces your gain from having fewer levels!
• A curiosity only?

58
B-trees

• All leaves are on the same level
• All nodes except for the root and the leaveshave
• at least m/2 children
• at most m children
• B trees
• All the keys in the nodes are dummies
• Only the keys in the leaves point to real data
• Linking the leaves
• Ability to scan the collection in orderwithout
  passing through the higher nodes

Each node is at least half full of keys
59
B-trees

• B trees
• All the keys in the nodes are dummies
• Only the keys in the leaves point to real data
• Data records kept in a separate area

60
B-trees - Scanning in order

• B trees
• Linking the leaves
• Ability to scan the collection in orderwithout
  passing through the higher nodes

61
B-trees - Use

• Use - Large Databases
• Reading a disc block is much slower than reading
  memory ( ms vs ns )
• Put each block of keys in one disc block

Physical disc blocks
62
B-trees - Insertion

• Insertion
• B-tree property block is at least half-full of
  keys
• Insertion into block with m keys
• block overflows
• split block
• promote one key
• split parent if necessary
• if root is split, tree becomes one level deeper

63
B-trees - Insertion

• Insertion
• Insert 9
• Leaf node overflows,split it
• Promote middle (8)
• Root overflows,split it
• Promote middle (6)
• New root node formed
• Height increased by 1

64
B-trees on disc

• Disc blocks
• 512 - 8k bytes
• 100s of keys
• Use binary search within the block
• Overall
• O( log n )
• Matched to hardware!
• Deletion similar
• But merge blocks to maintain B-tree property (at
  least half full)