Balanced Trees (AVL and RedBlack)

1
Balanced Trees (AVL and RedBlack)
2
Binary Search Trees

• Optimal Behavior
• O(log2N) perfectly balanced tree (e.g. complete
  tree with all levels filled)
• Degenerate Behavior
• O(N) tree becomes list
• How to avoid degenerate trees?
• Keep them perfectly balanced
• Impossible
• Keep them balanced within some criteria
• Maintain balance over insertions and deletions

3
Balanced Trees

• Balanced binary trees
• AVL Tree
• Red-Black Tree
• Balanced N-ary trees
• B-Trees
• B-Trees

4
Height of BST

• A BST is not balanced
• E.g.,

5
12
23
35
65
85
5
AVL Trees

• Discovered by two Russian mathematicians,
  Adelson-Velskii and
• Landis, in 1962.
• Mathematical Definition
• If T is a a nonempty binary search tree, then T
  is an AVL tree if and
• only if its left and right subtrees are AVL trees
  and
• left height right height lt 1.
• In English Definition
• A binary search tree that is somewhat balanced,
  but not
• necessarily complete or full.

6
AVL Trees?
7
AVL Trees

• Representation
• Use the linked representation, with an addition
  to the
• node class. The addition is a balance factor
  (bf), which
• is defined as the following for a node x.
• bf(x) height of xs left subtree height of
  xs right subtree
• The possible balance factors are 1, 0, and 1.

8
AVL Trees

• Insertion
• Using the standard insertion logic for a binary
  search
• tree could cause problems with an AVL tree
  because
• its possible that some of the properties could
  be
• violated.
• In other words, its possible that the tree could
  become
• unbalanced.

9
AVL Trees
Adding the node DUS results in the tree below
10
AVL Trees

• When an insertion is performed, the balancing
  information for ALL of the nodes on the path back
  to the root must be updated.
• If the AVL properties were destroyed, they can be
  fixed by a simple modification to the tree, which
  is known as a rotation.

11
AVL Trees

• To fix the tree, start at the inserted node and
  trace its
• path back to the root, stopping at the first node
  that
• violates the AVL property (call this node A).
• There are four possible violations that could
  occur
• 1. Inserted into the left subtree of As left
  child
• 2. Inserted into the right subtree of As right
  child
• 3. Inserted into the right subtree of As left
  child
• 4. Inserted into the left subtree of As right
  child

12
AVL Trees

• 1. Inserted into the left subtree of As left
  child (lets call this B).
• This problem is fixed by applying a single right
  rotation.
• - Change the parent of A to point to B
• - Make As left link equal to Bs right link
• - Make Bs right link point to A

13
AVL Trees

• 2. Inserted into the right subtree of As right
  child (lets call this B).
• This problem is fixed by applying a single left
  rotation.
• - Change the parent of A to point to B
• - Make As right link equal to Bs left link
• - Make Bs left link point to A

14
AVL Trees

• Lets build a tree using the nodes 3, 2, 1, 4, 5,
  6, 7.

15
AVL Trees

• Lets build a tree using the nodes 3, 2, 1, 4, 5,
  6, 7.

16
AVL Trees

• Lets build a tree using the nodes 3, 2, 1, 4, 5,
  6, 7.

17
AVL Trees

• Lets build a tree using the nodes 3, 2, 1, 4, 5,
  6, 7.

18
AVL Trees

• Lets build a tree using the nodes 3, 2, 1, 4, 5,
  6, 7.

19
AVL Trees

• Lets build a tree using the nodes 3, 2, 1, 4, 5,
  6, 7.

20
AVL Trees

• 3. Inserted into the right subtree of As left
  child (lets call this B).
• This problem is fixed by applying a left-right
  rotation.
• - Make As left link equal to Bs right link
  (call this C)
• - Make Bs right link equal to Cs left link
• - Make Cs left link equal to B
• - Change the parent of A to point to C
• - Make As left link equal to Cs right link
• - Make Cs right link point to A

21
AVL Trees

• 4. Inserted into the left subtree of As right
  child (lets call this B).
• This problem is fixed by applying a right-left
  rotation.
• - Make As right link equal to Bs left link
  (call this C)
• - Make Bs left link equal to Cs right link
• - Make Cs right link equal to B
• - Change the parent of A to point to C
• - Make As right link equal to Cs left link
• - Make Cs left link point to A

22
AVL Trees

• Lets build onto the previous example by adding
  the nodes
• 17, 16, 15, 13, 14.

Right-left rotation
23
AVL Trees

• Lets build onto the previous example by adding
  the nodes
• 17, 16, 15, 13, 14.

Right-left rotation
24
AVL Trees

• Lets build onto the previous example by adding
  the nodes
• 17, 16, 15, 13, 14.

Right-left rotation
25
AVL Trees

• Lets build onto the previous example by adding
  the nodes
• 17, 16, 15, 13, 14.

Right-left rotation
26
AVL Trees

• Lets build onto the previous example by adding
  the nodes
• 17, 16, 15, 13, 14.

4
Right-left rotation
2
6
1
3
5
7
27
AVL Trees

• Lets build onto the previous example by adding
  the nodes
• 17, 16, 15, 13, 14.

Right-left rotation
28
AVL Trees

• Lets build onto the previous example by adding
  the nodes
• 17, 16, 15, 13, 14.

single left rotation
29
AVL Trees

• Lets build onto the previous example by adding
  the nodes
• 17, 16, 15, 13, 14.

Left - right rotation
14
15
30
Red-Black Trees

• Similar idea to AVL Trees
• Similar worse case time complexity
• Keep tree somewhat balanced
• not as rigidly balanced as AVL
• A Red-Black tree with n nodes has height at most
  2log(n1), i.e., most of the common operations
  are performed in O(log n).
• Longest path is no more than twice as long as the
  shortest branch

31
Red-Black Trees

• A red-black tree is an extended binary tree
  (every node has two children or is a leaf) which
  satisfies the following properties
• The root is black (if non-empty)
• Root property
• If a node is red, its parent must be black
• Red property
• Every path from the root to a leaf/single-child
  node contains the same number of black nodes.
  This number is called the black-height of the
  tree.
• Path property

32
Red-Black?
33
Red-Black?
yes
no
yes
34
Red-Black?
35
Red-Black?
no
36
Properties Revisited

• The root is black (if non-empty)
• Root property
• If a node is red, its parent must be black
• Red property
• Every path from the root to a leaf/single-child
  node contains the same number of black nodes.
  This number is called the black-height of the
  tree.
• Path property

37
Adding null leaf nodes

• Every node has 2 children
• Dont include null nodes in the black-height of
  the tree

38
Properties Revisited

• Objective
• Restore red property violation while preserving
  other properties (i.e. path property).
• Techniques re-color and/or rotate.

39
Property Restoration

• Invariant
• There is exactly one red node x in the tree whose
  parent may be red.
• When a new node is added, it is always added as a
  red node
• Strategy
• Fix the violation of red property at x.
• This may violate the condition at some ancestor.
• Continue fixing the property at the ancestor
  (ancestor becomes x)

40
Property Restoration

• Terminology
• x is the current node in violation of the red
  property
• It is red, and its parent is also red
• parent(x) is its parent
• parent(parent(x) ) is its grandparent (i.e.
  grandparent(x))
• The other sibling of parent(x) is its uncle (i.e.
  uncle(x))

41
Property Restoration

• There are several cases
• Case 1
• X is the root
• Case 2
• Parent(x) is black
• Case 3
• Parent(x) and Uncle(x) are both red
• Case 4
• Parent(x) is red, but Uncle(x) is black
• Multiple parts of this case

42
Property Restoration

• Case 1
• X is the root
• Change its color to black.

x
x
43
Property Restoration

• Case 2
• Parent(x) is black
• Dont do anything

p
p
x
x
44
Property Restoration

• Case 3
• Parent(x) and Uncle(x) are both red
• Change the color of the parent and its uncle
  to black.
• Change the color of the grandparent node to
  red, and repeat with x grandparent(x)

g
g
u
p
u
p
x
x
45
Property Restoration

• Case 4
• Parent(x) is red, but Uncle(x) is black
• There are 4 different cases, much like the 4
  different AVL cases
• left-left left-right
  right-left right-right

g
g
g
g
u
p
u
p
p
u
p
u
x
x
x
x
46
Property Restoration

• Case 4 cont
• left-left and right-right are symmetric
• left-right and right-left are symmetric
• left-left left-right
  right-left right-right

g
g
g
g
u
p
u
p
p
u
p
u
x
x
x
x
47
Property Restoration

• Case 4 left-left
• First rotate the parent RIGHT
• Then recolor by switching colors of P and
  G
• Case 4 right-right is symmetric simply rotate
  LEFT instead

g
p
p
u
p
g
x
g
x
3
x
u
1
4
5
2
3
u
1
2
3
1
2
4
5
4
5
48
Property Restoration

• Case 4 left-right
• First rotate x LEFT (shown below)
• then rotate the parent RIGHT and recolor
  by switching colors of P and G
• Case 4 right-left is symmetric simply rotate
  RIGHT then LEFT

g
g
u
p
u
x
1
x
p
4
5
3
4
5
1
2
49
Property Restoration

• Case 4 left-right
• First rotate x LEFT (shown previously)
• then rotate the parent RIGHT and recolor by
  switching colors of X and G (just like left-left
  case but where p and x switch rolls)
• Case 4 right-left is symmetric simply rotate
  RIGHT then LEFT

g
x
x
u
x
g
p
g
p
3
p
u
1
4
5
2
3
u
1
2
3
1
2
4
5
4
5
50
Insertion Complexity

• Observations
• Every one of the three cases take constant time.
• Case 3 brings x two steps closer to the root and
  only re-colors nodes without any rotations.
• For the multiple cases in 4, one or two rotations
  plus a re-coloring are required and we are done.
• Lemma
• An insertion into a red-black tree with n nodes
  takes O(log2 n) time and performs at most two
  rotations.

51
Insertion Example

• Snap-shots taken from
• http//www.ececs.uc.edu/franco/C321/html/RedBlack
  /redblack.html
• Note their rules are numbered differently than
  our
• Insert 50, 30, 70 (recall new nodes always insert
  as red)
• Rule 1 for 50, Rule 2 for 30 and 70
• Null nodes (all black are not shown)

52
Insertion Example

• Consider the following actions
• Insert 50, 30, 70, 65
• Red property violation ? re-color (Case 3 Parent
  Uncle are red)
• Re-color parent and uncle black while re-coloring
  grandparent red

53
Insertion Example

• Continue with grandparent as X
• Root must be black! (Rule 1)
• color it black

54
Insertion Example

• Consider the following actions
• Insert 50, 30, 70, 65, 80
• No violations (rule 2)

55
Insertion Example

• Consider the following actions
• Insert 50, 30, 70, 65, 80, 75
• Case 3 Uncle parent are both red
• re-color them black and the grandparent red

56
Insertion Example

• Consider the following actions
• Insert 50, 30, 70, 65, 80, 75
• Continue on recursively with grandparent being x
• Case 2 no violations

57
Insertion Example

• Consider the following actions
• Insert 50, 30, 70, 65, 80, 75, 68, 67
• Case 4 Parent uncle diff. color right-left
• Balance first rotation 67 right, then left

58
Insertion Example

• Consider the following actions
• Insert 50, 30, 70, 65, 80, 75, 68, 67
• Flip colors of G (65) and X (67)

59
Insertion Example

• Consider the following actions
• Insert 50, 30, 70, 65, 80, 75, 68, 67

60
Insertion Example

• Consider the following actions
• Insert 50, 30, 70, 65, 80, 75, 68, 67, 55
• Case 3 parent uncle same color ? re-color them
  and grandparent

61
Insertion Example

• Consider the following actions
• Insert 50, 30, 70, 65, 80, 75, 68, 67, 55
• Recursively continue with 67 being the new x
• Case 4 70 and 30 are diff colors right-left
• First rotate 67 right

62
Insertion Example

• Consider the following actions
• Insert 50, 30, 70, 65, 80, 75, 68, 67, 55
• Then rotate 67 left

63
Insertion Example

• Consider the following actions
• Insert 50, 30, 70, 65, 80, 75, 68, 67, 55
• Then flip colors of 67 and 50

64
Insertion Example

• Consider the following actions
• Insert 50, 30, 70, 65, 80, 75, 68, 67, 55
• Done!

65
Benefits/Usage of Red-Black Trees

• Properties
• O(log2N) and T(log2N) for search, insert, remove
  operations
• Usage
• Underlying implementation structure for hash
  maps, hash sets, multimaps, multisets in Standard
  Template Library
• New scheduler for Linux

66
Comparison of Red-Black and AVL

• Both are Binary Search Trees
• Both have the same worst case big-O time
  complexity for
• Search
• Insert
• Delete
• AVL trees are more rigidly balanced
• AVL trees are slightly faster for lookup
• AVL trees are more costly for insert and delete
• Most people are using Red-Black trees in their
  implementations