Minimum Spanning Tree Algorithms

1
Minimum Spanning Tree Algorithms

• Prims Algorithm

2
What is A Spanning Tree?

• A spanning tree for an undirected graph G(V,E)
  is a subgraph of G that is a tree and contains
  all the vertices of G
• Can a graph have more than one spanning tree?
• Can an unconnected graph have a spanning tree?

a
u
b
e
v
c
f
d
3
Minimal Spanning Tree.
a
4
4

• The weight of a subgraph is the sum of the
  weights of it edges.
• A minimum spanning tree for a weighted graph is a
  spanning tree with minimum weight.
• Can a graph have more then one minimum spanning
  tree?

9
3
u
b
e
2
14
10
15
v
c
f
3
8
d
4
Example of a Problem that Translates into a MST

• The Problem
• Several pins of an electronic circuit must be
  connected using the least amount of wire.
• Modeling the Problem
• The graph is a complete, undirected graph G (
  V, E ,W ), where V is the set of pins, E is the
  set of all possible interconnections between the
  pairs of pins and w(e) is the length of the wire
  needed to connect the pair of vertices.
• Find a minimum spanning tree.

5
Greedy Choice

• We will show two ways to build a minimum spanning
  tree.
• A MST can be grown from the current spanning tree
  by adding the nearest vertex and the edge
  connecting the nearest vertex to the MST. (Prim's
  algorithm)
• A MST can be grown from a forest of spanning
  trees by adding the smallest edge connecting two
  spanning trees. (Kruskal's algorithm)

6
Notation

• Tree-vertices in the tree constructed so far
• Non-tree vertices rest of vertices

Prims Selection rule

• Select the minimum weight edge between a
  tree-node and a non-tree node and add to the tree

7
The Prim algorithm Main Idea

• Select a vertex to be a tree-node
• while (there are non-tree vertices)
• if there is no edge connecting a tree node with
  a non-tree node
• return no spanning tree
• select an edge of minimum weight between a tree
  node and a non-tree node
• add the selected edge and its new vertex to the
  tree
• return tree

8
Implementation Issues

• How is the graph implemented?
• Assume that we just added node u to the tree.
• The distance of the nodes adjacent to u to the
  tree may now be decreased.
• There must be fast access to all the adjacent
  vertices.
• So using adjacency lists seems better
• Alternatively, adjacency matrix can also be used
  easily
• How should the set of non-tree vertices be
  represented?
• The operations are
• build set
• delete node closest to tree
• decrease the distance of a non-tree node from the
  tree
• check whether a node is a non- tree node

9
Implementation Issues

• How should the set of non-tree vertices be
  represented?
• A priority queue PQ may be used with the
  priority Dv equal to the minimum distance of
  each non-tree vertex v to the tree.
• Each item in PQ contains Dv, the vertex v, and
  the shortest distance edge (v, u) where u is a
  tree node
• This means
• build a PQ of non-tree nodes with initial values
  -
• fast build heap O (V )
• building an unsorted list O(V)
• building a sorted list O(V) (special case)

10
Implementation Issues

• delete node closest to tree (extractMin)
• O(lg V ) if heap and
• O(V) if unsorted list
• O(1) sorted list
• decrease the distance of a non-tree node to the
  tree
• We need to find the location i of node v in the
  priority queue and then execute
  (decreasePriorityValue(i, p)) where p is the new
  priority
• decreasePriorityValue(i, p)
• O(lg V) for heap,
• O(1) for unsorted list
• O(V ) for sorted list (too slow)

11
Implementation Issues

• What is the location i of node v in a priority
  queue?
• Find in Heaps, and sorted lists O(n)
• Unsorted if the nodes are numbered 1 to n and
  we use an array where node v is the v item in the
  array O(1)
• Extended heap
• We will use extended heaps that contain a
  handle to the location of each node in the
  heap.
• When a node is not in PQ the handle will
  indicate that this is the case
• This means that we can access a node in the
  extended heap in O(1), and check v ?? PQ in O(1)
• Note that the handle must be updated whenever a
  heap operation is applied

12
Implementation Issues

• 2. Unsorted list
• Array implementation where node v can be accesses
  as PQv in O(1), and the value of PQv
  indicates when the node is not in PQ.

13
Prims Algorithm

• 1. for each u ?V2. do D u ? ?3. D r
  ? 0
• 4. PQ ?make-heap(D,V, )//No edges 5. T ?
  ?6.7. while PQ ? ? do 8. (u,e ) ?
  PQ.extractMin() 9. add (u,e) to T10.
  for each v ? Adjacent (u )
• // execute relaxation 11.
  do if v ?PQ w( u, v ) lt D v 12.
  then D v ? w (u, v) 13.
  PQ.decreasePriorityValue ( Dv, v, (u,v ))
  14. return T // T is a mst.

• Lines 1-5 initialize the priority queue PQ to
  contain all Vertices. Ds for all vertices except
  r, are set to infinity.
• r is the starting vertex of the TThe T so far
  is empty
• Add closest vertex and edge to current T
• Get all adjacent vertices v of u,update D of
  each non-tree vertex adjacent to u
• Store the current minimum weight edge, and
  updated distance in the priority queue

14
Prims Algorithm Initialization

• Prim (G )
• 1. for each u ?V2. do D u ? ?3. D r
  ? 0
• 4. PQ ?make-heap(D,V, )//No edges 5. T ? ?

15
Building the MST

• // solution check
• 7. while PQ ? ? do//Selection and
  feasibility 8. (u,e ) ? PQ.extractMin()
  // T contains the solution so far . 9. add
  (u,e) to T10. for each v ? Adjacent (u )
  11. do if v ? PQ w( u, v ) lt D v
  12. then D v ? w (u, v) 13.
  PQ.decreasePriorityValue
• (Dv, v, (u,v) ) 14. return T

16
Time Analysis
Using unsorted PQ Lines 1 - 6 run in O (V
) Max Size of PQ is V Count7 O (V )
Count7(8) O (V ) ? O(V ) Count7(10)
O(?deg(u ) ) O( E ) Count7(10(11)) O(1)?O( E
) Count7(10(11(12))) O(1) ?O( E
) Count7(10(13)) O( 1) ? O( E ) So total time
for Prim's Algorithm is O (V V2 E ) O (V2
) For Sparse/Dense graph O( V2 ) Note growth
rate unchanged for adjacency matrix graph
representation
1. for each u ?V2. do D u ? ?3. D r ? 0
4. PQ ?make-PQ(D,V, )//No edges 5. T ?
?6.7. while PQ ? ? do 8. (u,e ) ?
PQ.extractMin() 9. add (u,e) to T10.
for each v ? Adjacent (u ) 11. do if v ?PQ
w( u, v ) lt D v 12. then D v
? w (u, v) 13. PQ.decreasePriorityV
alue (Dv, v, (u,v)) 15. return T // T is
a mst.
17
Prim - extended HeapAfter Initialization
PQ
T
1
A
0, (A, )
2
B
6
5
C
G
?, (B, )
?, (C, )
2
3
Prim (G, r)1. for each u ?V2. do D u ?
?3. D r ? 0 4. PQ ?make-heap(D,V, )5. T
? ?
G
A B C
C 6

B 2
A 2
C 5
A 6
B 5
18
Prim - extended Heap Build tree - after
PQ.extractMin
PQ
T (A, )
A
2
1
?, (C, )
6
B
5
C
G
?, (B, )
7. while PQ ? ? do 8. (u,e) ?
PQ.extractMin()9. add (u,e) to T
2
19
Update B adjacent to A
PQ
T (A, )
A
1
2, (B, A, B)
2
6
B
5
C
G
?, (C, )
2
10. for each v ? Adjacent (u ) 11. //
relaxation operation
// relaxation 11. do if v ?PQ w( u, v ) lt
D v 12. then D v ? w (u, v) 13.
PQ.decreasePriorityValue ( Dv, v,
(u,v))
20
Update C adjacent to A
PQ
T (A, )
1
2, (B, A, B)
6, (C, A, C)
A
2
2
B
6
5
C
G
21
Build tree - after PQ.extractMin
PQ
T (A, ) (B, A, B)
6, (C, A, C)
1
A
7. while PQ ? ? do 8. (u,e) ?
PQ.extractMin()9. add (u,e) to T
2
B
6
5
C
G
22
Update C adjacent to B
PQ
T (A, )
T (A, ) (B, A, B)
5, (C, B, C)
1
A
10. for each v ? Adjacent (u ) 11. //
relaxation operation
2
B
6
5
C
G
23
Build tree - after PQ.extractMin
PQ
T (A, )
T (A, ) (B, A, B) (C, B, C)
A
7. while PQ ? ? do 8. (u,e) ?
PQ.extractMin()9. add (u,e) to T
2
B
6
5
C
G
24
Prim - unsorted list After Initialization
PQ
T
A
B
A
C
12
0, (A, )
?, (B, )
?, (C, )
B
4
5
C
G
G
A B C
C 4

B 12
Prim (G, r)1. for each u ?V2. do D u ?
?3. D r ? 0 4. PQ ?make-PQ(D,V, )5. T ?
?
A 12
C 5
A 4
B 5
25
Build tree - after PQ.extractMin
PQ
T (A, )
A
B
A
C
12
?, (B, )
?, (C, )
Null
B
4
5
C
G
7. while PQ ? ? do 8. (u,e) ?
PQ.extractMin()9. add (u,e) to T
26
Update B, C adjacent to A
PQ
T (A, )
A
A
B
C
12
12, (B, A, B)
4, (C, A, C)
Null
B
4
5
C
G
10. for each v ? Adjacent (u ) 11. //
relaxation operation
27
Build tree - after PQ.extractMin
T (A, ) (C, A, C)
A
PQ
12
B
4
5
A
B
C
C
Null
12, (B, A, B)
Null
G
7. while PQ ? ? do 8. (u,e) ?
PQ.extractMin()9. add (u,e) to T
28
Update B adjacent to C
PQ
T (A, )
T (A, ) (C, A, C)
A
12
B
4
5
A
B
C
C
Null
5, (B, C, B)
Null
G
10. for each v ? Adjacent (u ) 11. //
relaxation operation
29
Build tree - after PQ.extractMin
PQ
T (A, )
T (A, ) (C, A, C) (B, C, B)
A
12
B
4
5
A
B
C
C
Null
Null
Null
G
7. while PQ ? ? do 8. (u,e) ?
PQ.extractMin()9. add (u,e) to T
30
Lemma 1

• Let G ( V, E) be a connected, weighted
  undirected graph. Let T be a promising subset
  of E. Let Y be the set of vertices connected
  by the edges in T.
• If e is a minimum weight edge that connects a
  vertex in Y to a vertex in V - Y, then T ? e
  is promising.

Note A feasible set is promising if it can be
extended to produce not only a solution , but an
optimal solution. In this algorithm A feasible
set of edges is promising if it is a subset of a
Minimum Spanning Tree for the connected graph.
31
Outline of Proof of Correctness of Lemma 1

• T is the promising subset, and e the minimum cost
  edge of Lemma 1
• Let T ' be the MST such that T ? T '
• We will show that if e ? T' then there must be
  another MST T" such that T ?e ? T".
• Proof has 4 stages
• 1. Adding e to T', closes a cycle in T' ?e.
• 2. Cycle contains another edge e'? T' but e' ? T
• 3. T"T' ? e - e is a spanning tree
• 4. T" is a MST

32
The Promising Set of Edges Selected by Prim
MST T' but e ? T'
33
Since T' is a spanning tree, it is connected.
Adding the edge, e, creates a cycle.
T' ? e
Stage 1
u
v
e
In T' there is a path from u?Y to v ? V -Y.
Therefore the path must include another edge e'
with one vertex in Y and the other in V-Y.
Stage 2
e'
34

• If we remove e from T ? e the cycle
  disappears.
• TT ? e - e is connected. T is
  connected. Every pair of vertices connected by a
  path that does not include e is still connected
  in T. Every pair of vertices connected by a
  path that included e , is still connected in T
  because there is a path in T" T ? e - e
  connecting the vertices of e.

Stage 3
w( e ) ? w( e' )By the way Prim picks the next
edge
35

• w( e ) ? w( e' ) by the way Prim picks the next
  edge.
• The weight of T", w(T") w (T' ) w( e ) - w(
  e' ) ? w(T').
• But w (T' ) ? w(T") because T' is a MST.
• So w (T' ) w(T") and T" is a MST

Stage 4
Conclusion T ? e is promising
36
Theorem Prim's Algorithm always produces a
minimum spanning tree.

• Proof by induction on the set T of promising
  edges.Base case Initially, T ? is
  promising.Induction hypothesis The current
  set of edges T selected by Prim is
  promising.Induction step After Prim adds the
  edge e, T ? e is promising.Proof T ? e
  is promising by Lemma 1.
• Conclusion When G is connected, Prim terminates
  with T V -1, and T is a MST.