Minimum Spanning Trees

1
Minimum Spanning Trees

• Problem Description
• Why Compute MST?
• MST in Unweighted Graphs
• MST in Weighted Graphs
• Kruskals Algorithm
• Prims Algorithm

2
Spanning Tree Definition

• A Spanning tree a subset of edges from a
  connected graph that
• touches all vertices in the graph (spans the
  graph)
• forms a tree (is connected and contains no cycles)

A
9
5
1
C
B
2
4
7
D
A weighted graph
Three spanning trees

• Minimum Spanning Tree Spanning tree with the
  least total edge cost

3
MST Problem

• Given a weighted, undirected graph G(V, E),
  compute the minimum cost spanning tree
• MST may not be unique (unless all edge weights
  are distinct)

7
8
c
d
b
9
4
2
4
Cost 37
14
e
i
11
a
6
7
10
8
f
g
h
2
1
4
Why Compute MST?

• Minimize length of gas pipelines between cities
• Find cheapest way to wire a house (with minimum
  cable)
• Find a way to connect various routers on a
  network that minimizes total delay
• Eliminate loops in a switched LAN so that
  broadcast packets will not circle around
  indefinitely

5
Some Basic Facts about Free Trees

• Notice An MST is a free tree
• A free tree with n vertices has exactly n-1
  edges
• There exists a unique path between any two
  vertices of a free tree
• Adding any edge to a free tree creates a unique
  cycle. Breaking any edge on this cycle restores a
  free tree

6
Computing MSTUnweighted Graphs

• What if the graph is unweighted or all edge
  weights are equal?
• Simply run BFS or DFS and the resulting tree is a
  MST

7
Computing MST Weighted Graphs

• We will present two greedy algorithms for
  computing MSTs in weighted graphs
• Kruskals Algorithm
• Prims Algorithm
• A greedy algorithm
• always makes choices that currently seem the best
• Short-sighted no consideration of long-term or
  global issues
• Locally optimal does not always mean globally
  optimal. but works in some cases, e.g., MST,
  shortest-paths, Huffman coding,

8
Greedy MST Algorithms

• Let G (V, E) be an undirected, connected graph
  whose edges have numeric edge weights, which may
  be positive, negative or zero
• The intuition behind the greedy algorithms is
  simple

• Maintain a subset of edges A, initially empty
• Add edges to A one-by-one until A equals the MST
• A lt- EmptySet
• One-by-one add a safe edge to A, until A equals
  the MST

9
When is an Edge Safe?

• Let S be a subset of the vertices S lt V.
• A cut (S, V-S) is just a bipartition of the
  vertices into two disjoint subsets
• An edge (u, v) crosses the cut if one endpoint is
  in S and the other is in V-S.

c
d
b
S
V-S
e
i
a
f
g
h
10
MST Lemma

• Let G (V, E) be a connected, undirected graph
  with real-valued weights on the edges.
• Let (S, V-S) be any cut
• Let (u, v) be an edge that crosses this cut has
  the minimum weight, i.e., (u, v) is the light
  edge
• Then edge (u, v) is a safe edge

c
d
b
S
V-S
e
i
a
f
g
h
11
MST Lemma Proof
9
4
4
4
u
u
v
u
v
v
7
6
x
x
x
8
8
y
y
y
T (u,v)
A
T T (x,y) (u,v)

• Let us assume that all edge weights are distinct
• Let T denote the MST
• If T contains (u, v) we are done
• If not, there must be another edge (x, y) that
  crosses the cut and is part of the MST
• Let us now add (u, v) to T, thus creating a cycle
• Now remove (x, y). We get another spanning tree,
  call it T

12
MST Lemma Proof
9
4
4
4
u
u
v
u
v
v
7
6
x
x
x
8
8
y
y
y
T (u,v)
A
T T (x,y) (u,v)

• We have w(T) w(T) w(x, y) w(u, v)
• Since (u, v) is the lightest edge crossing the
  cut, we have w(u, v) lt w(x, y). Thus w(T) lt w(T)
  contradicting the assumption that T was an MST

13
Kruskals Algorithm

• Generic MST Algorithm
• A lt- EmptySet
• One-by-one add a safe edge to A, until A equals
  the MST

• Kruskals Algorithm works by attempting to add
  edges to A in increasing order of weight
  (lightest edges first).
• If the next edge does not induce a cycle among
  the current set of edges, then it is added to A
• If it does, then this edge is passed over, and we
  consider the next edge in order

14
Kruskals Algorithm

• Kruskal(G (V, E))
• A // Initially A is empty
• Sort E in increasing order by weight w
• for each ((u, v) from the sorted list)
• if (adding (u, v) does not induce a cycle in
  A) Add (u, v) to A
• //end-if
• //end-for
• return A // A is the MST
• //end-Kruskal

15
Kruskals Algorithm Example
7
8
c
d
b
9
4
2
4
14
e
a
i
11
6
7
10
8
f
g
h
2
1
Sorted Edge List
(i, h)
(i, h)
(i, g)
(i, g)
(a, b)
(c, f)
(a, b)
(c, f)
(h, g)
(i, c)
(g, f)
(h, g)
(i, c)
(g, f)
(d, f)
(d, f)
(e, f)
(b, h)
(e, f)
(b, h)
(d, e)
(d, e)
(a, h)
(b, c)
(a, h)
(b, c)
(c, d)
(c, d)
16
Kruskals Algorithm Example
7
8
c
d
b
9
4
2
4
14
e
a
i
11
6
7
10
8
f
g
h
2
1

• As this algorithm runs, the edges of A will
  induce a forest on the vertices.
• As the algorithm continues, the trees of this
  forest are merged together, until we have a
  single tree containing all the vertices

17
Kruskals Algorithm Correctness

• Observe that the strategy leads to a correct
  algorithm. Why?
• Consider the edge (u, v) that Kruskals algorithm
  seeks to add next and suppose that this edge does
  not induce a cycle in A
• Let A denote the tree of the forest A that
  contains vertex u
• Consider the cut (A, V-A)
• Every edge crossing the cut is not in A, and (u,
  v) is the light edge across the cut (because any
  lighter edge would have been considered earlier
  by the algorithm)
• Thus by the MST Lemma, (u, v) is safe

18
Kruskals Algo Implementation

• Kruskal(G (V, E))
• A // Initially A is empty
• Sort E in increasing order by weight w
• for each ((u, v) from the sorted list)
• if (adding (u, v) does not induce a cycle in
  A) Add (u, v) to A
• //end-if
• //end-for
• return A // A is the MST
• //end-Kruskal

How to detect the cycle?

• The only tricky part in the algorithm is how to
  detect whether the addition of an edge will
  create a cycle in A

19
Kruskals Algo Implementation

• This can be done by the disjoint set (Union-Find)
  data struc., which supports 3 operations
• CreateSet(u) Create a set containing a single
  item u
• Find(u) Find the set that contains a given item
  u
• Union(u, v) Merge the set containing u and the
  set containing v into a common set
• It is sufficient to know that each of these
  operations can be performed in O(logn) time
• In fact, there are faster implementations

20
Kruskals Algo Implementation

• In Kruskals Algorithm, the vertices of the graph
  will be the elements to be stored in the sets
• The sets will be vertices in each tree of A
• The set A can be stored as a simple list of edges

21
Kruskals Algo Final Version

• Kruskal(G (V, E))
• A // Initially
  A is empty
• for each (u in V)
• CreateSet(u) // Create a set for
  each vertex
• Sort E in increasing order by weight w //
  O(eloge)
• 
  // e lt n2? loge lt 2logn
• for each ((u, v) from the sorted list) //
  O(elogn)
• if (Find(u) ! Find(v)) // if u
  and v are in
• Add (u, v) to A // different
  trees
• Union(u, v)
• //end-if
• //end-for
• return A
• //end-Kruskal

22
Kruskals Algorithm Example
(h, g)
(h, g)
7
8
c
d
b
(i, c)
(i, c)
9
4
2
4
(g, f)
(g, f)
14
e
i
a
11
6
7
(a, b)
(a, b)
10
8
f
g
(c, f)
(c, f)
h
2
1
(i, g)
(i, g)
(i, h)
(i, h)
(c, d)
(c, d)
(a, h)
(a, h)
(b, c)
(b, c)
(d, e)
(d, e)
(e, f)
(e, f)
(b, h)
(b, h)
(d, f)
(d, f)
23
Prims Algorithm

• Prims Algorithm is another greedy algorithm for
  MST
• Differs from Kruskals Algorithm only in how it
  selects the next safe edge to add at each step

24
Why study Prims Algorithm?

• 2 reasons to study Prims Algorithm
• To show that there is more than one way to solve
  a problem
• An important lesson to learn in algorithm design
• Prims algorithm looks very much like another
  Greedy Algorithm, called Dijkstras Algorithm
  used to compute shortest paths
• Thus not only Prims algorithm is a different way
  to solve a different problem, it is also the same
  way to solve a different problem

25
Prims Algorithm Pseudocode

• Prim(G (V, E))
• Start with a root vertex r (any vertex
  in the graph)
• A r
• for (i1 iltn-1 i)
• 1. Consider the cut (A, V-A)
• 2. Let (u, v) be the light edge that
  crosses the cut
• such that
  u ? A v ? V-A
• 3. Add v to A, i.e., A A U v
• //end-for
• return A // A is the MST
• //end-Prim

26
Prims Algo Growing the Tree
12
10
6
r
7
11
u
u
4
5
Current Tree Vertices (A)
27
Prims Algo Growing the Tree

• Observe that we consider the set of vertices A
  current part of the tree, and its complement
  (V-A)
• We have a cut of the graph (A, V-A)
• Which edge should we add next?
• MST Lemma tells us that it is safe to add the
  light edge

28
Prims Algorithm Example
7
8
9
4
2
4
14
11
6
7
10
8
2
1
29
Prims Algo Implementation

• Prim(G (V, E))
• A r
• for (i1 iltn-1 i)
• 1. Consider the cut (A, V-A)
• 2. Let (u, v) be the light edge that
  crosses the cut
• such that
  u ? A v ? V-A
• 3. Add v to A, i.e., A A U v
• //end-for
• return A // A is the MST
• //end-Prim

• The key question in the efficient implementation
  of Prims algorithm is
• how to update the cut efficiently
• how to determine the light edge quickly

30
Prims Algo Implementation - I

• Prim(G (V, E))
• for all u in V do coloru white
• colorr black
• for (i1 iltn-1 i)
• min INFINITY // cost of the light edge
• (x, y) (?, ?) // light edge
• for all (u, v) in E do
• if (coloru black colorv
  white
• w(u,v) lt min)
• (x, y) (u, v) // (u, v) is current
  light edge
• min w(u, v)
• //end-if
• //end-for
• colory black // Add y to A
• //end-for
• return A // A is the MST
• //end-Prim

Running Time?
O(nxe)
31
Prims Algo Implementation - II

• For faster implementation, we will make use of
  your favorite DS, the priority queue or a heap.
• Recall that a heap stores a set of items, where
  each item is associated with a key value, and
  supports 3 operations (all can be implemented in
  O(logn))
• Insert(Q, u, key) Insert u with key value key in
  Q
• u Extract_Min(Q) Extract the item with the
  minimum key value in Q
• Decrease_Key(Q, u, new_key) Decrease the value
  of us key value to new_key

32
Prims Algo Implementation

• What do we store in the priority queue?
• The idea is the following
• For each vertex in u e V-A (i.e. not part of the
  current spanning tree), we associate u with a key
  value keyu, which is the weight of the lightest
  edge going from u to any vertex in A
• We also store in predu the end of this edge in
  A
• If there is no edge from u to a vertex in V-A,
  then we set its key value to infinity
• We also need to know which vertices are in A
• We do this by coloring the vertices in A black
• Here is the algorithm

33
Prims Algo Implementation

• Prim(G, w, r)
• For each (u in V) //
  Initialization
• keyu infinity
• coloru white
• //end-for
• keyr 0 //
  Start at root
• Predr nil
• Q build initial queue with all vertices
• while (Non_Empty(Q)) // Until all
  vertices in MST
• u Extract_Min(Q) // Vertex with
  lightest edge
• for each (v in Adju)
• if (colorv white (w(u, v) lt
  keyv))
• keyv w(u, v) // New lighter
  edge out of v
• Decrease_Key(Q, v, keyv)
• predv u
• //end-if
• //end-for
• coloru black

Running Time?
O(nlogn elogn)
34
Prims Algorithm Example
7
8
8
8
8
4
4
2
9
4
8
0
0
14
11
8
7
6
8
10
8
8
8
8
1
2
35
Prims Algorithm Example
7
8
8
4
8
4
8
4
2
9
4
8
0
14
11
8
7
6
8
10
8
8
8
1
2
36
Prims Algorithm Example
7
8
8
8
7
4
8
4
2
9
4
8
2
0
14
11
8
7
6
8
10
8
4
8
8
1
2
37
Prims Algorithm Example
7
8
7
4
8
4
2
9
2
4
2
0
14
11
8
7
6
8
10
4
8
8
6
7
1
2
38
Prims Algorithm Example
7
8
7
4
8
4
2
9
4
2
0
14
11
8
10
7
6
8
10
4
4
7
6
2
1
2
39
Prims Algorithm Example
7
8
7
4
8
4
2
9
4
2
0
14
11
10
7
6
8
10
4
7
2
1
2
1
2
40
Prims Algorithm Example
7
8
7
4
8
4
2
9
4
2
0
14
11
10
7
6
8
10
4
1
2
1
1
2
41
Prims Algorithm Example
7
8
7
7
4
8
4
2
9
4
2
0
14
11
10
9
7
6
8
10
4
1
2
1
2
42
Prims Algorithm Example
7
8
7
4
8
4
2
9
4
2
0
14
11
9
9
7
6
8
10
4
1
2
1
2
43
Prims Algorithm Example
7
8
7
4
8
4
2
9
4
2
0
14
11
9
7
6
8
10
4
1
2
1
2
prev pointers define the MST as an inverted tree
rooted at r