Todays Material

1
Todays Material

• Minimum Spanning Trees
• Problem Definition Chapter 23
• Kruskals MST Algorithm Chapter 23
• Prims MST Algorithm Chapter 23

2
MST Problem Definition

• A common problem in communication networks and
  circuit design is that of connecting together a
  set of nodes (communication sites or circuit
  components) by a network of minimal total length,
  where the length is the sum of the lengths of the
  connecting wires
• We assume that the network is undirected

3
MST Problem Definition

• Notice that the connecting network must be
  acyclic since we can break any cycle without
  destroying connectivity and decrease total length
• Since the resulting connecting network is
  connected, undirected and acyclic, it is a free
  tree
• This computational problem is called the Minimum
  Spanning Tree (MST)

4
MST Formal Definition

• Given a connected, undirected graph G (V, E), a
  spanning tree is an acyclic subset of edges T lt
  E that connects all the vertices together.
• Assuming that each edge (u, v) of G has a numeric
  weight or cost, w(u, v) (may be zero or negative)
  we define the cost of a spanning tree T to be the
  sum of the edge weights in the spanning tree
• w(T) Sum_(u, v) e T w(u, v)

5
MST Example
7
8
c
d
b
9
4
2
4
Cost 37
14
e
i
11
a
6
7
10
8
f
g
h
2
1
7
8
c
d
b
9
4
2
4
Cost 37
14
e
i
11
a
6
7
10
8
f
g
h
2
1

• MST may not be unique
• But if all edge weights are distinct, then the
  MST will be unique

6
Steiner Minimum Trees (SMT)
7
8
c
d
b
9
4
2
4
Cost 27
14
e
i
11
a
6
7
10
8
f
g
h
2
1

• Given an undirected graph G (V, E) with edge
  weights and a subset of vertices V lt V, called
  the terminals, compute a connected acyclic
  subgraph of G, i.e., a free tree, that includes
  all the terminals
• Note that the MST is an SMT where V V, that
  is, all the vertices must be part of the tree
• Used by multicasting algorithms
• MST computation is easy, while SMT is NP-complete

7
Computing MST

• We will present two greedy algorithms for
  computing MSTs
• Kruskals Algorithm
• Prims Algorithm

8
Some Basic Facts about Free Trees

• Notice An MST is a free tree
• A free tree with n vertices has exactly n-1
  edges
• There exists a unique path between any two
  vertices of a free tree
• Adding any edge to a free tree creates a unique
  cycle. Breaking any edge on this cycle restores a
  free tree

9
Greedy MST Algorithms

• Let G (V, E) be an undirected, connected graph
  whose edges have numeric edge weights, which may
  be positive, negative or zero
• The intuition behind the greedy algorithms is
  simple
• Maintain a subset of edges A, initially empty
• Add edges one at a time, until A equals MST
• A lt- EmptySet
• One by one add a safe edge to A

10
When is an Edge Safe?

• Let S be a subset of the vertices S lt V.
• A cut (S, V-S) is just a bipartition of the
  vertices into two disjoint subsets
• An edge (u, v) crosses the cut if one endpoint is
  in S and the other is in V-S.

c
d
b
S
V-S
e
i
a
f
g
h
11
MST Lemma

• Let G (V, E) be a connected, undirected graph
  with real-valued weights on the edges.
• Let (S, V-S) be any
• Let (u, v) be an edge that crosses this cut has
  the minimum weight, i.e., (u, v) is the light
  edge
• Then edge (u, v) is a safe edge

c
d
b
S
V-S
e
i
a
f
g
h
12
MST Lemma Proof
9
4
4
4
u
u
v
u
v
v
7
6
x
x
x
8
8
y
y
y
T (u,v)
A
T T (x,y) (u,v)

• Let us assume that all edge weights are distinct
• Let T denote the MST
• If T contains (u, v) we are done
• If not, there must be another edge (x, y) that
  crosses the cut and is part of the MST
• Let us now add (u, v) to T, thus creating a cycle
• Now remove (x, y). We get another spanning tree,
  call it T

13
MST Lemma Proof
9
4
4
4
u
u
v
u
v
v
7
6
x
x
x
8
8
y
y
y
T (u,v)
A
T T (x,y) (u,v)

• We have w(T) w(T) w(x, y) w(u, v)
• Since (u, v) is the lightest edge crossing the
  cut, we have w(u, v) lt w(x, y). Thus w(T) lt w(T)
  contradicting the assumption that T was an MST

14
Kruskals Algorithm

• Recall Generic Greedy MST Algorithm
• Maintain a subset of edges A, initially empty
• Add edges one at a time, until A equals MST
• A lt- EmptySet
• One by one add a safe edge to A
• Kruskals Algorithm works by attempting to add
  edges to A in increasing order of weight
  (lightest edges first).
• If the next edge does not induce a cycle among
  the current set of edges, then it is added to A
• If it does, then this edge is passed over, and we
  consider the next edge in order

15
Kruskals Algorithm

• Kruskal(G (V, E))
• A // Initially
  A is empty
• Sort E in increasing order by weight w
• for each ((u, v) from the sorted list)
• if (adding (u, v) does not induce a cycle in
  A) Add (u, v) to A
• //end-if
• //end-for
• return A // A is the MST
• //end-Kruskal

16
Kruskals Algorithm Example
7
8
c
d
b
9
4
2
4
14
e
a
i
11
6
7
10
8
f
g
h
2
1

• As this algorithm runs, the edges of A will
  induce a forest on the vertices.
• As the algorithm continues, the trees of this
  forest are merged together, until we have a
  single tree containing all the vertices

17
Kruskals Algorithm Correctness

• Observe that the strategy leads to a correct
  algorithm. Why?
• Consider the edge (u, v) that Kruskals algorithm
  seeks to add next and suppose that this edge does
  not induce a cycle in A
• Let A denote the tree of the forest A that
  contains vertex u
• Consider the cut (A, V-A)
• Every edge crossing the cut is not in A, and (u,
  v) is the light edge across the cut (because any
  lighter edge would have been considered earlier
  by the algorithm)
• Thus by the MST Lemma, (u, v) is safe

18
Kruskals Algo Implementation

• The only tricky part in the algorithm is how to
  detect whether the addition of an edge will
  create a cycle in A
• We can perform a DFS on subgraph induced by the
  edges of A, but this will take too much time
• We want a fast test that tells us whether u and v
  are in the same tree of A

19
Kruskals Algo Implementation

• This can be done by a data structure called the
  disjoint set UNION-FIND, which supports 3
  operations
• Create_Set(u) Create a set containing a single
  item u
• Find_Set(u) Find the set that contains a given
  item u
• Union(u, v) Merge the set containing u and the
  set containing v into a common set
• It is sufficient to know that each of these
  operations can be performed in O(logn) time
• In fact, there are faster implementations

20
Kruskals Algo Implementation

• In Kruskals Algorithm, the vertices of the graph
  will be the elements to be stored in the sets
• The sets will be vertices in each tree of A
• The set A can be stored as a simple list of edges

21
Kruskals Algo Final Version

• Kruskal(G (V, E))
• A // Initially
  A is empty
• for each (u in V)
• Create_Set(u) // Create a set for
  each vertex
• Sort E in increasing order by weight w //
  O(eloge)
• 
  // e lt n2? loge lt 2logn
• for each ((u, v) from the sorted list) //
  O(elogn)
• if (Find_Set(u) ! Find_Set(v)) // if u
  and v are in
• Add (u, v) to A // different
  trees
• Union(u, v)
• //end-if
• //end-for
• return A
• //end-Kruskal

22
Prims Algorithm

• Prims Algorithm is another greedy algorithm for
  MST
• Differs from Kruskals Algorithm only in how it
  selects the next safe edge to add at each step
• Its running time is essentially the same as
  Prims O((ne)logn)

23
Why study Prims Algorithm?

• 2 reasons to study Prims Algorithm
• To show that there is more than one way to solve
  a problem
• An important lesson to learn in algorithm design
• Prims algorithm looks very much like another
  Greedy Algorithm, called Dijkstras Algorithm
  used to compute shortest paths
• Thus not only Prims algorithm is a different way
  to solve a different problem, it is also the same
  way to solve a different problem

24
Prims Algo Growing the Tree

• Kruskals Algorithm worked by ordering the edges,
  and inserting them one by one into the MST,
  taking care never to introduce a cycle
• Intuitively, Kruskals works by merging or
  splicing two trees together until all vertices
  are in the same tree
• Prims algorithm builds the tree up by adding
  leaves one at a time to the current tree
• We start with a root vertex r (any vertex)
• At any time, the subset of edges A forms a single
  tree
• In Kruskals it formed a forest
• We look to add a single vertex as a leaf to the
  tree

25
Prims Algo Growing the Tree
12
12
10
10
6
6
r
r
7
7
11
3
u
u
4
9
5
5
Current Tree Vertices (S)
After u is added
26
Prims Algo Growing the Tree

• Observe that we consider the set of vertices S
  current part of the tree, and its complement
  (V-S)
• We have a cut of the graph (S, V-S)
• Which edge should we add next?
• MST Lemma tells us that it is safe to add the
  light edge

27
Prims Algorithm Example
7
8
9
4
2
4
14
11
6
7
10
8
2
1
28
Prims Algo Implementation

• The key question in the efficient implementation
  of Prims algorithm is
• how to update the cut efficiently
• how to determine the light edge quickly

29
Prims Algo Implementation

• To do this, we will make use of your favorite DS,
  the priority queue or a heap.
• Recall that a heap stores a set of items, where
  each item is associated with a key value, and
  supports 3 operations (all can be implemented in
  O(logn))
• Insert(Q, u, key) Insert u with key value key in
  Q
• u Extract_Min(Q) Extract the item with the
  minimum key value in Q
• Decrease_Key(Q, u, new_key) Decrease the value
  of us key value to new_key

30
Prims Algo Implementation

• What do we store in the priority queue?
• The idea is the following
• For each vertex in u e V-S (i.e. not part of the
  current spanning tree), we associate u with a key
  value keyu, which is the weight of the lightest
  edge going from u to any vertex in S
• We also store in predu the end of this edge in
  S
• If there is no edge from u to a vertex in V-S,
  then we set its key value to infinity
• We also need to know which vertices are in S
• We do this by coloring the vertices in S black
• Here is the algorithm

31
Prims Algo Implementation

• Prim(G, w, r)
• For each (u in V) //
  Initialization
• keyu infinity
• coloru white
• //end-for
• keyr 0 //
  Start at root
• Predr nil
• Q build initial queue with all vertices
• while (Non_Empty(Q)) // Until all
  vertices in MST
• u Extract_Min(Q) // Vertex with
  lighter edge
• for each (v in Adju)
• if (colorv white (w(u, v) lt keyv))
• keyv w(u, v) // New lighter edge
  out of v
• Decrease_Key(Q, v, keyv)
• predv u
• //end-if
• //end-for
• coloru black

32
Prims Algorithm Example
7
8
i
i
4
9
4
2
4
14
i
i
11
6
7
10
8
i
i
8
2
1
7
8
8
i
9
4
2
4
14
i
i
11
6
7
10
8
i
i
8
2
1
33
Prims Algorithm Example
7
8
8
i
9
4
2
4
14
i
i
11
6
7
10
8
i
i
8
2
1
7
8
7
9
4
2
4
14
i
2
11
6
7
10
8
4
i
8
2
1
34
Prims Algorithm Example
7
8
7
9
4
2
4
14
i
2
11
6
7
10
8
4
i
8
2
1
7
8
7
9
4
2
4
14
i
11
6
7
10
8
4
6
7
2
1
35
Prims Algorithm Example
7
8
7
9
4
2
4
14
i
11
6
7
10
8
4
6
7
2
1
7
8
7
9
4
2
4
14
10
11
6
7
10
8
2
7
2
1
36
Prims Algorithm Example
7
8
7
9
4
2
4
14
10
11
6
7
10
8
2
7
2
1
7
8
7
9
4
2
4
14
10
11
6
7
10
8
1
2
1
37
Prims Algorithm Example
7
8
7
9
4
2
4
14
10
11
6
7
10
8
1
2
1
7
8
7
9
4
2
4
14
10
11
6
7
10
8
2
1
38
Prims Algorithm Example
7
8
7
9
4
2
4
14
10
11
6
7
10
8
2
1
7
8
9
4
2
4
14
9
11
6
7
10
8
2
1
39
Prims Algorithm Example
7
8
9
4
2
4
14
9
11
6
7
10
8
2
1
7
8
9
4
2
4
14
11
6
7
10
8
2
1
40
Prims Algo Running Time

• Initialization O(n)
• While loop continues n times
• Within the loop we do
• Extract_Min O(logn)
• For each incident edge, we spend potentially
  O(logn) time decreasing the key of the
  neighboring vertex
• deg(u)logn
• T(n) Sum_u e V (logn deg(u)logn)
• Sum_u e V (1 deg(u))logn
• logn Sum_u e V (1 deg(u))
• logn (n2e) O((ne)logn)