Minimum Spanning Tree Problem

1
Minimum Spanning Tree Problem
Topic 10
ITS033 Programming Algorithms
Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program,
Image and Vision Computing Lab. School of
Information, Computer and Communication
Technology (ICT) Sirindhorn International
Institute of Technology (SIIT) Thammasat
University http//www.siit.tu.ac.th/bunyaritbunya
rit_at_siit.tu.ac.th02 5013505 X 2005
2
ITS033
Midterm

• Topic 01 - Problems Algorithmic Problem Solving
• Topic 02 Algorithm Representation Efficiency
  Analysis
• Topic 03 - State Space of a problem
• Topic 04 - Brute Force Algorithm
• Topic 05 - Divide and Conquer
• Topic 06 - Decrease and Conquer
• Topic 07 - Dynamics Programming
• Topic 08 - Transform and Conquer
• Topic 09 - Graph Algorithms
• Topic 10 - Minimum Spanning Tree
• Topic 11 - Shortest Path Problem
• Topic 12 - Coping with the Limitations of
  Algorithms Power
• http//www.siit.tu.ac.th/bunyarit/its033.php
• http//www.vcharkarn.com/vlesson/7

3
Outline

• Minimum Spanning Trees
• Definitions
• A crucial fact
• The Prim-Jarnik Algorithm
• Kruskal's Algorithm

4
Minimum Spanning Tree Problem
Topic 10.1
ITS033 Programming Algorithms
Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program,
Image and Vision Computing Lab. School of
Information, Computer and Communication
Technology (ICT) Sirindhorn International
Institute of Technology (SIIT) Thammasat
University http//www.siit.tu.ac.th/bunyaritbunya
rit_at_siit.tu.ac.th02 5013505 X 2005
5
Concrete example
Imagine 1. You wish to connect all the
computers in an office building using the least
amount of cable
6
Concrete example
Imagine 2. If you want to build a railway
network to all provinces in Thailand with
minimum cost, i.e., minimum total length of the
rail way.
7
(No Transcript)
8
Imagine 3. Or a network of super-highway to
connect all the cities of Bangkok with minimum
cost
9
Concrete example
Imagine 4. We want to connect electricity to
all of the houses in the whole village with
minimum total of cable length
10
Minimum Spanning Tree Problem
Each vertex in a graph G represents a computer,
a province or a house Each edge represents the
amount of cable, length of a railway needed to
connect all vertices Thats a
weighted graph problem !! A problem of finding
A Minimum Spanning Tree
11
Minimum Spanning Tree (MST)

• Given a connected, undirected graph, a spanning
  tree of that graph is a sub-graph which is a tree
  and connects all the vertices together. A single
  graph can have many different spanning trees.
• We can assign a weight to a spanning tree by
  computing the sum of the weights of the edges in
  that spanning tree.
• A minimum spanning tree or minimum weight
  spanning tree is then a spanning tree with weight
  less than or equal to the weight of every other
  spanning tree.

ORD
10
1
PIT
DEN
6
7
9
3
DCA
STL
4
5
8
2
DFW
ATL
12
We are interested in Finding a tree T that
contains all the vertices of a graph G (spanning
tree) and has the least total weight over
all. such trees is minimum-spanning tree (MST)
13
Minimum Spanning Tree
14
Properties
Possible multiplicity There may be several
minimum spanning trees of the same weight in
particular, if all weights are the same, every
spanning tree is minimum.
15
Properties
Uniqueness If each edge has a distinct weight
then there will only be one, unique minimum
spanning tree. Minimum-cost subgraph If the
weights are non-negative, then a minimum spanning
tree is in fact the minimum-cost subgraph
connecting all vertices, since subgraphs
containing cycles necessarily have more total
weight. Cycle property For any cycle C in the
graph, if the weight of an edge e of C is larger
than the weights of other edges of C, then this
edge cannot belong to a MST.
16
Cycle Property

• Cycle Property
• Let T be a minimum spanning tree of a weighted
  graph G
• Let e be an edge of G that is not in T and C let
  be the cycle formed by e with T
• For every edge f of C, weight(f) ? weight(e)
• Proof
• By contradiction
• If weight(f) gt weight(e) we can get a spanning
  tree of smaller weight by replacing e with f

Replacing f with e yieldsa better spanning tree
17
Partition Property
U
V
7
f

• Partition Property
• Consider a partition of the vertices of G into
  subsets U and V
• Let e be an edge of minimum weight across the
  partition
• There is a minimum spanning tree of G containing
  edge e
• Proof
• Let T be an MST of G
• If T does not contain e, consider the cycle C
  formed by e with T and let f be an edge of C
  across the partition
• By the cycle property, weight(f) ? weight(e)
• Thus, weight(f) weight(e)
• We obtain another MST by replacing f with e

4
9
5
2
8
3
8
e
7
Replacing f with e yieldsanother MST
U
V
7
f
4
9
5
2
8
3
8
e
7
18
Topic 10.2
ITS033 Programming Algorithms
Prims Algorithm
Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program,
Image and Vision Computing Lab. School of
Information, Computer and Communication
Technology (ICT) Sirindhorn International
Institute of Technology (SIIT) Thammasat
University http//www.siit.tu.ac.th/bunyaritbunya
rit_at_siit.tu.ac.th02 5013505 X 2005
19
Try to use exhaustive search ?

• If we were to try an exhaustive-search approach
  to constructing a minimum spanning tree, we would
  face two serious obstacles.
• 1) the number of spanning trees grows
  exponentially with the graph size (at least for
  dense graphs).
• 2) generating all spanning trees for a given
  graph is not easy

20
Greedy Approach

• The greedy approach suggests constructing a
  solution through a sequence of steps, each
  expanding a partially constructed solution
  obtained so far, until a complete solution to the
  problem is reached. On each stepand this is the
  central point of this techniquethe choice made
  must be
• feasible, i.e., it has to satisfy the problems
  constraints.
• locally optimal, i.e., it has to be the best
  local choice among all feasible choices available
  on that step.
• irrevocable, i.e., once made, it cannot be
  changed on subsequent steps of the algorithm.

21
Prims Algorithm

• Prims algorithm constructs a minimum spanning
  tree through a sequence of expanding subtrees.
• The initial subtree in such a sequence consists
  of a single vertex selected arbitrarily from the
  set V of the graphs vertices.
• On each iteration, we expand the current tree in
  the greedy manner by simply attaching to it the
  nearest vertex not in that tree.
• The algorithm stops after all the graphs
  vertices have been included in the tree being
  constructed.

22
Prims Algorithm
23
Prims Algorithm

• The nature of Prims algorithm makes it necessary
  to provide each vertex not in the current tree
  with the information about the shortest edge
  connecting the vertex to a tree vertex.
• We can provide such information by attaching two
  labels to a vertex
• (1) The name of the nearest tree vertex and the
  length (the weight) of the corresponding edge.
• (2) Vertices that are not adjacent to any of the
  tree vertices canbe given the 8 label indicating
  their infinite distance to the tree vertices
  and a null label for the name of the nearest tree
  vertex.

24
Prims Algorithm

• We pick an arbitrary vertex s and we grow the MST
  as a cloud of vertices, starting from s
• We store with each vertex v a label d(v) the
  smallest weight of an edge connecting v to a
  vertex in the cloud At each step
• We add to the cloud the vertex u outside the
  cloud with the smallest distance label
• We update the labels of the vertices adjacent to
  u

25
Prims Algorithm

• All vertices are marked as not visited
• 2. Any vertex v you like is chosen as starting
  vertex and
• is marked as visited (define a cluster C)
• The smallest- weighted edge e (v,u), which
  connects
• one vertex v inside the cluster C with
  another vertex u outside
• of C, is chosen and is added to the MST.
• The process is repeated until a spanning tree is
  formed

26
Example
Step 1
?
D
7
2
B
4
9
?
8
5
F
2
C
8
3
8
E
A
7
7
0
27
Example
Step 2
28
Example
Step 3
29
Example
Step 4
7
D
7
2
B
4
4
9
5
5
F
2
C
8
3
8
E
A
7
7
0
30
Example (contd.)
7
D
7
2
B
4
9
4
5
5
F
2
C
8
3
8
E
A
3
7
0
Step 5
31
Example (contd.)
Step 6
7
D
7
2
B
4
4
9
5
5
F
2
C
8
3
8
E
A
3
7
0
32
Prims Algorithm
5
A
B
6
4
2
D
2
C
3
1
2
3
F
E
4
33
Prims Algorithm
5
A
B
6
4
2
D
2
C
3
1
2
3
F
E
4
34
Prims Algorithm
5
A
B
6
4
2
D
2
C
3
1
2
3
F
E
4
35
Prims Algorithm
A
B
2
D
2
C
3
1
2
3
F
E
4
36
Prims Algorithm
A
B
2
D
2
C
3
1
2
3
F
E
37
Prims Algorithm
A
B
2
D
2
C
3
1
2
3
F
E
38
Prims Algorithm
A
B
2
D
2
C
1
2
3
F
E
39
Prims Algorithm
A
B
2
D
2
C
1
2
3
F
E
40
Prims Algorithm
minimum- spanning tree
A
B
2
D
2
C
1
2
3
F
E
41
Prims Algorithm Analysis

• Efficiency of Prims algorithm depends on the
  data structures chosen for the graph and for the
  priority queue of the set V-VT whose vertex
  priorities are the distances to the nearest tree
  vertices.
• For example, if a graph is represented by its
  weight matrix and the priority queue is
  implemented as an unordered array, the
  algorithms running time will be in ?(V 2).

42
Prims Algorithm

• A min-heap is a complete binary tree in which
  every element is less than or equal to its
  children.
• Deletion of the smallest element from and
  insertion of a new element into a min-heap of
  size n are O(log n) operations, and so is the
  operation of changing an elements priority
• If a graph is represented by its adjacency linked
  lists and the priority queue is implemented as a
  min-heap, the running time of the algorithm is in
  O(E log V ).
• This is because the algorithm performs V - 1
  deletions of the smallest element and makes E
  verifications and, possibly, changes of an
  elements priority in a min-heap of size not
  greater than V . Each of these operations, as
  noted earlier, is a O(log V ) operation. Hence,
  the running time of this implementation of Prims
  algorithm is in
• (V - 1 E)O(log V )
  O(E log V )

43
Prim Minimum Spanning Tree Exercise
44
Kruskals Algorithm
Topic 10.3
ITS033 Programming Algorithms
Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program,
Image and Vision Computing Lab. School of
Information, Computer and Communication
Technology (ICT) Sirindhorn International
Institute of Technology (SIIT) Thammasat
University http//www.siit.tu.ac.th/bunyaritbunya
rit_at_siit.tu.ac.th02 5013505 X 2005
45
Minimum Spanning Tree Problem

• Kruskal's algorithm is an algorithm in graph
  theory that finds a minimum spanning tree for a
  connected weighted graph.
• If the graph is not connected, then it finds a
  minimum spanning forest (a minimum spanning tree
  for each connected component).
• Kruskal's algorithm is also an example of a
  greedy algorithm.

46
Minimum Spanning Tree Problem

• It works as follows
• Each vertex is in its own cluster
• 2. Take the edge e with the smallest weight
• - if e connects two vertices in
  different clusters,
• then e is added to the MST and the two
  clusters,
• which are connected by e, are merged
  into a single cluster
• - if e connects two vertices, which are
  already in the same
• cluster, ignore it
• 3. Continue until n-1 edges were selected

47
9.2 Kruskals Algorithm
48
Kruskals Algorithm Analysis

• On each of its iterations, Kruskals algorithm
  has to check whether the addition of the next
  edge to the edges already selected would create a
  cycle.
• A new cycle is created if and only if the new
  edge connects two vertices already connected by a
  path, i.e., if and only if the two vertices
  belong to the same connected component
• Each connected component of a subgraph generated
  by Kruskals algorithm is a tree because it has
  no cycles.

49
Kruskals Algorithm Analysis
New edge connecting two vertices may (a) or may
not (b) create a cycle
50
Kruskals Algorithm
5
A
B
6
4
2
D
2
C
3
1
2
3
F
E
4
51
Kruskals Algorithm
5
A
B
6
4
2
D
2
C
3
1
2
3
F
E
4
52
Kruskals Algorithm
5
A
B
6
4
2
D
2
C
3
1
2
3
F
E
4
53
Kruskals Algorithm
5
A
B
6
4
2
D
2
C
3
1
2
3
F
E
4
54
Kruskals Algorithm
5
A
B
6
4
2
D
2
C
3
1
2
3
F
E
4
55
Kruskals Algorithm
5
A
B
6
4
2
D
2
cycle!!
C
3
1
2
3
F
E
4
56
Kruskals Algorithm
5
A
B
6
4
2
D
2
C
3
1
2
3
F
E
4
57
Kruskals Algorithm
5
A
B
6
4
2
D
2
C
3
1
2
3
F
E
4
58
Kruskals Algorithm
minimum- spanning tree
A
B
2
D
2
C
1
2
3
F
E
59
Kruskal Example
2704
BOS
867
849
PVD
ORD
187
740
144
JFK
1846
621
1258
184
802
SFO
BWI
1391
1464
337
1090
DFW
946
LAX
1235
1121
MIA
2342
60
Example
61
Example
62
Example
63
Example
64
Example
65
Example
66
Example
67
Example
68
Example
69
Example
70
Example
71
Example
72
Example
740
144
1846
621
184
802
1391
1464
337
1090
946
1235
1121
2342
73
Kruskals Algorithm Analysis

• The check whether two vertices belong to the same
  tree is crucial in determining running time of
  Kruskals algorithm
• There is efficient algorithm that perform this
  check gt union-find algorithm
• With an efficient union- find algorithm, the
  running time of Kruskals algorithm will be
  dominated by the time needed for sorting the edge
  weights of a given graph.
• Hence, with an efficient sorting algorithm, the
  time efficiency of Kruskals algorithm will be in
  O(E log E).

74
Data Structure for Kruskal Algortihm

• The algorithm maintains a forest of trees
• An edge is accepted it if connects distinct trees
• We need a data structure that maintains a
  partition, i.e., a collection of disjoint sets,
  with the operations

75
Kruskals Minimum Spanning Tree
76
(No Transcript)
77
(No Transcript)
78
ITS033
Midterm

• Topic 01 - Problems Algorithmic Problem Solving
• Topic 02 Algorithm Representation Efficiency
  Analysis
• Topic 03 - State Space of a problem
• Topic 04 - Brute Force Algorithm
• Topic 05 - Divide and Conquer
• Topic 06 - Decrease and Conquer
• Topic 07 - Dynamics Programming
• Topic 08 - Transform and Conquer
• Topic 09 - Graph Algorithms
• Topic 10 - Minimum Spanning Tree
• Topic 11 - Shortest Path Problem
• Topic 12 - Coping with the Limitations of
  Algorithms Power
• http//www.siit.tu.ac.th/bunyarit/its033.php
• http//www.vcharkarn.com/vlesson/7

79
End of Chapter 10

• Thank you!