Graph Algorithms

1
Graph Algorithms

• ECE573 Data Structures and Algorithms
• Electrical and Computer Engineering Dept.
• Rutgers University
• http//www.cs.rutgers.edu/vchinni/dsa/

2
Graphs - Definitions

• Graph
• Set of vertices (nodes) and edges connecting them
• Write G ( V, E )
• V is a set of vertices
  V vi
• An edge connects two vertices e ( vi
  , vj )
• E is a set of edges
  E (vi , vj )

Vertices
Edges
3
Graphs - Definitions

• Path
• A path p, of length k, is a sequence of connected
  vertices
• p ltv0,v1,...,vkgt where (vi,vi1) Î
  E

lt i, c, f, g, h gtPath of length 5
lt a, b gtPath of length 2
4
Graphs - Definitions

• Cycle
• A graph contains no cycles if there is no path
• p ltv0,v1,...,vkgt such that v0 vk
  
  

lt i, c, f, g, i gtis a cycle
5
Graphs - Definitions

• Spanning Tree
• A spanning tree is a set of V-1 edges that
  connect all the vertices of a graph

The red path connects all vertices, so its a
spanning tree
6
Graphs - Definitions

• Minimum Spanning Tree
• Generally there is more than one spanning tree
• If a cost cij is associated with edge eij
  (vi,vj)
• then the minimum spanning tree is the set of
  edges Espan such that
• C S ( cij " eij Î Espan )
• is a minimum

The red tree is the Min ST
7
Graphs - Kruskals Algorithm

• Calculate the minimum spanning tree
• Put all the vertices into single node trees by
  themselves
• Put all the edges in a priority queue
• Repeat until weve constructed a spanning tree
• Extract cheapest edge
• If it forms a cycle, ignore itelse add it to the
  forest of trees(it will join two trees into a
  larger tree)
• Return the spanning tree

8
Kruskals Algorithm in C
Forest MinimumSpanningTree( Graph g, int n,
double costs )
Forest T Queue q Edge e T
ConsForest( g ) q ConsEdgeQueue( g, costs
) for(i0ilt(n-1)i) do
e ExtractCheapestEdge( q ) while (
!Cycle( e, T ) ) AddEdge( T, e )
return T
9
Kruskals Algorithm

• Cycle detection
• Uses a Union-find structure
• For which we need to understand a partition of a
  set
• Partition
• A set of sets of elements of a set
• Every element belongs to one of the sub-sets
• No element belongs to more than one sub-set
• Formally
• Set, S si
• Partition(S) Pi , where Pi
  si
• " siÎ S, si Î Pj
• " j, k Pj Ç Pk Æ
• S È Pj

10
Kruskals Algorithm

• Partition
• The elements of each set of a partition
• are related by an equivalence relation
• equivalence relations are
• reflexive
• transitive
• symmetric
• The sets of a partition are equivalence classes
• Each element of the set is related to every other
  element

x x
if x y and y z, then x z
if x y, then y x
11
Kruskals Algorithm

• Partitions
• In the MST algorithm,the connected vertices form
  equivalence classes
• Being connected is the equivalence relation
• Initially, each vertex is in a class by itself
• As edges are added, more vertices become related
  and the equivalence classes shrink
• Until finally all the vertices are in a single
  equivalence class

12
Kruskals Algorithm

• Representatives
• One vertex in each class may be chosen as the
  representative of that class
• We arrange the vertices in lists that lead to the
  representative
• This is the union-find structure

13
Kruskals Algorithm

• Cycle determination
• If two vertices have the same representative,they
  re already connected and adding a further
  connection between them is pointless
• Procedure
• For each end-point of the edge that youre going
  to add follow the lists and find its
  representative
• if the two representatives are equal, then the
  edge will form a cycle

14
Kruskals Algorithm in operation
All the vertices are in single element trees
Each vertex is its own representative
15
Kruskals Algorithm in operation
The cheapest edge is h-g
Add it to the forest, joining h and g into
a 2-element tree
Choose g as its representative
16
Kruskals Algorithm in operation
The next cheapest edge is c-i
Add it to the forest, joining c and i into
a 2-element tree
Choose c as its representative
Our forest now has 2 two-element trees and 5
single vertex ones
17
Kruskals Algorithm in operation
The next cheapest edge is a-b
Add it to the forest, joining a and b into
a 2-element tree
Choose b as its representative
Our forest now has 3 two-element trees and 4
single vertex ones
18
Kruskals Algorithm in operation
The next cheapest edge is c-f
Add it to the forest, merging two 2-element trees
Choose c (rep of one) as its representative
19
Kruskals Algorithm in operation
The next cheapest edge is g-i
The rep of g is c
The rep of i is also c
\ g-i forms a cycle
Its clearly not needed!
20
Kruskals Algorithm in operation
The next cheapest edge is c-d
The rep of c is c
The rep of d is d
c-d joins two trees, so we add it
.. and keep c as the representative
21
Kruskals Algorithm in operation
The next cheapest edge is h-i
The rep of h is c
The rep of i is c
\ h-i forms a cycle, so we skip it
22
Kruskals Algorithm in operation
The next cheapest edge is a-h
The rep of a is b
The rep of h is c
\ a-h joins two trees, and we add it
23
Kruskals Algorithm in operation
The next cheapest edge is b-c
But b-c forms a cycle
So add d-e instead
... and we now have a spanning tree
24
Greedy Algorithms

• At no stage did we attempt to look ahead
• We simply made the naïve choice
• Choose the cheapest edge!
• MST is an example of a greedy algorithm
• Greedy algorithms
• Take the best choice at each step
• Dont look ahead and try alternatives
• Dont work in many situations
• Try playing chess with a greedy approach!
• Are often difficult to prove
• because of their naive approach
• what if we made this other (more expensive)
  choice now and later on ..... ???

25
Proving Greedy Algorithms

• MST Proof
• Proof by contradiction is usually the best
  approach!
• Note that
• any edge creating a cycle is not needed
• Each edge must join two sub-trees
• Suppose that the next cheapest edge, ex, would
  join trees Ta and Tb
• Suppose that instead of ex we choose ez - a more
  expensive edge, which joins Ta and Tc
• But we still need to join Tb to Ta or some other
  tree to which Ta is connected
• The cheapest way to do this is to add ex
• So we should have added ex instead of ez
• Proving that the greedy approach is correct for
  MST

26
MST - Time complexity

• Initialize forest O( V )
• Sort edges O( ElogE )
• Loop
• Check edge for cycles O( V ) x
• Number of edges O( V ) O(
  V2 )
• Total
  O( VElogEV2 )
• Since E O( V2 ) O( V2 logV
  )
• Thus we would class MST as O( n2 log n ) for a
  graph with n vertices
• This is an upper bound,some improvements on this
  are known ...
• Prims Algorithm can be O( EVlogV )
  using Fibonacci heaps
• even better variants are known for restricted
  cases, such as sparse graphs ( E V )

27
Key Points

• Minimum Spanning Tree
• Spanning Tree of minimum cost
• Cost measure may be anything!
• Operation
• All vertices into single element trees,
• Sort edges,
• Add next cheapest edge unless it forms a cycle
• Finish when weve added V-1 edges
• Greedy algorithms
• Dont look ahead ..
• Make a naive choice at every step
• Sometimes difficult to prove
• Usually proof by contradiction employed

• Efficient Algorithms
• Divide and Conquer
• Dynamic
• Greedy

28
Equivalence Classes

• Cycle Determination in MST
• Relies on Union Find structure
• Lists pointing to a representative
• Vertices are partitioned into equivalence classes
• Equivalence relation a path connects them
• Edge creates a cycle if both its end-points
• have the same representative
• are in the same equivalence class

29
Program verification

• Experienced Software Engineers
• Include special values also
• Boundaries
• Identities - 0 and 1
• Never choose boundaries, identities, etc as a
  representative
• Even though this might be formally OK!
• Dont forget
• Nulls
• Specification should tell you what will happen on
  empty or degenerate data sets
• Empty lists, single node graphs, etc.
• Illegal input
• Violates pre-conditions
• Should raise assertions!

30
Graphs - Data Structures

• Vertices
• Map to consecutive integers
• Store vertices in an array
• Edges
• Adjacency Matrix
• Booleans - TRUE - edge exists FALSE - no edge
• O(V2) space
• Can be compacted
• 1 bit/entry
• If undirected, top half only

31
Graphs - Data Structures

• Edges
• Adjacency Lists
• For each vertex
• List of vertices attached to it
• For each edge
• 2 entries
• One in the list for each end
• O(E) space
• Better for sparse graphs

Undirected representation
32
Graphs - Traversing - Depth-First
struct t_graph int n_nodes graph_node
nodes int visited AdjMatrix am
static int search_index 0 void search(
graph g ) int k for(k0kltg-gtn_nodesk)
g-gtvisitedk 0 search_index 0
for(k0kltg-gtn_nodesk) if (
!g-gtvisitedk ) visit( g, k )
Graph data structure
Mark all nodes not visited
Visit all the nodes attached to node 0, then ..
33
Graphs - Traversing - Depth-First
void visit( graph g, int k ) int j
g-gtvisitedk search_index
for(j0jltg-gtn_nodesj) if ( adjacent(
g-gtam, k, j ) ) if ( !g-gtvisitedj )
visit( g, j )
Mark the order in which this node was visited
Visit all the nodes adjacent to this one
34
Graphs - Traversing - Depth-First
Adjacency List version of visit
void visit( graph g, int k ) AdjListNode
al_node g-gtvisitedk search_index
al_node ListHead( g-gtadj_listk ) while( n
! NULL ) j ANodeIndex( ListItem( al_node
) ) if ( !g-gtvisitedj ) visit( g, j )
al_node ListNext( al_node )
Assumes a List ADT with methods ListHead
ANodeIndex ListItem ListNext
35
Graphs - Traversing - Depth-First

• Adjacency List
• Time complexity
• Visited set for each node
• Each edge visited twice
• Once in each adjacency list
• O(V E)
• O(V2) for dense E V2 graphs
• but O(V) for sparse E V graphs
• Adjacency Lists perform better for sparse graphs

36
Graph - Breadth-first Traversal

• Breadth-first requires a FIFO queue

static queue q void search( graph g ) q
ConsQueue( g-gtn_nodes ) for(k0kltg-gtn_nodesk
) g-gtvisitedk 0 search_index 0
for(k0kltg-gtn_nodesk) if (
!g-gtvisitedk ) visit( g, k )
37
Graph - Breadth-first Traversal
void visit( graph g, int k ) al_node
al_node int j AddIntToQueue( q, k )
while( !Empty( q ) ) k QueueHead( q )
g-gtvisitedk search_index al_node
ListHead( g-gtadj_listk) while( al_node !
NULL ) j ANodeIndex(al_node) if
( !g-gtvisitedj ) AddIntToQueue( g, j
) g-gtvisitedj -1 / C hack, 0
false! / al_node ListNext( al_node )

Put this node on the queue
38
Graphs - Shortest Paths

• Application
• In a graph in which edges have costs ..
• Find the shortest path from a source to a
  destination
• Surprisingly ..
• While finding the shortest path from a source to
  one destination,
• we can find the shortest paths to all over
  destinations as well!
• Common algorithm for single-source
  shortest paths is due to Edsger Dijkstra

39
Dijkstras Algorithm - Data Structures

• For a graph, G ( V, E
  )
• Dijkstras algorithm keeps two sets of vertices
• S Vertices whose shortest paths have
  already been determinedV-S Remainder
• Also
• d Best estimates of shortest path to
  each vertexp Predecessors for each
  vertex

40
Predecessor Sub-graph

• Array of vertex indices, pj, j 1 .. V
• pj contains the predecessor for node j
• js predecessor is in ppj, and so on ....
• The edges in the predecessor sub-graph are
  ( pj, j )

41
Dijkstras Algorithm - Operation

• Initialize d and p
• For each vertex, j, in V
• dj
• pj nil
• Source distance, ds 0
• Set S to empty
• While V-S is not empty
• Sort V-S based on d
• Add u, the closest vertex in V-S, to S
• Relax all the vertices still in V-S connected to u

Initial estimates are all
No connections
Add s first!
42
Dijkstras Algorithm - Operation

• The Relaxation process

Relax the node v attached to node u
Edge cost matrix
If the current best estimate to v is greater than
the path through u ..
relax( Node u, Node v, double w ) if dv
gt du wu,v then dv du
wu,v piv u
Update the estimate to v
Make vs predecessor point to u
43
Dijkstras Algorithm - Full

• The Shortest Paths algorithm

Initialise d, p, S, vertex Q
Given a graph, g, and a source, s
shortest_paths( Graph g, Node s )
initialise_single_source( g, s ) S 0
/ Make S empty / Q Vertices( g ) /
Put the vertices in a PQ / while not
Empty(Q) u ExtractCheapest( Q )
AddNode( S, u ) / Add u to S / for
each vertex v in Adjacent( u ) relax(
u, v, w )
While nodes in Q Greedy!
Update the estimate of the shortest paths to all
nodes attached to u
44
Dijkstras Algorithm - Operation

• Initial Graph

Source Mark 0
Distance to all nodes marked
45
Dijkstras Algorithm - Operation

• Initial Graph

Source
Relax vertices adjacent to source
Red arrows show predecessors
46
Dijkstras Algorithm - Operation
Sort vertices and choose closest
Source is now in S
47
Dijkstras Algorithm - Operation
Relax u because a shorter path via x exists
Relax y because a shorter path via x exists
Source is now in S
48
Dijkstras Algorithm - Operation
Change us predecessor also
Relax y because a shorter path via x exists
Source is now in S
49
Dijkstras Algorithm - Operation
Sort vertices and choose closest, y
S is now s, x
50
Dijkstras Algorithm - Operation
Relax v because a shorter path via y exists
S is now s, x
51
Dijkstras Algorithm - Operation
S is now s, x, y
Sort vertices and choose closest, u
52
Dijkstras Algorithm - Operation
S is now s, x, y, u
Finally add v
Predecessors show shortest paths sub-graph
53
Dijkstras Algorithm - Proof

• Greedy Algorithm
• Proof by contradiction best
• Lemma 1
• Shortest paths are composed of shortest paths
• Proof
• If there was a shorter path than any sub-path,
  then substitution of that path would make the
  whole path shorter

54
Dijkstras Algorithm - Proof

• Denote
• d(s,v) - the cost of the shortest path from s
  to v
• Lemma 2
• If s...uv is a shortest path from s to v,then
  after u has been added to S and relax(u,v,w)
  called,dv d(s,v) and dv is not changed
  thereafter.
• Proof
• Follows from the fact that at all times dv ³
  d(s,v)
• See Cormen (or any other text) for the details

55
Dijkstras Algorithm - Proof

• Using Lemma 2
• After running Dijkstras algorithm, we assert
• dv d(s,v) for all v
• Proof (by contradiction)
• Suppose that u is the first vertex added to S for
  which du ¹ d(s,u)
• Note
• v is not s because ds 0
• There must be a path s...u, otherwise du
  would be
• Since theres a path, there must be a shortest
  path

56
Dijkstras Algorithm - Proof

• Proof (by contradiction)
• Suppose that u is the first vertex added to S for
  which du ¹ d(s,u)
• Let sxyu be the shortest pathsu, where x is
  in S and y is the first outside S
• When x was added to S, dx d(s,x)
• Edge xy was relaxed at that time, so dy
  d(s,y)

57
Dijkstras Algorithm - Proof

• Proof (by contradiction)
• Edge xy was relaxed at that time, so dy
  d(s,y) d(s,u) du
• But, when we chose u,both u and y where in
  V-S,so du dy (otherwise we would have
  chosen y)
• Thus the inequalities must be equalities
• dy d(s,y) d(s,u) du
• And our hypothesis (du ¹ d(s,u)) is
  contradicted!

58
Dijkstras Algorithm - Time Complexity

• Dijkstras Algorithm
• Similar to MST algorithms
• Key step is sort on the edges
• Complexity is
• O( (EV)logV ) or
• O( n2 log n )
• for a dense graph with n V