Shortest Paths

1
Shortest Paths

• Definitions
• Single Source Algorithms
• Bellman Ford
• DAG shortest path algorithm
• Dijkstra
• All Pairs Algorithms
• Using Single Source Algorithms
• Matrix multiplication
• Floyd-Warshall
• Both of above use adjacency matrix representation
  and dynamic programming
• Johnsons algorithm
• Uses adjacency list representation

2
Single Source Definition

• Input
• Weighted, connected directed graph G(V,E)
• Weight (length) function w on each edge e in E
• Source node s in V
• Task
• Compute a shortest path from s to all nodes in V

3
All Pairs Definition

• Input
• Weighted, connected directed graph G(V,E)
• Weight (length) function w on each edge e in E
• We will typically assume w is represented as a
  matrix
• Task
• Compute a shortest path from all nodes in V to
  all nodes in V

4
Comments

• If edges are not weighted, then BFS works for
  single source problem
• Optimal substructure
• A shortest path between s and t contains other
  shortest paths within it
• No known algorithm is better at finding a
  shortest path from s to a specific destination
  node t in G than finding the shortest path from s
  to all nodes in V

5
Negative weight edges

• Negative weight edges can be allowed as long as
  there are no negative weight cycles
• If there are negative weight cycles, then there
  cannot be a shortest path from s to any node t
  (why?)
• If we disallow negative weight cycles, then there
  always is a shortest path that contains no cycles

6
Relaxation technique

• For each vertex v, we maintain an upper bound
  dv on the length of shortest path from s to v
• dv initialized to infinity
• Relaxing an edge (u,v)
• Can we shorten the path to v by going through u?
• If dv gt du w(u,v), dv du w(u,v)
• This can be done in O(1) time

7
Bellman-Ford Algorithm

• Bellman-Ford (G, w, s)
• Initialize-Single-Source(G,s)
• for (i1 to V-1)
• for each edge (u,v) in E
• relax(u,v)
• for each edge (u,v) in E
• if dv gt du w(u,v)
• return NEGATIVE WEIGHT CYCLE

G1
G2
8
Running Time

• for (i1 to V-1)
• for each edge (u,v) in E
• relax(u,v)
• The above takes (V-1)O(E) O(VE) time
• for each edge (u,v) in E
• if dv gt du w(u,v)
• return NEGATIVE WEIGHT CYCLE
• The above takes O(E) time

9
Proof of Correctness

• Theorem If there is a shortest path from s to
  any node v, then dv will have this weight at
  end of Bellman-Ford algorithm
• Theorem restated
• Define linksv to be the minimum number of edges
  on a shortest path from s to v
• After i iterations of the Bellman-Ford for loop,
  nodes v with linksv i will have their dv
  value set correctly
• Prove this by induction on linksv
• Base case linksv 0 which means v is s
• ds is initialized to 0 which is correct unless
  there is a negative weight cycle from s to s in
  which case there is no shortest path from s to s.
• Induction hypothesis After k iterations for k
  0, dv must be correct if linksv k
• Inductive step Show after k1 iterations, dv
  must be correct if linksv k1
• If linksv k, then by IH, dv is correctly
  set after kth iteration
• If linksv k1, let p (e1, e2, , ek1)
  (v0, v1, v2, , vk1) be a shortest path from s
  to v
• s v0, v vk1, ei (vi-1, vi)
• In order for linksv k1, then linksvk k.
• By the inductive hypothesis, dvk will be
  correctly set after the kth iteration.
• During the k1st iteration, we relax all edges
  including edge ek1 (vk,v)
• Thus, at end of the k1st iteration, dv will be
  correct

10
Negative weight cycle

• for each edge (u,v) in E
• if dv gt du w(u,v)
• return NEGATIVE WEIGHT CYCLE
• If no neg weight cycle, dv du w(u,v) for
  all (u,v)
• If there is a negative weight cycle C, for some
  edge (u,v) on C, it must be the case that dv gt
  du w(u,v).
• Suppose this is not true for some neg. weight
  cycle C
• sum these (dv du w(u,v)) all the way
  around C
• We end up with Sv in C dv (Su in C du)
  weight(C)
• This is impossible unless weight(C) 0
• But weight(C) is negative, so this cannot happen
• Thus for some (u,v) on C, dv gt du w(u,v)

11
DAG shortest path algorithm
5

• DAG-SP (G, w, s)
• Initialize-Single-Source(G,s)
• Topologically sort vertices in G
• for each vertex u, taken in sorted order
• for each edge (u,v) in E
• relax(u,v)

3
4
-4
s
5
1
3
8
12
Running time Improvement

• O(VE) for the topological sorting
• We only do 1 relaxation for each edge O(E) time
• for each vertex u, taken in sorted order
• for each edge (u,v) in E
• relax(u,v)
• Overall running time O(VE)

13
Proof of Correctness

• If there is a shortest path from s to any node v,
  then dv will have this weight at end
• Let p (e1, e2, , ek) (v1, v2, v3, , vk1)
  be a shortest path from s to v
• s v1, v vk1, ei (vi, vi1)
• Since we sort edges in topological order, we will
  process node vi (and edge ei) before processing
  later edges in the path.

14
Dijkstras Algorithm

• Dijkstra (G, w, s)
• / Assumption all edge weights non-negative /
• Initialize-Single-Source(G,s)
• Completed
• ToBeCompleted V
• While ToBeCompleted is not empty
• u EXTRACT-MIN(ToBeCompleted)
• Completed u
• for each edge (u,v) relax(u,v)

15
Running Time Analysis

• While ToBeCompleted is not empty
• u EXTRACT-MIN(ToBeCompleted)
• Completed u
• for each edge (u,v) relax(u,v)
• Each edge relaxed at most once O(E)
• Need to decrease-key potentially once per edge
• Need to extract-min once per node
• The nodes dv is then complete

16
Running Time Analysis contd

• Priority Queue operations
• O(E) decrease key operations
• O(V) extract-min operations
• Three implementations of priority queues
• Array O(V2) time
• decrease-key is O(1) and extract-min is O(V)
• Binary heap O(E log V) time assuming E V
• decrease-key and extract-min are O(log V)
• Fibonacci heap O(V log V E) time
• decrease-key is O(1) amortized and extract-min is
  O(log V)

17
Compare Dijkstras algorithm to Prims algorithm
for MST

• Dijsktra
• Priority Queue operations
• O(E) decrease key operations
• O(V) extract-min operations
• Prim
• Priority Queue operations
• O(E) decrease-key operations
• O(V) extract-min operations
• Is this a coincidence or is there something more
  here?

18
Proof of Correctness

• Assume that Dijkstras algorithm fails to compute
  length of all shortest paths from s
• Let v be the first node whose shortest path
  length is computed incorrectly
• Let S be the set of nodes whose shortest paths
  were computed correctly by Dijkstra prior to
  adding v to the processed set of nodes.
• Dijkstras algorithm has used the shortest path
  from s to v using only nodes in S when it added v
  to S.
• The shortest path to v must include one node not
  in S
• Let u be the first such node.

19
Proof of Correctness

• The length of the shortest path to u must be at
  least that of the length of the path computed to
  v.
• Why?
• The length of the path from u to v must be lt 0.
• Why?
• No path can have negative length since all edge
  weights are non-negative, and thus we have a
  contradiction.

20
Computing paths (not just distance)

• Maintain for each node v a predecessor node p(v)
• p(v) is initialized to be null
• Whenever an edge (u,v) is relaxed such that d(v)
  improves, then p(v) can be set to be u
• Paths can be generated from this data structure

21
All pairs algorithms using single source
algorithms

• Call a single source algorithm from each vertex s
  in V
• O(V X) where X is the running time of the given
  algorithm
• Dijkstra linear array O(V3)
• Dijkstra binary heap O(VE log V)
• Dijkstra Fibonacci heap O(V2 log V VE)
• Bellman-Ford O(V2 E) (negative weight edges)

22
Two adjacency matrix based algorithms

• Matrix-multiplication based algorithm
• Let Lm(i,j) denote the length of the shortest
  path from node i to node j using at most m edges
• What is our desired result in terms of Lm(i,j)?
• What is a recurrence relation for Lm(i,j)?
• Floyd-Warshall algorithm
• Let Lk(i,j) denote the length of the shortest
  path from node i to node j using only nodes
  within 1, , k as internal nodes.
• What is our desired result in terms of Lk(i,j)?
• What is a recurrence relation for Lk(i,j)?

23
Conceptual pictures
Shortest path using nodes 1 through k-1
k
OR
24
Running Times

• Matrix-multiplication based algorithm
• O(V3 log V)
• log V executions of matrix-matrix
  multiplication
• Not quite matrix-matrix multiplication but same
  running time
• Floyd-Warshall algorithm
• O(V3)
• V iterations of an O(V2) update loop
• The constant is very small, so this is a fast
  O(V3)

25
Johnsons Algorithm

• Key ideas
• Reweight edge weights to eliminate negative
  weight edges AND preserve shortest paths
• Use Bellman-Ford and Dijkstras algorithms as
  subroutines
• Running time O(V2 log V VE)
• Better than earlier algorithms for sparse graphs

26
Reweighting

• Original edge weight is w(u,v)
• New edge weight
• w(u,v) w(u,v) h(u) h(v)
• h(v) is a function mapping vertices to real
  numbers
• Key observation
• Let p be any path from node u to node v
• w(p) w(p) h(u) h(v)

27
Computing vertex weights h(v)

• Create a new graph G (V, E) by
• adding a new vertex s to V
• adding edges (s,v) for all v in V with w(s,v) 0
• Set h(v) to be the length of the shortest path
  from this new node s to node v
• This is well-defined if G does not contain
  negative weight cycles
• Note that h(v) h(u) w(u,v) for all (u,v) in
  E
• Thus, w(u,v) w(u,v) h(u) h(v) 0

28
Algorithm implementation

• Run Bellman-Ford on G from new node s
• If no negative weight cycle, then use h(v) values
  from Bellman-Ford
• Now compute w(u,v) for each edge (u,v) in E
• Now run Dijkstras algorithm using w
• Use each node as source node
• Modify du,v at end by adding h(v) and
  subtracting h(u) to get true path weight
• Running time
• O(VE) from one run of Bellman-Ford
• O(V2 log V VE) from V runs of Dijkstra