CSE 326: Data Structures Part 8 Graphs

1
CSE 326 Data StructuresPart 8Graphs

• Henry Kautz
• Autumn Quarter 2002

2
Outline

• Graphs (TO DO READ WEISS CH 9)
• Graph Data Structures
• Graph Properties
• Topological Sort
• Graph Traversals
• Depth First Search
• Breadth First Search
• Iterative Deepening Depth First
• Shortest Path Problem
• Dijkstras Algorithm

3
Graph ADT

• Graphs are a formalism for representing
  relationships between objects
• a graph G is represented as G (V, E)
• V is a set of vertices
• E is a set of edges
• operations include
• iterating over vertices
• iterating over edges
• iterating over vertices adjacent to a specific
  vertex
• asking whether an edge exists connected two
  vertices

V Han, Leia, Luke E (Luke, Leia),
(Han, Leia), (Leia, Han)
4
What Graph is THIS?
5
ReferralWeb(co-authorship in scientific papers)
6
Biological Function Semantic Network
7
Graph Representation 1 Adjacency Matrix

• A V x V array in which an element (u, v) is
  true if and only if there is an edge from u to v

Han
Luke
Leia
Han
Luke
Runtime iterate over vertices iterate ever
edges iterate edges adj. to vertex edge exists?
Leia
Space requirements
8
Graph Representation 2 Adjacency List

• A V-ary list (array) in which each entry stores
  a list (linked list) of all adjacent vertices

Han
Luke
Runtime iterate over vertices iterate ever
edges iterate edges adj. to vertex edge exists?
Leia
space requirements
9
Directed vs. Undirected Graphs

• In directed graphs, edges have a specific
  direction
• In undirected graphs, they dont (edges are
  two-way)
• Vertices u and v are adjacent if (u, v) ? E

Han
Luke
Leia
Han
Luke
Leia
10
Graph Density

• A sparse graph has O(V) edges
• A dense graph has ?(V2) edges
• Anything in between is either sparsish or densy
  depending on the context.

11
Weighted Graphs
Each edge has an associated weight or cost.
20
Clinton
Mukilteo
30
Kingston
Edmonds
35
Bainbridge
Seattle
60
Bremerton
There may be more information in the graph as
well.
12
Paths and Cycles

• A path is a list of vertices v1, v2, , vn such
  that (vi, vi1) ? E for all 0 ? i lt n.
• A cycle is a path that begins and ends at the
  same node.

Chicago
Seattle
Salt Lake City
San Francisco
Dallas
p Seattle, Salt Lake City, Chicago, Dallas,
San Francisco, Seattle
13
Path Length and Cost

• Path length the number of edges in the path
• Path cost the sum of the costs of each edge

Chicago
3.5
Seattle
2
2
Salt Lake City
2
2.5
2.5
2.5
3
San Francisco
Dallas
length(p) 5
cost(p) 11.5
14
Connectivity

• Undirected graphs are connected if there is a
  path between any two vertices
• Directed graphs are strongly connected if there
  is a path from any one vertex to any other
• Directed graphs are weakly connected if there is
  a path between any two vertices, ignoring
  direction
• A complete graph has an edge between every pair
  of vertices

15
Trees as Graphs

• Every tree is a graph with some restrictions
• the tree is directed
• there are no cycles (directed or undirected)
• there is a directed path from the root to every
  node

A
B
C
D
E
F
H
G
BAD!
J
I
16
Directed Acyclic Graphs (DAGs)

• DAGs are directed graphs with no cycles.

main()
mult()
if program call graph is a DAG, then all
procedure calls can be in-lined
add()
read()
access()
Trees ? DAGs ? Graphs
17
Application of DAGs Representing Partial Orders
reserve flight
check in airport
call taxi
pack bags
take flight
locate gate
taxi to airport
18
Topological Sort

• Given a graph, G (V, E), output all the
  vertices in V such that no vertex is output
  before any other vertex with an edge to it.

reserve flight
check in airport
call taxi
take flight
taxi to airport
locate gate
pack bags
19
Topo-Sort Take One

• Label each vertexs in-degree ( of inbound
  edges)
• While there are vertices remaining
• Pick a vertex with in-degree of zero and output
  it
• Reduce the in-degree of all vertices adjacent to
  it
• Remove it from the list of vertices

runtime
20
Topo-Sort Take Two

• Label each vertexs in-degree
• Initialize a queue (or stack) to contain all
  in-degree zero vertices
• While there are vertices remaining in the queue
• Remove a vertex v with in-degree of zero and
  output it
• Reduce the in-degree of all vertices adjacent to
  v
• Put any of these with new in-degree zero on the
  queue

runtime
21
Recall Tree Traversals
a
e
d
b
c
i
h
j
f
g
k
l
a b f g k c d h i l j e
22
Depth-First Search

• Pre/Post/In order traversals are examples of
  depth-first search
• Nodes are visited deeply on the left-most
  branches before any nodes are visited on the
  right-most branches
• Visiting the right branches deeply before the
  left would still be depth-first! Crucial idea is
  go deep first!
• Difference in pre/post/in-order is how some
  computation (e.g. printing) is done at current
  node relative to the recursive calls
• In DFS the nodes being worked on are kept on a
  stack

23
Iterative Version DFSPre-order Traversal

• Push root on a Stack
• Repeat until Stack is empty
• Pop a node
• Process it
• Push its children on the Stack

24
Level-Order Tree Traversal

• Consider task of traversing tree level by level
  from top to bottom (alphabetic order)
• Is this also DFS?

25
Breadth-First Search

• No! Level-order traversal is an example of
  Breadth-First Search
• BFS characteristics
• Nodes being worked on maintained in a FIFO Queue,
  not a stack
• Iterative style procedures often easier to design
  than recursive procedures
• Put root in a Queue
• Repeat until Queue is empty
• Dequeue a node
• Process it
• Add its children to queue

26
QUEUE

• a
• b c d e
• c d e f g
• d e f g
• e f g h i j
• f g h i j
• g h i j
• h i j k
• i j k
• j k l
• k l
• l

27
Graph Traversals

• Depth first search and breadth first search also
  work for arbitrary (directed or undirected)
  graphs
• Must mark visited vertices so you do not go into
  an infinite loop!
• Either can be used to determine connectivity
• Is there a path between two given vertices?
• Is the graph (weakly) connected?
• Important difference Breadth-first search
  always finds a shortest path from the start
  vertex to any other (for unweighted graphs)
• Depth first search may not!

28
Demos on Web PageDFSBFS
29
Is BFS the Hands Down Winner?

• Depth-first search
• Simple to implement (implicit or explict stack)
• Does not always find shortest paths
• Must be careful to mark visited vertices, or
  you could go into an infinite loop if there is a
  cycle
• Breadth-first search
• Simple to implement (queue)
• Always finds shortest paths
• Marking visited nodes can improve efficiency, but
  even without doing so search is guaranteed to
  terminate

30
Space Requirements

• Consider space required by the stack or queue
• Suppose
• G is known to be at distance d from S
• Each vertex n has k out-edges
• There are no (undirected or directed) cycles
• BFS queue will grow to size kd
• Will simultaneously contain all nodes that are at
  distance d (once last vertex at distance d-1 is
  expanded)
• For k10, d15, size is 1,000,000,000,000,000

31
DFS Space Requirements

• Consider DFS, where we limit the depth of the
  search to d
• Force a backtrack at d1
• When visiting a node n at depth d, stack will
  contain
• (at most) k-1 siblings of n
• parent of n
• siblings of parent of n
• grandparent of n
• siblings of grandparent of n
• DFS queue grows at most to size dk
• For k10, d15, size is 150
• Compare with BFS 1,000,000,000,000,000

32
Conclusion

• For very large graphs DFS is hugely more memory
  efficient, if we know the distance to the goal
  vertex!
• But suppose we dont know d. What is the
  (obvious) strategy?

33
Iterative Deepening DFS

• IterativeDeepeningDFS(vertex s, g)
• for (i1truei)
• if DFS(i, s, g) return
• // Also need to keep track of path found
• bool DFS(int limit, vertex s, g)
• if (sg) return true
• if (limit-- lt 0) return false
• for (n in children(s))
• if (DFS(limit, n, g)) return true
• return false

34
Analysis of Iterative Deepening

• Even without marking nodes as visited,
  iterative-deepening DFS never goes into an
  infinite loop
• For very large graphs, memory cost of keeping
  track of visited vertices may make marking
  prohibitive
• Work performed with limit lt actual distance to G
  is wasted but the wasted work is usually small
  compared to amount of work done during the last
  iteration

35
Asymptotic Analysis

• There are pathological graphs for which
  iterative deepening is bad

nd
G
S
36
A Better Case

• Suppose each vertex n has k out-edges, no cycles
• Bounded DFS to level i reaches ki vertices
• Iterative Deepening DFS(d)

ignore low order terms!
37
(More) Conclusions

• To find a shortest path between two nodes in a
  unweighted graph, use either BFS or Iterated DFS
• If the graph is large, Iterated DFS typically
  uses much less memory
• Later well learn about heuristic search
  algorithms, which use additional knowledge about
  the problem domain to reduce the number of
  vertices visited

38
Single Source, Shortest Path for Weighted Graphs

• Given a graph G (V, E) with edge costs c(e),
  and a vertex s ? V, find the shortest (lowest
  cost) path from s to every vertex in V
• Graph may be directed or undirected
• Graph may or may not contain cycles
• Weights may be all positive or not
• What is the problem if graph contains cycles
  whose total cost is negative?

39
The Trouble with Negative Weighted Cycles
2
A
B
10
-5
1
E
2
C
D
40
Edsger Wybe Dijkstra (1930-2002)

• Invented concepts of structured programming,
  synchronization, weakest precondition, and
  "semaphores" for controlling computer processes.
  The Oxford English Dictionary cites his use of
  the words "vector" and "stack" in a computing
  context.
• Believed programming should be taught without
  computers
• 1972 Turing Award
• In their capacity as a tool, computers will be
  but a ripple on the surface of our culture. In
  their capacity as intellectual challenge, they
  are without precedent in the cultural history of
  mankind.

41
Dijkstras Algorithm for Single Source Shortest
Path

• Classic algorithm for solving shortest path in
  weighted graphs (with only positive edge weights)
• Similar to breadth-first search, but uses a
  priority queue instead of a FIFO queue
• Always select (expand) the vertex that has a
  lowest-cost path to the start vertex
• a kind of greedy algorithm
• Correctly handles the case where the lowest-cost
  (shortest) path to a vertex is not the one with
  fewest edges

42
Pseudocode for Dijkstra

• Initialize the cost of each vertex to ?
• costs 0
• heap.insert(s)
• While (! heap.empty())
• n heap.deleteMin()
• For (each vertex a which is adjacent to n along
  edge e)
• if (costn edge_coste lt costa) then
• cost a costn edge_coste
• previous_on_path_toa n
• if (a is in the heap) then heap.decreaseKey(a)
• else heap.insert(a)

43
Important Features

• Once a vertex is removed from the head, the cost
  of the shortest path to that node is known
• While a vertex is still in the heap, another
  shorter path to it might still be found
• The shortest path itself from s to any node a can
  be found by following the pointers stored in
  previous_on_path_toa

44
Dijkstras Algorithm in Action
45
DemoDijkstras
46
Data Structures for Dijkstras Algorithm
V times
Select the unknown node with the lowest cost
findMin/deleteMin
O(log V)
E times
as cost min(as old cost, )
decreaseKey
O(log V)
runtime O(E log V)
47
CSE 326 Data StructuresLecture 8.BHeuristic
Graph Search

• Henry Kautz
• Winter Quarter 2002

48
Homework Hint - Problem 4

• You can turn in a final version of your answer to
  problem 4 without penalty on Wednesday.

49
Outline

• Best First Search
• A Search
• Example Plan Synthesis
• This material is NOT in Weiss, but is important
  for both the programming project and the final
  exam!

50
Huge Graphs

• Consider some really huge graphs
• All cities and towns in the World Atlas
• All stars in the Galaxy
• All ways 10 blocks can be stacked
• Huh???

51
Implicitly Generated Graphs

• A huge graph may be implicitly specified by rules
  for generating it on-the-fly
• Blocks world
• vertex relative positions of all blocks
• edge robot arm stacks one block

stack(blue,table)
stack(green,blue)
stack(blue,red)
stack(green,red)
stack(green,blue)
52
Blocks World

• Source initial state of the blocks
• Goal desired state of the blocks
• Path source to goal sequence of actions
  (program) for robot arm!
• n blocks ? nn vertices
• 10 blocks ? 10 billion vertices!

53
Problem Branching Factor

• Cannot search such huge graphs exhaustively.
  Suppose we know that goal is only d steps away.
• Dijkstras algorithm is basically breadth-first
  search (modified to handle arc weights)
• Breadth-first search (or for weighted graphs,
  Dijkstras algorithm) If out-degree of each
  node is 10, potentially visits 10d vertices
• 10 step plan 10 billion vertices visited!

54
An Easier Case

• Suppose you live in Manhattan what do you do?

S
52nd St
G
51st St
50th St
10th Ave
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
3rd Ave
2nd Ave
55
Best-First Search

• The Manhattan distance (? x ? y) is an estimate
  of the distance to the goal
• a heuristic value
• Best-First Search
• Order nodes in priority to minimize estimated
  distance to the goal h(n)
• Compare BFS / Dijkstra
• Order nodes in priority to minimize distance from
  the start

56
Best First in Action

• Suppose you live in Manhattan what do you do?

S
52nd St
G
51st St
50th St
10th Ave
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
3rd Ave
2nd Ave
57
Problem 1 Led Astray

• Eventually will expand vertex to get back on the
  right track

S
G
52nd St
51st St
50th St
10th Ave
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
3rd Ave
2nd Ave
58
Problem 2 Optimality

• With Best-First Search, are you guaranteed a
  shortest path is found when
• goal is first seen?
• when goal is removed from priority queue (as with
  Dijkstra?)

59
Sub-Optimal Solution

• No! Goal is by definition at distance 0 will be
  removed from priority queue immediately, even if
  a shorter path exists!

(5 blocks)
S
52nd St
h2
h5
G
h4
51st St
h1
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
60
Synergy?

• Dijkstra / Breadth First guaranteed to find
  optimal solution
• Best First often visits far fewer vertices, but
  may not provide optimal solution
• Can we get the best of both?

61
A (A star)

• Order vertices in priority queue to minimize
• (distance from start) (estimated distance to
  goal)
• f(n) g(n) h(n)
• f(n) priority of a node
• g(n) true distance from start
• h(n) heuristic distance to goal

62
Optimality

• Suppose the estimated distance (h) is always
  less than or equal to the true distance to the
  goal
• heuristic is a lower bound on true distance
• Then when the goal is removed from the priority
  queue, we are guaranteed to have found a shortest
  path!

63
Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 9th 0 5 5


S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
64
Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 4th 5 2 7
51st 9th 1 4 5

S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
65
Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 4th 5 2 7
51st 8th 2 3 5
50th 9th 2 5 7
S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
66
Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 4th 5 2 7
51st 7th 3 2 5
50th 9th 2 5 7
50th 8th 3 4 7
S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
67
Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 4th 5 2 7
51st 6th 4 1 5
50th 9th 2 5 7
50th 8th 3 4 7
50th 7th 4 3 7
S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
68
Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 4th 5 2 7
51st 5th 5 0 5
50th 9th 2 5 7
50th 8th 3 4 7
50th 7th 4 3 7
S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
69
Problem 2 Revisited
vertex g(n) h(n) f(n)
52nd 4th 5 2 7
50th 9th 2 5 7
50th 8th 3 4 7
50th 7th 4 3 7
S
(5 blocks)
52nd St
G
51st St
50th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
DONE!
70
What Would Dijkstra Have Done?
S
(5 blocks)
52nd St
G
51st St
50th St
49th St
48th St
47th St
9th Ave
8th Ave
7th Ave
6th Ave
5th Ave
4th Ave
71
Proof of A Optimality

• A terminates when G is popped from the heap.
• Suppose G is popped but the path found isnt
  optimal
• priority(G) gt optimal path length c
• Let P be an optimal path from S to G, and let N
  be the last vertex on that path that has been
  visited but not yet popped.
• There must be such an N, otherwise the optimal
  path would have been found.
• priority(N) g(N) h(N) ? c
• So N should have popped before G can pop.
  Contradiction.

non-optimal path to G
S
G
undiscovered portion of shortest path
portion of optimal path found so far
N
72
What About Those Blocks?

• Distance to goal is not always physical
  distance
• Blocks world
• distance number of stacks to perform
• heuristic lower bound number of blocks out of
  place

out of place 2, true distance to goal 3
73
3-Blocks State Space Graph
ABCh2
CABh3
BACh2
ABCh1
CBAh3
A CBh2
BCAh1
BC Ah3
CB Ah3
C A Bh3
AC Bh3
BA Ch2
AB Ch0
start
goal
74
3-Blocks Best First Solution
ABCh2
CABh3
BACh2
ABCh1
CBAh3
A CBh2
BCAh1
BC Ah3
CB Ah3
C A Bh3
AC Bh3
BA Ch2
AB Ch0
start
goal
75
3-Blocks BFS Solution
ABCh2
expanded, but not in solution
CABh3
BACh2
ABCh1
CBAh3
A CBh2
BCAh1
BC Ah3
CB Ah3
C A Bh3
AC Bh3
BA Ch2
AB Ch0
start
goal
76
3-Blocks A Solution
ABCh2
expanded, but not in solution
CABh3
BACh2
ABCh1
CBAh3
A CBh2
BCAh1
BC Ah3
CB Ah3
C A Bh3
AC Bh3
BA Ch2
AB Ch0
start
goal
77
Other Real-World Applications

• Routing finding computer networks, airline
  route planning
• VLSI layout cell layout and channel routing
• Production planning just in time optimization
• Protein sequence alignment
• Many other NP-Hard problems
• A class of problems for which no exact polynomial
  time algorithms exist so heuristic search is
  the best we can hope for

78
Coming Up

• Other graph problems
• Connected components
• Spanning tree

79
CSE 326 Data StructuresPart 8.CSpanning Trees
and More

• Henry Kautz
• Autumn Quarter 2002

80
Today

• Incremental hashing
• MazeRunner project
• Longest Path?
• Finding Connected Components
• Application to machine vision
• Finding Minimum Spanning Trees
• Yet another use for union/find

81
Incremental Hashing
82
Maze Runner
20 15 --------------------


---- - - -
-- ------
-----------
- X
- -- --- - - ------

- - - -
--
-
-

---- -
-
-- --
------------
-------
- ----
---
- - - -
----
- --- -- - -

-
--------------------

• DFS, iterated DFS, BFS, best-first, A
• Crufty old C code from fresh clean Java code
• Win fame and glory by writing a nice real-time
  maze visualizer

83
Java Note
Java lacks enumerated constants enum DOG,
CAT, MOUSE animal animal a DOG Static
constants not type-safe static final int DOG
1 static final int CAT 2 static final int
BLUE 1 int favoriteColor DOG
84
Amazing Java Trick
public final class Animal private Animal()
public static final Animal DOG new Animal()
public static final Animal CAT new
Animal() public final class Color private
Color() public static final Animal BLUE new
Color() Animal x DOG Animal x BLUE //
Gives compile-time error!
85
Longest Path Problem

• Given a graph G(V,E) and vertices s, t
• Find a longest simple path (no repeating
  vertices) from s to t.
• Does reverse Dijkstra work?

86
Dijkstra

• Initialize the cost of each vertex to ?
• costs 0
• heap.insert(s)
• While (! heap.empty())
• n heap.deleteMin()
• For (each vertex a which is adjacent to n along
  edge e)
• if (costn edge_coste lt costa) then
• cost a costn edge_coste
• previous_on_path_toa n
• if (a is in the heap) then heap.decreaseKey(a)
• else heap.insert(a)

87
Reverse Dijkstra

• Initialize the cost of each vertex to ?
• costs 0
• heap.insert(s)
• While (! heap.empty())
• n heap.deleteMax()
• For (each vertex a which is adjacent to n along
  edge e)
• if (costn edge_coste gt costa) then
• cost a costn edge_coste
• previous_on_path_toa n
• if (a is in the heap) then heap.increaseKey(a)
• else heap.insert(a)

88
Does it Work?
a
6
t
5
3
b
1
s
89
Problem

• No clear stopping condition!
• How many times could a vertex be inserted in the
  priority queue?
• Exponential!
• Not a good algorithm!
• Is the better one?

90
Counting Connected Components

• Initialize the cost of each vertex to ?
• Num_cc 0
• While there are vertices of cost ?
• Pick an arbitrary such vertex S, set its cost to
  0
• Find paths from S
• Num_cc

91
Using DFS

• Set each vertex to unvisited
• Num_cc 0
• While there are unvisited vertices
• Pick an arbitrary such vertex S
• Perform DFS from S, marking vertices as visited
• Num_cc

Complexity O(VE)
92
Using Union / Find

• Put each node in its own equivalence class
• Num_cc 0
• For each edge E ltx,ygt
• Union(x,y)
• Return number of equivalence classes

Complexity
93
Using Union / Find

• Put each node in its own equivalence class
• Num_cc 0
• For each edge E ltx,ygt
• Union(x,y)
• Return number of equivalence classes

Complexity O(VE ack(E,V))
94
Machine Vision Blob Finding
95
Machine Vision Blob Finding
1
2
3
5
4
96
Blob Finding

• Matrix can be considered an efficient
  representation of a graph with a very regular
  structure
• Cell vertex
• Adjacent cells of same color edge between
  vertices
• Blob finding finding connected components

97
Tradeoffs

• Both DFS and Union/Find approaches are
  (essentially) O(EV) O(E) for binary
  images
• For each component, DFS (recursive labeling)
  can move all over the image entire image must
  be in main memory
• Better in practice row-by-row processing
• localizes accesses to memory
• typically 1-2 orders of magnitude faster!

98
High-Level Blob-Labeling

• Scan through image left/right and top/bottom
• If a cell is same color as (connected to) cell to
  right or below, then union them
• Give the same blob number to cells in each
  equivalence class

99
Blob-Labeling Algorithm

• Put each cell ltx,ygt in its own equivalence class
• For each cell ltx,ygt
• if colorx,y colorx1,y then
• Union( ltx,ygt, ltx1,ygt )
• if colorx,y colorx,y1 then
• Union( ltx,ygt, ltx,y1gt )
• label 0
• For each root ltx,ygt
• blobnumx,y label
• For each cell ltx,ygt
• blobnumx,y blobnum( Find(ltx,ygt) )

100
Spanning Tree

• Spanning tree a subset of the edges from a
  connected graph that
• touches all vertices in the graph (spans the
  graph)
• forms a tree (is connected and contains no
  cycles)
• Minimum spanning tree the spanning tree with the
  least total edge cost.

4
7
9
2
1
5
101
Applications of Minimal Spanning Trees

• Communication networks
• VLSI design
• Transportation systems

102
Kruskals Algorithm for Minimum Spanning Trees

• A greedy algorithm
• Initialize all vertices to unconnected
• While there are still unmarked edges
• Pick a lowest cost edge e (u, v) and mark it
• If u and v are not already connected, add e to
  the minimum spanning tree and connect u and v

Sound familiar? (Think maze generation.)
103
Kruskals Algorithm in Action (1/5)
2
2
3
B
A
F
H
2
1
1
10
4
9
G
4
C
8
2
D
E
7
104
Kruskals Algorithm in Action (2/5)
2
2
3
B
A
F
H
2
1
1
10
4
9
G
4
C
8
2
D
E
7
105
Kruskals Algorithm in Action (3/5)
2
2
3
B
A
F
H
2
1
1
10
4
9
G
4
C
8
2
D
E
7
106
Kruskals Algorithm in Action (4/5)
2
2
3
B
A
F
H
2
1
1
10
4
9
G
4
C
8
2
D
E
7
107
Kruskals Algorithm Completed (5/5)
2
2
3
B
A
F
H
2
1
1
10
4
9
G
4
C
8
2
D
E
7
108
Why Greediness Works

• Proof by contradiction that Kruskals finds a
  minimum spanning tree
• Assume another spanning tree has lower cost than
  Kruskals.
• Pick an edge e1 (u, v) in that tree thats not
  in Kruskals.
• Consider the point in Kruskals algorithm where
  us set and vs set were about to be connected.
  Kruskal selected some edge to connect them call
  it e2 .
• But, e2 must have at most the same cost as e1
  (otherwise Kruskal would have selected it
  instead).
• So, swap e2 for e1 (at worst keeping the cost the
  same)
• Repeat until the tree is identical to Kruskals,
  where the cost is the same or lower than the
  original cost contradiction!

109
Data Structures for Kruskals Algorithm
Once
E times
Initialize heap of edges
Pick the lowest cost edge
buildHeap
findMin/deleteMin
E times
If u and v are not already connected connect u
and v.
union
runtime
E E log E E ack(E,V)
110
Data Structures for Kruskals Algorithm
Once
E times
Initialize heap of edges
Pick the lowest cost edge
buildHeap
findMin/deleteMin
E times
If u and v are not already connected connect u
and v.
union
runtime
E E log E E ack(E,V)
O(ElogE)
111
Prims Algorithm

• Can also find Minimum Spanning Trees using a
  variation of Dijkstras algorithm
• Pick a initial node
• Until graph is connected
• Choose edge (u,v) which is of minimum cost among
  edges where u is in tree but v is not
• Add (u,v) to the tree
• Same greedy proof, same asymptotic complexity

112
Coming Up

• Application Sentence Disambiguation
• All-pairs Shortest Paths
• NP-Complete Problems
• Advanced topics
• Quad trees
• Randomized algorithms

113
Sentence Disambiguation

• A person types a message on their cell phone
  keypad. Each button can stand for three
  different letter (e.g. 1 is a, b, or c), but
  the person does not explicitly indicate which
  letter is meant. (Words are separated by blanks
  the 0 key.)
• Problem How can the system determine what
  sentence was typed?
• My Nokia cell phone does this!
• How can this problem be cast as a shortest-path
  problem?

114
(No Transcript)
115
Sentence Disambiguation as Shortest Path

• Idea
• Possible words are vertices
• Directed edge between adjacent possible words
• Weight on edge from W1 to W2 is probability that
  W2 appears adjacent to W1
• Probabilities over what?! Some large archive
  (corpus) of text
• Word bi-gram model
• Find the most probable path through the graph

116
W11
W12
W13
W21
W23
W22
W11
W31
W33
W41
W43
117
Technical Concerns

• Isnt most probable actually longest (most
  heavily weighted) path?!
• Shouldnt we be multiplying probabilities, not
  adding them?!

118
Logs to the Rescue

• Make weight on edge fromW1 to W2 be
• - log P(W2 W1)
• Logs of probabilities are always negative
  numbers, so take negative logs
• The lower the probability, the larger the
  negative log! So this is shortest path
• Adding logs is the same as multiplying the
  underlying quantities

119
To Think About

• This really works in practice 99 accuracy!
• Cell phone memory is limited how can we use as
  little storage as possible?
• How can the system customize itself to a user?

120
Question

• Which graph algorithm is asymptotically better
• ?(VElogV)
• ?(V3)

121
All Pairs Shortest Path

• Suppose you want to compute the length of the
  shortest paths between all pairs of vertices in a
  graph
• Run Dijkstras algorithm (with priority queue)
  repeatedly, starting with each node in the graph
• Complexity in terms of V when graph is dense

122
Dynamic Programming Approach
123
Floyd-Warshall Algorithm

• // C adjacency matrix representation of graph
• // Cij weighted edge i-gtj or ? if
  none
• // D computed distances
• FW(int n, int C , int D )
• for (i 0 i lt N i)
• for (j 0 j lt N j)
• Dij Cij
• Dii 0.0
• for (k 0 k lt N k)
• for (i 0 i lt N i)
• for (j 0 j lt N j)
• if (Dik Dkj lt Dij)
• Dij Dik Dkj

Run time
How could we compute the paths?