Evaluating Algorithmic Design Paradigms

1
Evaluating Algorithmic Design Paradigms

• Sashka Davis
• Advised by Russell Impagliazzo
• UC San Diego
• October 6, 2006

2
Suppose you have to solve a problem ?
No Greedy alg. exists? Or I didnt think of one?
Is there a Dynamic Programming algorithm that
solves ??
Is there a Greedy algorithm that solves ??
Is there a Backtracking algorithm that solves ??
Eureka! I have a DP Algorithm!
No Backtracking agl. exists? Or I didnt think of
one?
Is my DP algorithm optimal or a better one exists?
3
Suppose we a have formal model of each
algorithmic paradigm
Is there a Dynamic Programming alg. that solves ??
Is my algorithm optimal, or a better DP algorithm
exists?
No Greedy algorithm can solve ? exactly.
Is there a Greedy algorithm that solves ??
Is there a Backtracking algorithm that solves ??
No Backtracking algorithm can solve ? exactly.
DP helps!
Yes, it is! Because NO DP alg. can solve ? more
efficiently.
4
The goal

• To build a formal model of each of the basic
  algorithmic design paradigms which should capture
  the strengths of the paradigm.
• To develop lower bound technique, for each formal
  model, that can prove negative results for all
  algorithms in the class.

5
Using the framework we can answer the following
questions

• 1. When solving problems exactly
• What algorithmic design paradigm can help?
• No algorithm within a given formal model can
  solve the problem exactly.
• We find an algorithm that fits a given formal
  model.
• 2. Is a given algorithm optimal?
• Prove a lower bound matching the upper bound for
  all algorithms in the class.
• 3. Solving the problems approximately
• What algorithmic paradigm can help?
• Is a given approximation scheme optimal within
  the formal model?

6
Current hierarchy of formal models BNR03,
DI04, ABBO05,BODI06
Dynamic Programming
Backtracking Simple DP
Greedy
pBP prioritized Branching
Programs
pBT prioritized Branching
Trees
PRIORITY
PRIORITY
7
Some of our results

• PRIORITY algorithms (formal model of greedy
  algoritms)
• Dijkstras algorithm solves the Single Source
  Shortest Path (SSSP) in graphs with non-negative
  edges and cannot be simplified.
• No PRIORITY algorithm can solve the SSSP in
  graphs with negative weights.
• Proved lower bounds on the approximation ratio
  for Weighted Vertex Cover, Maximum Independent
  Set, and Steiner Tree problems.
• pBT algorithms (formal model of BT and simple DP
  algoritms)
• There is no efficient pBT algorithm which finds
  the shortest path in graphs with negative weights
  but no negative cycles efficiently.
• pBP algorithms ( formal model of Dynamic
  Programing)
• There is no efficient pBP algorithm which finds
  the maximum matching in bipartite graphs.

8
Some of our results
Maximum Matching in Bipartite graphs
Shortest Path in negative graphs no cycles
Shortest Path in no-negative graphs
Flow Algorithms
Bellman-Ford
pBP
pBT
ADAPTIVE PRIORITY
Dijkstras
FIXED
PRIORITY
Prims
Kruskals
Minimum Spanning Tree
9
PRIORITY a formal model of greedy algorithms

• Consider Kruskals algorithm
• orders edges of the graph ONCE according to
  weight,
• Inspects the next edge according to the order and
  makes irrevocable decision, to add or not, to the
  solution (MST)
• Consider Primss algoritm
• Proceeds in iteration
• each iteration orders edges in the cut in
  non-descending order according to weight,
• Inspects the next edge according to the order and
  makes irrevocable decision, to add or not, to the
  solution MST
• Questions
• Can we canonize all ADAPTIVE algorithms?
• Does there exist a FIXED priority algorithm for
  SSSP?

10
Consider one iteration of Dijkstras algorithm
NNot Yet Reached
RReached
d(1)
d(2)
u
s
t
v
d(3)
d(4)
Suppose d(3)mind(1),d(2),d(3),d(4)
then (u,v) is added to the solution.
11
Can Dijkstras algorithm be simplified?
ADAPTIVE PRIORITY

• FIXED priority algorithm
• Orders edges ONCE
• Inspects an edge makes a decision

?
Is there a FIXED priority algorithm that solves
SP?
If there is no FIXED priority algorithm for
ShortPath problem then Dijkstras algorithm
cannot be simplified.
12
ShortPath problem

• ShortPath problem
• Given a graph G(V,E) and s,t in V.
• Find the shortest path from s to t in G.
• Instance set of edges
• Solution a path in G connecting s, t
• Theorem There is no FIXED priority algorithm
  that solves ShortPath problem exactly.
• Corollaries
• Dijkstra solves the problem exactly and hence
  cannot be simplified
• The classes of FIXED and ADAPTIVE priority
  algorithms are distinct.

13
Lower bound setting

• Lower bound is a game between Adversary and
  Solver
• Existence of a FIXED priority algorithm is a
  strategy for Solver
• Existence of a strategy for the Adversary
  establishes the lower bound
• The winning strategy for the Adversary presents a
  nemesis graph, which can be modified so that the
  Solver either
• fails to output a solution
• outputs a path, but not the shortest one

14
Theorem 1 proof sketchThe Adversarys graph
15
Modification of the graph
If Solver considers edge (y,1) before edge (z,1)

• then the Adversary presents

u,k
a
y,1
t
s
x,1
z,1
b
16
If Solver considers (y,1) before (z,1)

• The Algorithm selects (y,1) first
• Case 1 (y,1) is added to MST
• Case 2 (y,1) is NOT added to MST
• The cases when the algorithm selects (u,k) or
• (x,1) first reduce to Case 1 and 2.

17
Case 1 Solver decides to add (y,1)
18
Case 2 Solver decides (y,1) is NOT part of the
path
u,k
a
y,1
t
s
x,1
b
Solver has failed to construct a path.
Adversary outputs a solution u,y
and wins the game.
19
The outcome of the game

• Solver fails to construct a tree in which t is
  reachable from s.
• When Solver succeeds, the approximation ratio
  achieved is (k1)/2.
• The Adversary can set k arbitrarily large and
  thus can gain any advantage.
• No FIXED priority algorithm can solve the
  ShortPath problem.
• Dijkstra solves ShortPath problem, hence it
  cannot be simplified.
• FIXED priority algorithms are properly contained
  in ADAPTIVE priority algorithms.

20
Conclusions

• Building formal models of basic algorithmic
  design paradigm and developing general lower
  bound techniques we can answer
• 1. What algorithmic design paradigm can help?
• (a) Certify the problem as hard by Proving NO
  algorithms in the class can solve it.
• (b) Or we find an algorithm within a given formal
  class.
• 2. If we solved the problem, then we can prove
  that our algorithm is optimal.
• By Proving a matching lower bound for ALL
  algorithms in the class.
• 3. If No technique can solve the problem exactly
  then we use the framework to
• (a) How good an approximation scheme can we get
  using different algorithmic techniques?
• (b) Certify that our approximation algorithm is
  optimal.