Approximation Algorithms:

1
Approximation Algorithms
problems, techniques, and their use in game
theory

• Éva Tardos
• Cornell University

2
What is approximation?

• Find solution for an optimization problem
  guaranteed to have value close to the best
  possible.
• How close?
• additive error (rare)
• E.g., 3-coloring planar graphs is NP-complete,
  but 4-coloring always possible
• multiplicative error
• ?-approximation finds solution for an
  optimization problem within an ? factor to the
  best possible.

3
Why approximate?

• NP-hard to find the true optimum
• Just too slow to do it exactly
• Decisions made on-line
• Decisions made by selfish players

4
Outline of talk

• Techniques
• Greedy
• Local search
• LP techniques
• rounding
• Primal-dual

• Problems
• Disjoint paths
• Multi-way cut and labeling
• network design, facility location

• Relation to Games
• local search ? price of anarchy
• primal dual ? cost sharing

5
Max disjoint paths problem

• Given graph G, n nodes, m edges, and source-sink
  pairs.
• Connect as many as possible via edge-disjoint
  path.

t
s
t
s
t
s
s
t
6
Greedy Algorithm
Greedily connect s-t pairs via disjoint paths, if
there is a free path using at most m½ edges
If there is no short path at all, take a single
long one.
7
Greedy Algorithm
Theorem m½ approximation. Kleinberg96 Proo
f One path used can block m½ better paths
Essentially best possible m½-? lower bound
unless PNP by Guruswami, Khanna, Rajaraman,
Shepherd, Yannakakis99
8
Disjoint pathsopen problem

• Connect as many as pairs possible via paths where
  2 paths may share any edge

• Same practical motivation
• Best greedy algorithm n½ - (and also m1/3 -)
  approximation Awerbuch, Azar, Plotkin93.
• No lower bound

9
Outline of talk

• Techniques
• Greedy
• Local search
• LP techniques
• rounding
• Primal-dual

• Problems
• Disjoint paths
• Multi-way cut and labeling
• network design, facility location

• Relation to Games
• local search ? Price of anarchy
• primal dual ? Cost sharing

10
Multi-way Cut Problem

• Given
• a graph G (V,E)
• k terminals s1, , sk
• cost we for each edge e
• Goal Find a partition that separates terminals,
  and minimizes the cost
• ?e separated we

Separated edges
s3
s1
s4
s2
11
Greedy Algorithm

• For each terminal in turn
• Find min cut separating si from other terminals

s1
s3
s4
s2
s1
s3
The next cut
s4
s2
12
Theorem Greedy is a2-approximation

• Proof Each cut costs at most the optimums cut
  Dahlhaus, Johnson, Papadimitriou, Seymour, and
  Yannakakis94
• Cuts found by algorithm

s3
s1
Optimum partition
s4
s2
Selected cuts, cheaper than optimums cut,
but each edge in optimum is counted twice.
13
Multi-way cuts extension

• Given
• graph G (V,E), we?0 for e ?E
• Labels L1,,k
• Lv ? L for each node v
• Objective Find a labeling of nodes such that
  each node v assigned to a label in Lv and it
  minimizes cost ?e separated we

part 3
part 1
Separated edges
part 2
part 4
14
Example
s1
cheap
medium
expensive
s2
s3

• Does greedy work?
• For each terminal in turn
• Find min cut separating si from other terminals

15
Greedy doesnt work

• Greedy
• For each terminal in turn
• Find min cut separating si from other terminals
• The first two cuts

s1
Remaining part not valid!
s2
s3
16
Local search

• Boykov Veksler Zabih CVPR98 2-approximation
• Start with any valid labeling.
• 2. Repeat (until we are tired)
• Choose a color c.
• b. Find the optimal move where a subset of the
  vertices can be recolored, but only with the
  color c.
• (We will call this a c-move.)

17
A possible -move
Thm Boykov, Vekler, Zabih The best -move
can be found via an (s,t) min-cut
18
Idea of the flow networkfor finding a -move
s all other terminals retain current color
G
sc change color to c
19
Theorem local optimum is a 2-approximation
Partition found by algorithm
Cuts used by optimum
The parts in optimum each give a possible local
move
20
Theorem local optimum is a 2-approximation
Partition found by algorithm
Possible move using the optimum
Changing partition does not help ? current cut
cheaper Sum over all colors Each edge in optimum
counted twice
21
Metric labeling ? classification open problem

• Given
• graph G (V,E) we?0 for e ?E
• k labels L
• subsets of allowed labels Lv
• a metric d(.,.) on the labels.
• Objective Find labeling f(v)?Lv for each node v
  to minimize
• ?e(v,w) we d(f(v),f(w))

Best approximation known O(ln k ln ln k)
Kleinberg-T99
22
Outline of talk

• Techniques
• Greedy
• Local search
• LP techniques
• rounding
• Primal-dual

• Problems
• Disjoint paths
• Multi-way cut and labeling
• network design, facility location

• Relation to Games
• local search ? Price of anarchy
• primal dual ? Cost sharing

23
Using Linear Programs for multi-way cuts

• Using a linear program
• fractional cut
• ? probabilistic assignment of nodes to parts

Idea Find optimal fractional labeling via
linear programming
24
Fractional Labeling

• Variables
• 0 ? xva ? 1 pnode, alabel in Lv
• xva ? fraction of label a
• used on node v
• Constraints

? xva 1
for all nodes v ? V
a?Lv

• each node is assigned to a label
• cost as a linear function of x
• ? we ½ ? xua - xva

e(u,v)
a?L
25
From Fractional x to multi-way cut

• The Algorithm (Calinescu, Karloff, Rabani, 98,
  Kleinberg-T,99)
• While there are unassigned nodes
• select a label a at random

26
The Algorithm (Cont.)

• While there are unassigned nodes
• select a label a at random

select 0 ? ? ? 1 at random assign all unassigned
nodes v to selected label a if xva ? ?
27
Why Is This Choice Good?

• select 0 ? ? ? 1 at random
• assign all unassigned nodes v to selected label a
  if xva ? ?
• Note
• Probability of assigning node v to label a is
  ? xva
• Probability of separating nodes u and v in this
  iteration is ?xua xva

28
From Fractional x to Multi-way cut (Cont.)

• Theorem Given a fractional x, we find multi-way
  cut with expected
• separation cost ? 2 (LP cost of x)
• Corollary if x is LP optimum . ?
  2-approximation
• Calinescu, Karloff, Rabani, 98
• 1.5 approximation for multi-way cut (does not
  work for labeling)
• Karger, Klein, Stein, Thorup, Young99 improved
  bound ? 1.3438..

29
Outline of talk

• Techniques
• Greedy
• Local search
• LP techniques
• rounding
• Primal-dual

• Problems
• Disjoint paths
• Multi-way cut and labeling
• network design, facility location

• Relation to Games
• local search ? Price of anarchy
• primal dual ? Cost sharing

30
Metric Facility Location

• F is a set of facilities (servers).
• D is a set of clients.
• cij is the distance between any i and j in D ? F.
• Facility i in F has cost fi.

31
Problem Statement
We need to 1) Pick a set S of facilities to
open. 2) Assign every client to an open
facility (a facility in S). Goal Minimize
cost of S ?p dist(p,S).
32
What is known?

• All techniques can be used
• Clever greedy Jain, Mahdian, Saberi 02
• Local search starting with Korupolu, Plaxton,
  and Rajaraman 98, can handle capacities
• LP and rounding starting with Shmoys, T, Aardal
  97
• Here primal-dual starting with Jain-Vazirani99

33
What is the primal-dual method?

• Uses economic intuition from cost sharing
• For each requirement, like
• ?a?Lv xva 1, someone has to pay to make it
  true
• Uses ideas from linear programming
• dual LP and weak duality
• But does not solve linear programs

34
Dual Problem Collect Fees

• Client p has a fee ap (cost-share)
• Goal collect as much as possible max ?p ap
• Fairness Do no overcharge for any subset A of
  clients and any possible facility i we must have
• ?p ?A ap dist(p,i) ? fi

amount client p would contribute to building
facility i.
35
Exact cost-sharing

• All clients connected to a facility
• Cost share ap covers connection costs for each
  client p
• Costs are fair
• Cost fi of selecting a facility i is covered by
  clients using it
• ?p ap f(S) ?p dist(p,S) , and
• both facilities are fees are optimal

36
Approximate cost-sharing

• Idea 1 each client starts unconnected, and with
  fee ap0
• Then it starts raising what it is willing to pay
  to get connected
• Raise all shares evenly a
• Example

client
possible facility with its cost
37
Primal-Dual Algorithm (1)
Its a 1 share could be used towards building a
connection to either facility
a 1

• Each client raises his fee a evenly what it is
  willing to pay

38
Primal-Dual Algorithm (2)
a 2
Starts contributing towards facility cost

• Each client raises evenly what it is willing to
  pay

39
Primal-Dual Algorithm (3)
a 3
Three clients contributing

• Each client raises evenly what it is willing to
  pay

40
Primal-Dual Algorithm (4)
4
a 3
Open facility
clients connected to open facility

• Open facility, when cost is covered by
  contributions

41
Primal-Dual Algorithm Trouble
4
i
j
a 3
p
Open facility

• Trouble
• one client p connected to facility i, but
  contributes to also to facility j

42
Primal-Dual Algorithm (5)
ghost
4
i
j
a 3
p
Open facility

• Close facility j will not open this facility.
• Will this cause trouble?
• Client p is close to both i and j ? facilities i
  and j are at most 2a from each other.

43
Primal-Dual Algorithm (6)
ghost
a 3
4
a 6
a 3
a 3
Open facility
no not need to pay more than 3

• Not yet connected clients raise their fee evenly
• Until all clients get connected

44
Feasibility fairness ??

• ? All clients connected to a facility
• ? Cost share ap covers connection costs of
  client p
• ? Cost fi of opening a facility i is covered by
  clients connected to it
• ?? Are costs fair ??

45
Are costs fair??

• a set of clients A, and any possible facility i
  we have ?p ?A ap dist(p,i)? fi
• Why? we open facility i if there is enough
  contribution, and do not raise fees any further
• But closed facilities are ignored! and may
  violate fairness

46
Are costs fair??
j
i
4
aq4
Closed facility, ignored
open facility
p
cause of closing
Fair till it reaches a ghost facility. Let aq
? aq be the fee till a ghost facility is reached
47
Feasibility fairness ??

• ? All clients connected to a facility
• ? Cost share ap covers connection costs for
  client p
• ? Cost ap also covers cost of selected a
  facilities
• ? Costs ap are fair
• How much smaller is a ? a ??

48
How much smaller is a ? a?

• q client met ghost facility j
• j became a ghost due to client p

j
i
4
q
p

• p stopped raising its share first
• ap ? aq ? aq
• Recall dist(i,j) ? 2 ap, so
• aq ? aq 2 ap ? 3aq

49
Primal-dual approximation

• The algorithm is a 3-approximation
  algorithm for the facility location problem
• Jain-Vazirani99, Mettu-Plaxton00
• Proof
• Fairness of the ap fees ?
• ?p ap ? min cost max ? min
• cost-recovery
• f(S) ?p dist(p,S) ?p ap
• a ? 3aq
• 3-approximation algorithm

50
Outline of talk

• Techniques
• Greedy
• Local search
• LP techniques
• rounding
• Primal-dual

• Problems
• Disjoint paths
• Multi-way cut and labeling
• network design, facility location

• Relation to Games
• primal dual ? Cost sharing
• local search ? Price of anarchy

51
primal dual ? Cost sharing

• Dual variables ap are natural cost-shares
• Recall
• fair no set is overcharged
• core allocation
• ?p ?Aap dist(p,i) ? fi for all A and i.
• Chardaire98 Goemans-Skutella00 strong
  connection between core cost-allocation and
  linear programming dual solutions
• See also Shapley67, Bondareva63 for other games

52
Primal-Dual ? Cost-sharing

• Primal dual for each requirement someone
  willing to pay to make it true
• Cost-sharing only players can have shares.
• Not all requirements are naturally associated
  with individual players.
• Real players need to share the cost.

53
primal dual ? Cost sharing

• Fair ? no subset is overcharged
• Stronger desirable property population monotone
  (cross-monotone)
• Extra clients do not increase cost-shares.
• Spanning-tree game Kent and Skorin-Kapov96 and
  Jain Vazirani01
• Facility location, single source rent-or-buy
  Pal-T02

54
Local search (for facility location)

• Local search simple search steps to improve
  objective
• add(s) adds new facility s
• delete(t) closes open facility t
• swap(s,t) replaces open facility s by a new
  facility t
• Key to approximation bound
• How bad can be a local optima?
• 3-approximation Charikar, Guha00

55
Local search ? Price of anarchy in games

• Price of anarchy facilities are operated by
  separate selfish agents
• Agents open/close facilities when it benefits
  their own objective.
• Agents best response dynamic
• Simple local steps analogous to local search.
• Price of anarchy
• How bad can be a stable state?
• 2-approximation in a related maximization game
  Vetta02

56
Conclusions for approximation

• Greedy, Local search
• clever greedy/local steps can lead to great
  results
• Primal-dual algorithms
• Elegant combinatorial methods
• Based on linear programming ideas, but fast,
  avoids explicitly solving large linear programs
• Linear programming
• very powerful tool, but slow to solve
• Interesting connections to issues in game theory