Potential Functions and the Inefficiency of Equilibria

1
Potential Functions and the Inefficiency
of Equilibria

• Tim Roughgarden
• Stanford University

2
Pigou's Example

• Example one unit of traffic wants to go from s
  to t
• Question what will selfish network users do?
• assume everyone wants smallest-possible cost
• Pigou 1920

cost depends on congestion
c(x)x
s
t
c(x)1
no congestion effects
3
Motivating Example

• Claim all traffic will take the top link.
• Reason
• ? gt 0 ? traffic on bottom is envious
• ? 0 ? equilibrium
• all traffic incurs one unit of cost

Flow 1-?
c(x)x
s
t
c(x)1
this flow is envious!
Flow ?
4
Can We Do Better?

• Consider instead traffic split equally
• Improvement
• half of traffic has cost 1 (same as before)
• half of traffic has cost ½ (much improved!)

Flow ½
c(x)x
s
t
c(x)1
Flow ½
5
Braesss Paradox

• Initial Network

Cost 1.5
6
Braesss Paradox

• Initial Network Augmented Network

½
½
x
1
0
s
t
½
½
x
1
Cost 1.5
Now what?
7
Braesss Paradox

• Initial Network Augmented Network

x
1
0
s
t
x
1
Cost 1.5
Cost 2
8
Braesss Paradox

• Initial Network Augmented Network
• All traffic incurs more cost! Braess 68
• also has physical analogs Cohen/Horowitz 91

x
1
0
s
t
x
1
Cost 1.5
Cost 2
9
High-Level Overview

• Motivation equilibria of noncooperative network
  games typically inefficient
• e.g., Pigou's example Braess's Paradox
• don't optimize natural objective functions
• Price of anarchy quantify inefficiency w.r.t
  some objective function
• Our goal when is the price of anarchy small?
• when does competition approximate cooperation?
• benefit of centralized control is small

10
Selfish Routing Games

• directed graph G (V,E)
• source-destination pairs (s1,t1), , (sk,tk)
• ri amount of traffic going from si to ti
• for each edge e, a cost function ce()
• assumed continuous and nondecreasing

Examples (r,k1)
c(x)x
c(x)1
½
c(x)x
½
c(x)0
s1
t1
s1
t1
c(x)1
½
½
c(x)x
c(x)1
11
Outcomes Network Flows

• Possible outcomes of a selfish routing game
• fP amount of traffic choosing si-ti path P
• outcomes of game flow vectors f
• flow vector nonnegative and total flow ? fP on
  si-ti paths equals traffic rate ri (for all i)

12
Outcomes Network Flows

• Possible outcomes of a selfish routing game
• fP amount of traffic choosing si-ti path P
• outcomes of game flow vectors f
• flow vector nonnegative and total flow ? fP on
  si-ti paths equals traffic rate ri (for all i)
• Question What are the equilibria (natural
  selfish outcomes) of this game?

13
Nash Flows

• Def Wardrop 52 A flow is at Nash equilibrium
  (or is a Nash flow) if no one can switch to a
  path of smaller cost. I.e., all flow is routed
  on min-cost paths. given current edge
  congestion

Examples
1
½
x
x
s
t
s
t
1
1
½
½
x
1
x
1
0
0
1
s
t
s
t
x
x
1
1
½
14
Our Objective Function

• Definition of social cost total cost C(f)
  incurred by the traffic in a flow f.
• Formally if cP(f) sum of costs of
  edges of P (w.r.t. flow f), then
• C(f) ?P fP cP(f)

15
Our Objective Function

• Definition of social cost total cost C(f)
  incurred by the traffic in a flow f.
• Formally if cP(f) sum of costs of
  edges of P (w.r.t. flow f), then
• C(f) ?P fP cP(f)
• Example

x
½
s
t
Cost ½½ ½1 ¾
½
1
16
The Price of Anarchy

• Defn
• definition from Koutsoupias/Papadimitriou 99

price of anarchy of a game
obj fn value of selfish outcome

optimal obj fn value
Example POA 4/3 in Pigou's example
1
½
x
x
s
t
s
t
1
1
½
Cost 3/4
Cost 1
17
A Nonlinear Pigou Network

• Bad Example (d large)
• equilibrium has cost 1, min cost ? 0

18
A Nonlinear Pigou Network

• Bad Example (d large)
• equilibrium has cost 1, min cost ? 0
• ? price of anarchy unbounded as d -gt infinity
• Goal weakest-possible conditions under which
  P.O.A. is small.

19
When Is the Price of Anarchy Bounded?

• Examples so far
• Hope imposing additional structure on the cost
  functions helps
• worry bad things happen in larger networks

xd
x
1
s
t
0
s
t
1
x
1
20
Polynomial Cost Functions

• Def linear cost fn is of form ce(x)aexbe
• Theorem Roughgarden/Tardos 00 for every
  network with linear cost functions
• 4/3

cost of Nash flow
cost of opt flow
21
Polynomial Cost Functions

• Def linear cost fn is of form ce(x)aexbe
• Theorem Roughgarden/Tardos 00 for every
  network with linear cost functions
• 4/3
• Bounded-deg polys (w/nonneg coeffs) replace 4/3
  by T(d/log d)

cost of Nash flow
cost of opt flow
xd
tight example
s
t
1
22
A General Theorem

• Thm Roughgarden 02, Correa/Schulz/Stier Moses
  03 fix any set of cost fns. Then, a Pigou-like
  example 2 nodes, 2 links, 1 link w/constant cost
  fn) achieves worst POA

23
Interpretation

• Bad news inefficiency of selfish routing grows
  as cost functions become "more nonlinear".
• think of "nonlinear" as "heavily congested"
• recall nonlinear Pigou's example
• Good news inefficiency does not grow with
  network size or of source-destination pairs.
• in lightly loaded networks, no matter how large,
  selfish routing is nearly optimal

24
Benefit of Overprovisioning

• Suppose network is overprovisioned by ß gt 0 (ß
  fraction of each edge unused).
• Then Price of anarchy is
  at most ½(11/vß).
• arbitrary network size/topology,
  traffic matrix
• Moral Even modest (10) over-provisioning
  sufficient for near-optimal routing.

25
Potential Functions

• potential games equilibria are actually optima
  of a related optimization problem
• has immediate consequences for existence,
  uniqueness, and inefficiency of equilibria
• see Beckmann/McGuire/Winsten 56, Rosenthal
  73, Monderer/Shapley 96, for original
  references
• see Roughgarden ICM 06 for survey

26
The Potential Function

• Key fact BMV 56 Nash flows
  minimize potential function
  ?e ?f ce(x)dx (over all flows).

ce(fe)
0
e
0
fe
0
27
The Potential Function

• Key fact BMV 56 Nash flows
  minimize potential function
  ?e ?f ce(x)dx (over all flows).
• Lemma 1 locally optimal solutions are precisely
  the Nash flows (derivative test).
• Lemma 2 all locally optimal solutions are also
  globally optimal (convexity).
• Corollary Nash flows exist, are unique.

ce(fe)
0
e
0
fe
0
28
Consequences for the Price of Anarchy

• Example linear cost functions.
• Compare cost potential function
• C(f) ?e fe ce(fe) ?e ae fe be fe
• PF(f) ?e ?f ce(x)dx ?e (ae fe)/2 be fe

2
2
e
0
29
Consequences for the Price of Anarchy

• Example linear cost functions.
• Compare cost potential function
• C(f) ?e fe ce(fe) ?e ae fe be fe
• PF(f) ?e ?f ce(x)dx ?e (ae fe)/2 be fe
• cost, potential fn differ by factor of 2
• gives upper bound of 2 on price on anarchy
• C(f) 2PF(f) 2PF(f) 2C(f)

2
2
e
0
30
Better Bounds?

• Similarly proves bound of d1 for degree-d
  polynomials (w/nonnegative coefficients).
• not tight, but qualitatively accurate
• e.g., price of anarchy goes to infinity with
  degree bound, but only linearly
• to get tight bounds, need "variational
  inequalities"
• see my ICM survey for details

31
Variational Inequality

• Claim
• if f is a Nash flow and f is feasible, then
• ?e fe ce(fe) ?e f ce(fe)
• proof use that Nash flow routes flow on shortest
  paths (w.r.t. costs ce(fe))

e
32
Pigou Bound

• Recall goal want to show Pigou-like examples are
  always worst cases.
• Pigou bound given set of cost functions (e.g.,
  degree-d polys), largest POA in a network
• two nodes, two links
• one function in given set
• one constant function
• constant cost of fully congested top edge

xd
s
t
1
33
Pigou Bound (Formally)

• Let S a set of cost functions.
• e.g., polynomials with degree at most d,
  nonnegative coefficients
• Definition the Pigou bound a(S) for S is
• max
• max is over all choices of cost fns
  c in S, traffic rate r ? 0, flow y ? 0

r c(r)
xd
s
t
y c(y) (r-y) c(r)
1
34
Pigou Bound (Example)

• Let S c c(x) ax b linear functions
• Recall the Pigou bound a(S) for S is
• max
• max is over all choices of cost fns
  c in S, traffic rate r ? 0, flow y ?
  0
• choose c(x) x r 1 y 1/2 ? get 4/3
• calculus a(S) 4/3 d/ln d for deg-d
  polynomials

r c(r)
x
s
t
y c(y) (r-y) c(r)
1
35
Main Theorem (Formally)

• Theorem Roughgarden 02, Correa/Schulz/Stier
  Moses 03 For every set S, for every selfish
  routing network G with cost functions in C, the
  POA in G is at most a(S).
• POA always maximized by Pigou-like examples
• That is, if f and f are Nash optimal flows in
  G, then C(f)/C(f) a(S).
• example POA 4/3 if G has affine cost fns

36
Proof of General Thm

• Let f and f are Nash optimal flows in G.

37
Proof of General Thm

• Let f and f are Nash optimal flows in G.
• Step 1 for each e, invoke Pigou bound with c
  ce, y f, r fe
• a(S) ? fe ce(fe)/f ce(f) (fe -f )
  ce(fe)

e
e
e
e
38
Proof of General Thm

• Let f and f are Nash optimal flows in G.
• Step 1 for each e, invoke Pigou bound with c
  ce, y f, r fe
• a(S) ? fe ce(fe)/f ce(f) (fe -f )
  ce(fe)
• Step 2 rearrange and sum over e
• C(f) ?e f ce(f)

e
e
e
e
e
e
39
Proof of General Thm

• Let f and f are Nash optimal flows in G.
• Step 1 for each e, invoke Pigou bound with c
  ce, y f, r fe
• a(S) ? fe ce(fe)/f ce(f) (fe -f )
  ce(fe)
• Step 2 rearrange and sum over e
• C(f) ?e f ce(f) ? ?e fe ce(fe)/a(S)
  ?e (f - fe) ce(fe)

e
e
e
e
e
e
e
40
Proof of General Thm

• Let f and f are Nash optimal flows in G.
• Step 1 for each e, invoke Pigou bound with c
  ce, y f, r fe
• a(S) ? fe ce(fe)/f ce(f) (fe -f )
  ce(fe)
• Step 2 rearrange and sum over e
• C(f) ?e f ce(f) ? ?e fe ce(fe)/a(S)
  ?e (f - fe) ce(fe)
• Step 3 apply VI

e
e
e
e
e
e
e
? 0
41
Proof of General Thm

• Let f and f are Nash optimal flows in G.
• Step 1 for each e, invoke Pigou bound with c
  ce, y f, r fe
• a(S) ? fe ce(fe)/f ce(f) (fe -f )
  ce(fe)
• Step 2 rearrange and sum over e
• C(f) ?e f ce(f) ? ?e fe
  ce(fe)/a(S)
• Step 3 apply VI, done!

e
e
e
e
e
e
C(f)
42
Recap

• selfish routing simple, basic routing game
• inefficient equilibria Pigou Braess examples
• price of anarchy ratio of objective fn values of
  selfish optimal outcomes
• potential functions equilibria actually solving
  a related optimization problem
• immediate consequence for existence, uniqueness,
  and inefficiency of equilibria

43
Recap

• variational inequality inequality based on
  "first-order condition" satisfied by equilibria
• Pigou bound given a set of cost functions,
  largest POA in a Pigou-like example
• main result for every set of cost fns, Pigou
  bound is tight (all multicommodity networks)
• POA depends only on complexity of cost functions,
  not on complexity of network structure

44
Outline

• Part I The Price of Anarchy in Selfish Routing
  Games
• Part II The Price of Stability in Network
  Connectivity Games

45
Selfish Network Design

• Given G (V,E),
• fixed costs ce for all e ? E,
• k vertex pairs (si,ti)
• Each player wants to build a network in which its
  nodes are connected.
• Player strategy select a path connecting si to
  ti.
• Anshelevich et al 04

46
Shapley Cost Sharing

• How should multiple players
• on a single edge split costs?
• Natural choice is fair sharing,
• or Shapley cost sharing
• Players using e pay for it evenly
  ci(P) S ce/ke
• Each player tries to minimize its cost.

e ? P
47
Comparison to Selfish Routing

• Note like selfish routing, except
• finite number of outcomes
• in selfish routing, outcomes fractional flows
• positive (not negative) externalities
• cost function (per player) ce/ke
• Objective C Si ci(Pi) S ce
• where S union of Pi's

e ? S
48
What's the POA?

• Example

t1, t2, tk
t
1?
k
s
s1, s2, sk
49
What's the POA?

• Example

t1, t2, tk
t
t
1?
k
1?
k
s
s
s1, s2, sk
OPT (also Nash eq)
50
What's the POA?

• Example

t1, t2, tk
t
t
t
1?
k
1?
k
1?
k
s
s
s
s1, s2, sk
OPT (also Nash eq)
another Nash eq
51
Multiple Equilibria

• Moral in Shapley network design games, different
  Nash eq can have different costs.
• Recall
• Note not well defined if Nash eq not unique.
• which one do we look at?

obj fn value of selfish outcome
POA of a game

optimal obj fn value
52
The Price of Stability

• General definition of POA KP99
• POA k in last example, uninteresting

cost(worst NE) cost(OPT)
Price of Anarchy
53
The Price of Stability

• General definition of POA KP99
• POA k in last example, uninteresting
• Alternative
• POS 1 in last example

cost(worst NE) cost(OPT)
Price of Anarchy
cost(best NE) cost(OPT)
Price of Stability
54
The Price of Stability

• Note small price of stability only guarantees
  that some Nash eq has low cost.
• much weaker guarantee than small POA
• Interpretation best solution consistent with
  self-interested players
• natural outcome for centralized planner to
  suggest e.g., network protocol designer

55
Example High Price of Stability
t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
56
Example High Price of Stability
cost(OPT) 1e
t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
57
Example High Price of Stability
cost(OPT) 1e but not a NE player k
pays (1e)/k, could pay 1/k
t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
58
Example High Price of Stability
so player k would deviate
t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
59
Example High Price of Stability
now player k-1 pays (1e)/(k-1),
could pay 1/(k-1)
t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
60
Example High Price of Stability
so player k-1 deviates too
t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
61
Example High Price of Stability
Continuing this process, all players defect.
This is a NE! (the only Nash) cost 1

t
1
1
1
1
1
k
2
3
k-1
. . .
1?
1
2
3
k
k-1
0
0
0
0
0
1 1
2 k
Price of Stability is Hk T(log k)!
62
The Price of Stability of Selfish Network Design

• Thus the price of stability of selfish network
  design can be as high as ln k. k players
• Our goals in all such games,
• there is at least one pure-strategy Nash eq
• one of them has cost ln k OPT
• i.e. price of stability always ln k
• Anshelevich et al 04
• Technique potential function method.

63
Potential Functions

• Recall potential function ? of a game function
  optimized by selfish players
• not necessarily a natural objective function
• Defn ? (fn from outcomes to reals) is a
  potential function if for all outcomes S, players
  i, and deviations by i from S
• ?? ?ci

64
Potential Functions

• So potential fn tracks deviations by players
• Thus equilibria of game local optima of ?
• so finite potential games have pure-strategy Nash
  equilibria (proof just do "best-response
  dynamics") Monderer/Shapley 96
• precursors Rosenthal 73, Beckmann et al 56

65
Potential Functions

• So potential fn tracks deviations by players
• Thus equilibria of game local optima of ?
• so finite potential games have pure-strategy Nash
  equilibria (proof just do "best-response
  dynamics") Monderer/Shapley 96
• precursors Rosenthal 73, Beckmann et al 56
• Claim every Shapley network design game has a
  potential function.

66
Proof of Potential Function

• Define ?e(S) ce1 1/2 1/3 1/ke
• where ke is players using e in S. Hk
• Let ?(S) S ?e(S)
• Consider some solution S (a path for each
  player).
• Suppose player i is unhappy and decides to
  deviate.
• What happens to ?(S)?

e
e ? S
67
Proof of Potential Function

• ?e(S) ce1 1/2 1/3 1/ke
• Suppose player is new path includes e.
• i pays ce/(ke1) to use e.
• ?e(S) increases by the same amount.
• If player i leaves an edge e,
• ?e(S) exactly reflects the change in is
  payment.

ce1 1/2 1/ke
e
i
e
ce1 1/2 1/ke
68
Proof of Potential Function

• ?e(S) ce1 1/2 1/3 1/ke
• Suppose player is new path includes e.
• i pays ce/(ke1) to use e.
• ?e(S) increases by the same amount.
• If player i leaves an edge e,
• ?e(S) exactly reflects the change in is
  payment.

ce1 1/2 1/kece/(ke1)
e
i
e
ce1 1/2 1/ke -ce/ke
69
Bound on Price of Stability

• Compare cost potential function
• C(S) ?e ce
• PF(S) ?e ce1 1/2 1/3 1/ke
• cost, potential fn differ by factor of Hk
• gives upper bound of Hk on price on stability
• let S min-potential soln note also a Nash
  eq
• let S opt solution
• C(S) PF(S) PF(S) Hk C(S)

70
Undirected Networks

• Open Question what is the POS in undirected
  graphs?
• best known lower bound 12/7
• Fiat et al 06 O(log log k) for special case

71
Shapley Cost-Sharing

• Summary with Shapley cost sharing,
• POA k, even in undirected graphs
• POS Hk in directed graphs
• (unknown in undirected graphs)
• Question 1 can we do better?
• Question 2 subject to what?

72
In Defense of Shapley

• Essential properties (non-negotiable)
• "budget-balanced" (total cost shares cost)
• "local" (cost shares computed edge-by-edge)
• pure-strategy Nash equilibria exist
• Bonus good properties (negotiable)
• "uniform" (same definition for all networks)
• "fair" (characterizes Shapley)

73
Other Cost Shares?

• Theorem Chen/Roughgarden/Valiant 07 Shapley
  minimizes POS among all uniform protocols in
  directed graphs.
• Shapley justified on efficiency grounds!
• non-uniform schemes not well understood

74
Other Cost Shares?

• Theorem Chen/Roughgarden/Valiant 07 Shapley
  minimizes POS among all uniform protocols in
  directed graphs.
• Shapley justified on efficiency grounds!
• non-uniform schemes not well understood
• Theorem Chen/Roughgarden/Valiant 07 Can do
  much better in undirected graphs.
• can get POA O(log2 k)
• better for special cases or non-uniform protocols

75
Wrap-Up

• network games arise in many CS applications
• price of anarchy/stability/etc a flexible tool to
  measure inefficiency of selfish behavior
• future direction inform protocol design
• potential functions are an easy-to-use, versatile
  techniques to bound POA/POS
• many open questions...
• looking forward to future theorems from you!