Regret Minimization and the Price of Total Anarchy

1
Regret Minimization and the Price of Total Anarchy

• Paper by A. Blum, M. Hajiaghayi, K. Ligett,
  A.Roth
• Presented by Michael Wunder

2
Nash Anarchy vs. Total Anarchy

• In a multiagent setting, want to find the ratio
  between the socially optimal value and the
  selfish agent outcome
• Traditionally, assumed to be Nash, where no agent
  has incentive to change
• Can also find the price of total anarchy, when
  selfish agents act repeatedly to minimize regret
  over previous actions

3
Why Regret Minimization?

• Finding Nash equilibria can be computationally
  difficult
• Not clear that agents would converge to it, or
  remain in one if there are several
• Regret minimization is realistic because there
  are efficient algorithms that minimize regret, it
  is locally computed, and players improve by
  lowering regret

4
Results comparing prices

• Shows how PoTA compares with PoA
• Four classes of games
• Hotelling Games
• Valid Games
• Atomic Linear Congestion Games
• Parallel Link Congestion Games

5
Preliminaries (maximization)

• Ai set of pure strategies for player i
• Si set of mixed strategies for player i
• (distributions over Ai )

• Social Utility Function
• Individual utility function
• Strategy set if player i changes from si to si

6
Preliminaries (cont.)

• Socially Optimal Value

• Regret of Player i given action sets A
• The difference between action taken and best
  available action over all timesteps

• Price of Total Anarchy
• Ratio of social value of best strategies to the
  regret minimizers

7
Hotelling Games

• Problem k sellers must set up a vendor stand on
  a graph to sell to n tourists, who buy from first
  seller along a path
• Strategy set Ai V

S1
S2
T1
8
Hotelling Games cont.

• Social welfare at time t
• To maximize fairness (and maximize the lowest
  player), split all vertices equally

OPT n/k
Si
T1
9
Hotelling Games cont.

• Claim Price of anarchy (2k-2)/k
• Proof Consider alternate set
• Some player h achieves
• If player i plays same strategy as
• h, the expected payoff is
• Therefore, Price of Anarchy

10
Hotelling with Total Anarchy

• The price of total anarchy is also (2k-2)/k
• Proof from symmetry Let Oti be the set of plays
  at time t by players other than i
• ?it-gtu be the difference between expected payoff
  from choosing from Oti at time step u, and
  n/(2k-2)
• For all i, for all 1ltt, ultT ?it-gtu ?iu-gtt
  gt0
• Imagine a (2k-2) player game where there is a
  time t and a time u player for each original
  player but i
• If player i replaces a random player, ai
  n/(2k-2)

11
Hotelling Total Anarchy Proof

• If player i replaces a time t player, and all
  other time t players are removed, player is
  payoff only improves

• The expected payoff of player i from picking an
  action oti uniformly at random from Oti and
  playing over all T rounds

12
Generalized Hotelling Games

• The above proof does not use specifics of the
  game as described
• In general, PoTA is (2k-2)/k even in the presence
  of arbitrarily many Byzantine players making
  arbitrary decisions
• Regret-minimizing players may not converge to a
  Nash equilibrium, and play can cycle forever

13
Valid Games, Price of Anarchy

• Valid games are a broad class of games that
  includes a market sharing game, the facility
  location problem, and others. Example Cable
  television market sharing
• Game is bipartite graph G ((V,U),E). Each v in
  V is a player, each u in U is a market
• Markets have value and cost
• Players have budget
• Players may enter adjacent markets, and receive
  value of market divided by players in market

14
Valid Games Definition

• For a set function f, define the derivative of f
  at X in V in direction D in V-X to be fD(X)f(X
  U D)-f(X)
• A game is valid if
• For X in A, ? i(X)gt ? i(A) for all i in V A
  (submodularity)

(Vickrey)
15
Valid Games Price of Anarchy

• Vetta shows that for any Nash equilibrium
  strategy S, if ? is non-decreasing, ?(S) gt OPT/2
• PoTA matches PoA
• While PoA does not hold with the addition of
  Byzantine players, PoTA does

16
Total Anarchy w/Byzantines
Show by contradiction
17
Total Anarchy w/Byzantines
So there is a regret minimizing player i which
violates the regret minimizing condition.
18
Atomic Congestion Games

• An atomic congestion game is a minimization game
  consisting of k players and a set of facilities V
  (ai over Vi)
• Each facility e has a latency function fe(le)
• Each player i has weight wi (unweighted wi 1)

• Player i experiences cost
• load on facility le

19
Atomic Congestion Games

• Linear Edge Costs
• Social utility

• Consider two types of social utility function
  linear and makespan in parallel link networks

20
Congestion Games PoA

• Price of Anarchy with unweighted players, sum
  social utility function, and linear cost
  functions is 2.5 (Christodoulou et al. 2005)
• Claim Price of Total Anarchy is the same By
  assuming regret minimization, each players time
  average cost is no better than the cost of best
  action in hindsight. That is, no better than
  optimal strategy.

21
Congestion Games PoTA

• Proof for all i

• Summing over
• all players
• After math

22
Congestion Games PoTA

• For atomic congestion games with unweighted
  players, sum social function, and polynomial
  latency functions of degree d, PoTA lt dd1-o(1)

23
Parallel Link Congestion Game

• n identical links, k weighted players
• Each player pays sum of weights of jobs on link
  chosen
• Social cost is total weight of worst loaded link
  (makespan)

24
2 Parallel Links PoTA

• For 2 links, Price of Total Anarchy matches Price
  of Anarchy 3/2, but only in expectation

25
n Parallel Links PoTA

• With n parallel links, PoTA is not the same as
  PoA
• PoTA with makespan utility and n links is O(n½),
  versus O(log n/ log log n) for PoA
• Proof with n links and n players, OPT 1
• We can construct a situation with negative regret
  but with maximum latency O(n½)

26
n Parallel Links PoTA

• Divide the players into groups of size n½/2 and
  rotate each group to take link 1
• The rest distribute evenly on the remaining links

• Each player has average latency 5/4 ½ (n-½)
• If a player plays a fixed link, the average
  latency is 2 ½ (n-½)
• Therefore, players have negative regret but
  maximum latency O(n½)

27
Conclusion

• Thank you!