Nash Equilibrium

1
Nash Equilibrium

• A strategy profile s (s1, , sn) is a Nash
  Equilibrium if, for all agents i, si is a best
  response to s-i.
• two strategies si and s-i are in Nash equilibrium
  if
• under the assumption that agent i plays si, agent
  j can do no better than play s-i and
• under the assumption that agent j plays s-i,
  agent i can do no better than play si.
• Each agent chooses a strategy that is a best
  response to the other agents strategies.

2
Nash Equilibrium (NE)

• How hard is the Nash Equilibrium to compute?

3
Normal Form Games

• Finite, n-person normal form game (N, A, O, m,
  u)
• N is a finite set of n players, indexed by i
• A A1, . . . , An is a set of actions for each
  player i
• a A is an action profile
• O is a set of outcomes
• m A O
• u u1, ... , un, a utility function for each
  player, where ui A

4
Game Definitions

• Common payoff games a game where all action
  profiles a A1 x x An for any pair of agents i,
  j, result in ui(a) uj(a).
• Constant sum games A game in which a constant c
  exists st for each strategy profile a A1 x A2 it
  is the case that u1(a) u2(a) c.

5
Linear Programming

• A linear program is defined by
• A set of real-valued variables
• A linear objective function
• A weighted sum of the variables
• A set of linear constraints
• The requirement that a weighted sum of the
  variables must be greater than or equal to some
  constant

6
Linear programming

• maximize Si wixi
• subject to Si wcixi bc c C
• xi 0 xi X
• These problems can be solved in polynomial time
  using interior point methods.
• Interestingly, the (worst-case exponential)
  simplex method is often faster in practice.

7
NE of two-player zero-sum games

• Easiest NE to find via linear programming in
  polynomial time.
• Let G (1,2, A1, A2, u1, u2) be a two-player
  zero-sum game.
• Ui is the unique expected utility for player i
  in equilibrium.
• We know that the players interests are opposing,
  what does this tell us about U1 and U2?
• U1 - U2

8
NE of two-player zero-sum games

• minimize U1
• subject to u1(aj1, ak2) sk2 U1
  j A1
• sk2 1
• sk2 0 k A2
• Variables
• U1 is the expected utility for player 1
• sa22 is player 2s probability of playing action
  a2 under his mixed strategy
• each u1(a1, a2) is a constant.

9
NE of two-player zero-sum games

• minimize U1
• subject to u1(aj1, ak2) sk2 rj1 U1
  j A1
• sa22 1
• sk2 0 k A2
• rj1 0 j A1
• Introduce slack variables, rj1.

10
NE of two-player general-sum games

• The NE of a two-player general-sum game cannot be
  represented by a linear program. Why?

11
NE of two-player general-sum games

• Computation of the NE in this case is not
  NP-Complete. Why?
• However, it does appear to be in the PPAD
  complexity class.

12
NE of two-player general-sum games

• PPAD Class of problems
• Define a family of directed graphs, G(n).
• Let each graph in G(n) contain a number of nodes
  that is exponential in n.
• Let Parent N N and Child N N
• Let there be one graph g G(n) for every such
  pair of Parent and Child functions as long as
• An edge exists from a node j to a node k iff
  Parent(k) j and Child(j) k.
• There must exist one distinguished node 0 N
  with exactly zero parents.

13
NE of two-player general-sum games

• Reformulate previous solution to explicitly
  consider both players.
• u1(aj1, ak2) sk2 rj1 U1 j A1
• u2(aj1, ak2) sk1 rj2 U2 k A1
• sj1 1, sk2 1
• sj1 0, sk2 0 j A1, k A2
• rj1 0, rk2 0 j A1 , k A2
• rj1 sj1 0, rk2 sk2 0 j A1 , k A2

14
NE of two-player general-sum games

• The complementarity condition requires that when
  an action is played by a particular player with
  positive probability, the corresponding slack
  variable must be zero.
• Each slack variable represents the players
  incentive to deviate from the corresponding
  action.
• Linear Complementarity Problem (LCP)

15
NE of two-player general-sum games

• Solve the LCP with the Lemke-Howson algorithm

a12
a22
How many pure strategies does agent a1 have?
a11
What about agent a2?
a21
a31
16
NE of two-player general-sum games

• Lemke-Howson algorithm

a12
a22
a11
a21
a31
How do we represent a2s Strategy space?
How do we represent a1s Strategy space?
17
NE of two-player general-sum games

• Lemke-Howson algorithm
• Label the strategies.
• A pair of strategies (s1, s2) is a NE iff it is
  completely labeled (L(s1) L(s2) A1 A2).

s31
s22
Where are the multiply labeled points?
(0,0,1)
(0,1)
(0,1/3,2/3)
(1/3, 2/3)
(1,0,0)
s11
(2/3, 1/3)
s12
(2/3,1/3,0)
(1,0)
(0,1,0)
s21
18
NE of two-player general-sum games

• Lemke-Howson algorithm
• Identify the NE

(0,0,1)
a11 , a21 , a12
(0,1)
a11 , a12
(0,1/3,2/3)
(1/3, 2/3)
a11 , a12 , a22
a11 , a21
a21 , a31 , a12
(1,0,0)
(2/3, 1/3)
(0,0,0)
a21 , a31
a11 , a21 , a31
(2/3,1/3,0)
a31 , a12 , a22
(0,0)
(1,0)
a31 , a22
a11 , a22
(0,1,0)
a11 , a31 , a22
((0,0,1), (0,1)), ((0,1/3,2/3), (2/3,
1/3)), ((2/3,1/3,0), (1/3, 2/3))
19
NE of two-player general-sum games

• Lemke-Howson algorithm
• Initialize
• (s1, s2) (0, 0)
• Find an s1 G1 s.t. s1 is adjacent to 0
• x 1
• Repeat
• sx sx
• let aji be the label that occurs in both s1 and
  s2
• Find an sx Gx s.t. sx is adjacent to sx and
  aji L(sx)
• x 3 x
• Until (s1, s2) is a completely labeled pair.

20
NE of two-player general-sum games
(0,0,1)
(0,0,1)
a11 , a21 , a12
a11 , a21 , a12
(0,1)
(0,1)
a11 , a12
a11 , a12
(0,1/3,2/3)
(0,1/3,2/3)
(1/3, 2/3)
(1/3, 2/3)
(1/3, 2/3)
a11 , a12 , a22
a11 , a12 , a22
a11 , a21
a11 , a21
a11 , a21
a21 , a31 , a12
(1,0,0)
(2/3, 1/3)
(2/3, 1/3)
(2/3, 1/3)
(0,0,0)
(0,0,0)
a21 , a31
a21 , a31
a21 , a31
a11 , a21 , a31
a11 , a21 , a31
(2/3,1/3,0)
(2/3,1/3,0)
a31 , a12 , a22
a31 , a12 , a22
(0,0)
(1,0)
(1,0)
(0,0)
(1,0)
a31 , a22
a31 , a22
a31 , a22
a11 , a22
a11 , a22
(0,1,0)
(0,1,0)
a11 , a31 , a22
a11 , a31 , a22
((2/3,1/3,0), (1/3,2/3)) ((0,0,0), (0,0))
((0,1,0), (0,1)) ((2/3,1/3,0), (0,1))
((2/3,1/3,0), (1/3,2/3))
((2/3,1/3,0), (1/3,2/3)) ((2/3,1/3,0),
(1/3,2/3)) ((0,1/3,2/3)), (1/3,2/3)
((0,1/3,2/3), (2/3, 1/3))
((0,0,1), (1,0)) ((0,0,0), (0,0)) ((0,0,1)),
(0,0) ((0,0,1), (1,0))
21
NE of two-player general-sum games

• Lemke-Howson algorithm
• Advantages
• Guaranteed to find a sample NE.
• Non-determinism is concentrated in the first
  move.
• Disadvantages
• Not guaranteed to find all NE.
• Does not provide guidance on choosing a good
  first move.

22
NE of two-player general-sum games

• Heuristics and the Support-Enumeration Method can
  be combined to provide an algorithm to find NE.
• Search for NE can be reduced to searching the
  space of supports.
• Use a feasibility program that tests the
  specified supports.
• Complete
• Worst case performance exponential

23
Specific NE of General-sum Games

• Idea is to find an equilibrium with a specific
  property.
• Properties
• Uniqueness
• Pareto optimal
• Guaranteed payoff
• Guaranteed social welfare
• Subset inclusion
• Subset containment
• NP-Hard when applied to NE.

24
NE for n-player general-sum games

• Cannot formulate as a linear complementarity
  problem.
• Sequence of linear complementarity problems
  (SLCP).
• Each LCP is an approximation of the problem and
  is used to developed the next approximation in
  the sequence.

25
NE for n-player general-sum games

• Formulate the problem as a minimum of a function.
• Constrained optimization problem
• Unconstrained optimization problem
• Disadvantages
• Both have local minima that do not correspond to
  the NE.

26
NE for n-player general-sum games

• Simplicial subdivision algorithms
• Consider
• the space of mixed strategies is a simlpex
• The players best response is a function from
  points on the simplex to other points on the
  simplex.
• Scarfs algorithm locates the fixed points.
• Add a variable that expresses the accuracy of the
  current iterations approximation.
• Worst case complexity exponential in the number
  of players and the number of digits of accuracy.

27
NE for n-player general-sum games

• Generalize SEM to the n-player case
• The feasibility program becomes non-linear.
• Algorithm must accommodate multiple variables in
  the feasibility problem.
• Use standard numerical techniques for non-linear
  optimization.
• Reverse the lexicographic ordering between size
  and balance of supports.

28
All NE of General-sum Games

• Idea is to determine all equilibria of a game.
• Important when designing a game and need to know
  all possible stable outcomes.
• Worst case exponential in the number of actions
  for each player.

29
Dominant Strategies

• A strategy dominates another when the first
  strategy is always at least as good as the
  second, independent of the other players
  actions.
• Iterative removal
• Strictly dominant strategies order does not
  matter.
• Very weakly and weakly dominant strategies
  removal order can have an affect.
• Potentially remove some equilibrium of the
  original game.
• Potentially remove a larger set of strategies and
  result in a smaller game.

30
Domination by a Pure Strategy

• for all pure strategies ai Ai for player i where
  ai ? si do
• dom true
• for all pure strategy profiles a-i A-i for the
  players other than i do
• if ui(si, a-i) ui(ai, a-i) then
• dom false
• break
• end if
• end for
• if dom true then return true
• end for
• return false

31
Domination by a Mixed Strategy

• Recall that mixed strategies cannot be
  enumerated.
• Strict Domination
• Requires a linear program.
• Minimize
• Subject to

32
Domination by a Mixed Strategy

• Very weakly dominate

33
Domination by a Mixed Strategy

• Weak domination

Maximize
Subject to
34
Iterated Dominance

• Strategy Elimination Does there exist some
  elimination path under which the strategy si is
  eliminated?
• Reduction Identity Given action subsets A-i Ai
  for each player i, does there exist a maximally
  reduced game where each player i has the actions
  A-i?
• Uniqueness Does every elimination path lead to
  the same reduced game?
• Reduction Size Given constants ki for each
  player i, does there exist a maximally reduced
  game where each player i has exactly ki actions?
• Iterated strict dominance problems are all in P.
• Iterated weak or very weak dominance problems
  are NP-complete.

35
Correlated Equilibrium n-player general-sum games
Variables p(a) constants ui(a)
Could find the social-welfare maximizing the
correlated equilibrium by adding an objective
function
Maximize
36
Correlated Equilibrium

• Complexity of P when applied to CE
• Uniqueness
• Pareto Optimal
• Guaranteed payoff
• Subset inclusion
• Subset containment

37
CE to NE Calculation
Intuitively, correlated equilibrium has only a
single randomization over outcomes, whereas in NE
this is constructed as a product of independent
probabilities.
Changing this program so that it finds NE
requires the first constraint to be