Game Dynamics Out of Sync

1
Game DynamicsOut of Sync

• Michael Schapira
• (Yale University and UC Berkeley)
• Joint work withAaron D. Jaggardand Rebecca N.
  Wright

2
Motivation Internet Routing

• Establish routes between Autonomous Systems
  (ASes).
• Currently handled by the Border Gateway
  Protocol (BGP).

3
Internet Routing as a GameLevin-S-Zohar

• Internet routing is a game!
• players ASes
• players types preferences over routes
• strategies outgoing edges
• BGP Best-Response Dynamics
• each AS constantly selects its best available
  route to each destination
• until a stable state ( PNE) is reached.

4
But
this talk

• Challenge I No synchronization ofplayers
  actions
• players can best-reply simultaneously.
• players can best-reply based on outdated
  information.
• Challenge II Are players incentivized to follow
  best-response dynamics?
• Can a player benefit from not best-replying?

Nisan-S-Valiant-Zohar
5
Game Dynamics and Asynchrony

• Dynamic environments
• Internet protocols
• large-scale markets
• social networks
• multi-processor computer architectures
• Game theory provides useful tools to analyze
  these interactions, but.
• has so far primarily concentrated on
  synchronous environments (simultaneous,
  sequential).

6
Illustration
Column Player


0,0
2,1
Row Player
0,0
1,2
7
Illustration
Column Player


0,0
2,1
Row Player
0,0
1,2
8
But
Column Player


0,0
2,1
Row Player
0,0
1,2
9
Agenda

• Model for asynchronous game dynamics
• Impossibility result
• Circumventing our impossibility result
• Complexity of asynchronous game dynamics
• Directions for future research

10
Simple Model Nodes Interacting

• n nodes 1,,n
• Node i has action space Ai
• AA1An
• A-iA1Ai-1Ai1An
• Node i has reaction function fiA?Ai
• f(f1,,fn)

11
Simple Model Dynamics

• Infinite sequence of discrete time steps t1,
• Initial state a0, Schedule s1, ?2n
• fair schedule
• The (a0,s)-dynamics
• Start at the initial state a0
• In each time step t let the nodes in s(t) react.

12
Simple Model Convergence

• Defn an action profile a(a1,,an) is a stable
  state if fi(a)ai for all i.
• that is, a is a fixed point of f.
• Defn The system is convergent if the
  (a0,s)-dynamics converges to a stable state for
  all choices of a0 and (fair) s.

13
Guaranteed Convergence?

• Defn f is node independent if, for each node i,
  fiA-i?Ai
• Thm If f is node independent, and there exist
  multiple stable states, then the system is not
  convergent.
• Can be generalized to reaction functions that are
• randomized
• bounded-recall
• non-stationary

14
Applications

• Internet protocols
• Internet routing Sami-S-Zohar
• congestion control Godfrey-S-Zohar-Shenker
• Best-response dynamics
• with consistent tie-breaking
• orthogonal to the results of Hart and Mas-Colell
• Diffusion of technologies in social networks
• 2 technologies A,B. Each node wants to be
  consistent with the majority of its neighbours.
• Circuit design

15
Tightness of Our Result

• Example 1 (node-dependent reactions)Each fi is
  such that for every a(a1,,an) it holds that
  fi(a)ai.

16
Tightness of Our Result

• Example 1 (node dependent reactions)Each fi is
  such that for every a(a1,,an) it holds that
  fi(a)ai.
• Example 2 (unbounded recall)
• 2 nodes, 1 and 2, each with action space a,b.
• Node 2 wants to match node 1s action.
• Node 1 selects b if node 2 changed its action
  from a to b in the past, and a otherwise.
• What happens at the initial state (b,a)?

17
Proving Our Result

• Thm If f is node independent, andthere exist
  multiple stable states, thenthe system is not
  convergent.
• Interesting connections to fundamental results in
  distributed computing theory.
• the Fischer-Lynch-Patterson impossibility result
  for consensus protocols (1983)
• But, neither result is a special case of the
  other.

18
The Dynamics Graph

State R

1. aRaS except aRifi(bS)
2. bRbS

i-transition
State S
knowledge transition
State T
action vector aS(aS1, aSn)knowledge vector
bS(bS1, bSn)

1. aTaS
2. bTaS

19
Visualising Dynamics

• The dynamics graph captures all dynamics.
• The scenario where
• the initial state is a0.
• nodes 1 and 3 react simultaneously.
• then nodes 2 and 3 react simultaneously.
• is captured as follows

20
Visualising Dynamics

• The dynamics graph captures all dynamics.
• The scenario where
• the initial state is a0.
• nodes 1 and 3 react simultaneously.
• then nodes 2 and 3 react simultaneously.
• is captured as follows

State SaSbSa0
21
Visualising Dynamics

• The dynamics graph captures all dynamics.
• The scenario where
• the initial state is a0.
• nodes 1 and 3 react simultaneously.
• then nodes 2 and 3 react simultaneously.
• is captured as follows

State SaSbSa0
1-transition
3-transition
k-transition
22
Visualising Dynamics

• The dynamics graph captures all dynamics.
• The scenario where
• the initial state is a0.
• nodes 1 and 3 react simultaneously.
• then nodes 2 and 3 react simultaneously.
• is captured as follows

State SaSbSa0
1-transition
3-transition
k-transition
2-transition
3-transition
k-transition
23
Stability and Fairness

• Defn A state S in the dynamics graph is stable
  if every outgoing edge from S leads to S.
• Defn A fair path in the dynamics graph is a path
  that (1) for each i, contains an i-transition
  and (2) also contains a knowledge transition.

24
Attractor Regions

• Defn The attractor region of a stable state S
  are all states from which any (long enough) fair
  path reaches S.

25
Proof Sketch (Cont.)

• Claim A fair cycle in the dynamics graph implies
  an oscillation in our model.
• Proposition If there are multiple stable states
  then there are states in the dynamics graph that
  are not in any attractor region (neutral
  states).

26
Colouring States

• Colour each attractor region in a different
  colour red, blue, etc.
• Colour the neutral states in purple.

27
Creating Oscillations

• Key idea We show that from every purple state
  there is a fair path that leads to another purple
  state.
• The number of purple states is finite and so this
  implies a fair cycle.

28
Proof Sketch (Cont.)

• Lemma There cannot be two edges leading from a
  purple state to two non-purple states that do not
  have the same colour.
• Intuition We can swap the order of activations
  without affecting the outcome.

a
b
a,b different transitions
a
b
?
29
Proof Sketch (Cont.)

• Fix a purple state p.
• Let R be a maximal fair path from p to another
  purple state.

R
q


p
30
Proof Sketch (Cont.)

• Let a be a transition that is not on R.
• Observe that a at q takes us to a non-purple
  state.

R
q


a
p
31
Proof Sketch (Cont.)

• Because q is purple it must have a fair path to a
  non-purple non-red state.

R
u


q


a
p
32
Proof Sketch (Cont.)

• Now, we prove that a at u must take us to a red
  state --- a contradiction!

R
u

a

q


a
p
33
Circumventing Our Impossibility Result Randomness

• Our result holds for randomized reaction
  functions.
• adversarially-chosen schedule
• What if the schedule is randomized?
• our impossibility result breaks
• but no general possibility result either

34
Circumventing Our Impossibility Result r-Fair
Schedules

• Defn A schedule s is r-fair if each node is
  activated at least once within every r
  consecutive time steps.
• Can we prove our impossibility result for
  schedules that are r-fair? If so, for what values
  of r?
• We present positive and negative results.

35
Complexity Results

• Thm Determining if a system with n nodes, each
  with two actions, is convergent requires
  exponential communication (in n).
• The proof requires reaction functions to be of
  exponential size.
• Combinatorial proof a Snake in the Box
  construction

36
Complexity Results

• What if the reaction functions can be succinctly
  described?
• Thm Determining if a system with n nodes is
  convergent is PSPACE-Complete.
• Hence, there is no short characterization of
  asynchronous convergence!

37
Directions for Future Research

• Other notions of asynchrony
• Other reaction functions
• fictitious play, regret minimization
• Observation regret minimization is much more
  resilient to asynchrony (different framework).
• Other restrictions on schedules
• random schedules
• r-fair schedules
• more

38
Thank You