Applications of Convergence Theorems

1
Applications of Convergence Theorems

• James Neel
• June 17, 2004

2
Presentation Overview

• Why we care about convergence
• What convergence means in a game
• Applications to some familiar concepts

3
Key Issues in CRN

• Steady State Existence
• Steady State Optimality
• Convergence
• Stability
• Scalability

Convergence How do initial conditions impact
the system steady state? How long does it
take to reach the steady state?
Stability How does system variations impact
the system? Do the steady states change?
Is convergence affected
Steady State Existence Is it possible to
predict behavior in the system? How many
different outcomes are possible?
Optimality Are these outcomes desirable?
Do these outcomes maximize the system target
parameters?
Scalability As the number of devices
increases, How is the system impacted?
Do previously optimal steady states remain
optimal?
4
How Issues Fit Into the Project
Fixed Portion April 03-Oct 04
Stochastic Portion Oct 04-April 06

• Steady State Existence
• Steady State Optimality
• Convergence
• s
• Scalability

• Steady State Existence
• Steady State Optimality
• Convergence
• Stability
• Scalability

5
Convergent Sequence

• A sequence in a metric space X with point
  such that for every , there is an
  integer N such that implies
• This can be equivalently written as
• or

6
Convergent Sequence in a Game

• A sequence in an action space A with
  action tuple such that for every
  , there is an integer N such that
  implies
• This can be equivalently written as
• or
• Note a will generally be a NE for most decision
  update algorithms.

7
Generating Action Sequences
Valuate
Decide
Decision Update
Observe
Metrics
a0
a2
a3
a1
Outside World
Act
Alter Waveform
8
Decision Timing Scenarios

• Simultaneous
• All nodes update at the same time
• Might happen in a cluster or synchronized network
• Round Robin
• All nodes take turns updating at regular
  intervals
• Might happen in a cluster or synchronized network
• Asynchronous
• Nodes update at random times, without any
  consideration to order
• Possibly multiple nodes at a time
• Most likely to be encountered in a general MANET

9
Ultimate Goal in Studying Convergence

• For the game (network) being studied, determine
  which decision update algorithms under what
  conditions will lead to a Nash Equilibrium (as
  well as which NE), and which algorithms will not.

10
Generating Action Sequences in Game Theory

• Preplay Communication
• Before the game, discuss their options. Note
  only NE are suitable candidates for coordination
  as one player could profitably violate any
  agreement.
• Rational Introspection (Best Response)
• Based on what each player knows about the other
  players, reason what the other players would do
  in its own best interest. Points where everyone
  would be playing correctly are the NE.
• Focal Point
• Some distinguishing characteristic of the tuple
  causes it to stand out. The NE stands out
  because its every players best response.
• Trial and Error (Better Response with Errors)
• Starting on some tuple which is not a NE a player
  discovers that deviating improves its payoff.
  This continues until no player can improve by
  deviating. Only guaranteed to work for Potential
  Games (later).

11
Problems with Traditional Game Theoretic
Approaches

• Preplay Communication
• Implies ability to coordinate behavior. This goes
  against a fundamental assumption of our work.
• Rational Introspection (Best Response)
• Implies higher order rationality a capability
  beyond what we are supposing for many devices
• Focal Point
• Implies an ability to search entire outcome
  space.
• Trial and Error (Better Response with Errors)
• The concepts exist in game theory for
  establishing convergence under this condition
  find limited application FIP, weak FIP

12
Path
Definition path (Voorneveld)
A path in A is a sequence ? (a0, a1,) such
that for every k ? 1 there exists a unique player
such that the strategy combinations (ak-1, ak)
differs in exactly one coordinate.
Equivalent formulation A path is a sequence of
unilateral deviations.
a0 is the initial point of ?. If ? is finite,
then the final point is the terminal point.
Definition cycle (closed) (Voorneveld)
A finite path ? (a0, a1,,ak) where ak a0
? is simple and closed if the only repeated
elements are the initial point and the terminal
point. Its length is the number of unique
elements in ? .
13
Improvement Path
Definition Improvement path (Shapley)
A path ? (a0, a1,) where for all k?1
where i is the unique deviator at k
Improvement Paths
?1 (( a1, a2), (a1, b2 ))
?2 (( a1, a2), (b1, a2 ))
?3 (( b1, a2), (b1, b2 ))
?4 (( a1, b2), (b1, b2 ))
?5 (?1, (b1, b2 ))
?6 (?1, (b1, b2 ))
14
Finite Improvement Path Property

• All Improvement paths in a finite game are finite
• No improvement cycles exist
• Exact potential games, ordinal potential games
  have FIP
• Better response, best response converge

15
Better response dynamic (round robin)

• During each stage game, player(s) choose an
  action that increases their payoff, presuming
  other players actions are fixed

B
A
a
1,-1
0,2
b
-1,1
2,2
16
? - Improvement Path
Consider a game with continuous, bounded action
sets and ? gt0. A path is an ?-improvement path if
for all k ? 1
Approximate Finite Improvement Path Property
(AFIP)
If for every ? gt0, every ?-improvement path is
finite.
Note similarity to Cauchy Sequence convergence
17
Weak Finite Improvement Path Property

• From all a there exists a finite improvement path
  that leads to a.
• There may be improvement cycles, but there is
  always some way of escaping.

18
Best response dynamic (round robin)

• During each stage game, player(s) choose the
  action that maximizes their payoff, presuming
  other players actions are fixed

B
A
C
a
-1,1
1,-1
0,2
b
1,-1
-1,1
1,2
c
2,1
2,0
2,2
19
Shortcoming in Game Models

• Convergence rate is not addressed
• Not always applicable
• Asynchronous update schemes rarely considered

20
Establishing Weak FIP

• If from every a?A, there is an improvement path
  that leads to a, then the game has the weak FIP.
• Perhaps easiest to show if best response
  (improvement) always leads to a. Note may be
  dependent on synchronous or round robin play.

21
Contraction Mapping

• Let be a metric space,
  is a contraction if there is a
  such that

Consider a 2-D metric space with the usual
Euclidian metric and several iterations of a
contraction mapping. Technically, this example is
not a contraction. Why?
22
Establishing Weak FIP

• Neel04 If best response function is a
  contraction with modulus ?, then game has a
  unique Nash Equilbrium and the weak FIP and
  converges at rate ?t.
• Proof
• NE is a direct result of Banachs.
• Rate is a direct result of Topkis
• If BA?A converges to a everywhere, then for
  every a there is an improvement path that leads
  towards a (thus weak FIP)

23
Pseudo-contraction

• Let be a metric space and
  with fixed point . f is a pseudo-contraction
  if there is such that

24
Establishing Weak FIP

• Neel04 If best response function is a
  pseudo-contraction with modulus ?, then game has
  a unique Nash Equilbrium and the weak FIP and
  converges at rate ?t.
• Proof
• NE, rate from Topkis
• Again pseudo-contraction means convergence, thus
  weak FIP.

25
Cartesian Product Contractions

• Define X as where
  and are given xii for
  each Xi.
• Define xi max xii (over i). This is the
  block-maximum norm.
• f X?X is a block contraction if f is a
  contraction on the block maximum norm.
• f X?X is a block pseudo-contraction if f is a
  pseudo-contraction on the block maximum norm.

26
Establishing Weak FIP

• Neel04 If best response function is a block
  contraction with modulus ?, then game has a
  unique Nash Equilbrium and the weak FIP and
  converges at rate ?t.
• Proof
• NE, rate from Topkis
• Again block contraction means convergence, thus
  weak FIP.

27
Establishing FIP

• OPG is characterized by lack of improvement
  cycles Voorneveld
• OPG ? FIP
• FIP if all improvement paths lead to a.
• Equivalent to showing convergence of all better
  response algorithms

28
Set Diameter

• Set Diameter
• Let X be a metric space and . The
  diameter of E is given by
• Note that if , is
  Cauchy iff

29
Establishing FIP

• Define
• Neel04 Suppose lim k?? diamBRk(a) ?0 for all
  a?A, then game has FIP, and game is OPG.
• Proof
• Implies all better response algorithms converge
• Thus FIP

(my sloppy notation)
30
Sequence of compact subsets

• Theorem 3.10 (b) Rudin
• If Kn is a sequence of compact sets in X such
  that and if
  then
• consists of exactly one point.
• Note that this can be used to establish both the
  existence and uniqueness of a fixed point if
  is generated by iterative evaluation of a
  function of the form

.
31
Establishing FIP

• Neel04 Suppose for every a there is some N such
  that then all
  better response algorithms converge, and the game
  has FIP (and thus is an OPG).
• Neel04 Suppose for every a there is some N such
  that then
  all better response algorithms converge, and the
  game has FIP (and thus is an OPG).