Applications of Game Theory: Part II

1
Applications of Game Theory Part II(b)
John C.S. Lui Computer Science Eng. Dept The
Chinese University of Hong Kong
2
First course
On the interaction between Overlay Routing and
Underlay Routing Y. Liu, H. Zhang, W. Gong, D.
Towsley INFOCOM 2005
3
Motivation Interactions Between Application
Level Network and Physical Network

• physical network control
• routing, congestion control,

• add an overlay

• and another

• Result?
• interactions?
• controllers mismatch?

4
Outline

• Problem Formulation
• Simulation Study
• Game-theoretic Study
• Conclusions

5
Routing in Underlay Network

• Routing on physical network level
• Inter-domain BGP, etc.
• Intra-domain OSPF, MPLS, etc.
• determine routes for all source-destination
  traffic demand pairs
• minimize network-wide delay, cost, etc.

traffic demand pair A-gtB traffic demand pair
A-gtC traffic demand pair C-gtB
6
Routing in Overlay Network

• An overlay network choose routes at application
  level to minimize its own delay or cost

Overlay demand A-gtB logical routes A-gtC-gtB and
A-gtB
C
A
B

• Overlay
• gains advantage
• better path delay, loss, throughput, etc
• is selfish
• potential performance degradation to other
  non-overlay traffic

C
D
E
A
B
demand pair A-gtC
demand pair C-gtB demand pair A-gtB
7
Considering overlay and underlay together ?

• How do they interact with each other?
• How does selfish behavior of overlay routing
• affect overall network performance?
• affect non-overlay traffic performance?
• affect its own performance?

8
Interactions Between Overlay Routing and
Underlay Routing
Overlay Routing Optimizer To minimize overlay
cost
Underlay Routing Optimizer To minimize overall
network cost
9
Approach by authors

• Focusing interaction in a single AS
• Considering two routing models for overlay and
  one routing model for underlay
• Simulating the interaction dynamic process
• Studying this process in a Game-theoretic
  framework

10
Routing Models

• Overlay routing model
• Selfish source routing
• Individual user controls infinitesimal amount of
  traffic, to minimize its own delay
• Optimal overlay routing
• A central entity minimizes the total delay of all
  overlay traffic demands

• Underlay routing model
• Optimal underlay routing
• A central entity minimizes the total delay of all
  network traffic, e.g. Traffic Engineering MPLS

11
Simulation Study Optimal Overlay and Optimal
Underlay
14 node tier-1 POP network (Medina et.al.
2002) bimodal normal model of traffic demand 3
overlay nodes
Node without overlay
Node with overlay
Link
12
Simulation Study ( case 1 8 overlay traffic)
Optimal Overlay and Optimal Underlay

• Iterative process
• Underlay takes turn at step 1, 3, 5,
• Overlay takes turn at step 2, 4, 6,

13
Simulation Study (case 2 10 overlay traffic)
Optimal Overlay and Optimal Underlay

• Iterative process
• Underlay takes turn at step 1, 3, 5,
• Overlay takes turn at step 2, 4, 6,

14
Game-theoretic Study

• Two-player non-zero sum game

Underlay
overlay
X strategy of overlay traffic allocation on
logical links Y strategy of underlay traffic
allocation on physical links
15
Game-theoretic Study

• Best-reply dynamics
• Nash Equilibrium

16
Optimal Underlay Routing v.s. Optimal Overlay
Routing

• Overlay
• One central entity calculates routes for all
  overlay demands, given current underlay routing
• Assumption it knows underlay topology and
  background traffic

X(k)
1-X(k)
Denote overlays routing decision with a single
variable X(k) overlays flow on path ACB after
round k
17
Best-reply Dynamics

• There exists unique Nash equilibrium x,
• x globally stable x(k) ?x, from any initial
  x(1)

When x(1)0, overlay performance improves
18
Best-reply Dynamics

• There exists unique Nash equilibrium x,
• x globally stable x(k) ?x, from any initial
  x(1)

When x(1)0.5, overlay performance degrades
Overlay Delay Evolution
Overlay Routing Evolution
Underlays turn
delay
Overlays turn
x(k)
BAD INTERACTION!
x(k)gtx(k1)gtx
x
x(k)ltx(k1)ltx
Round k
Round k
19
Conclusions Open Issues

• Selfish overlay routing can degrade performance
  of network as a whole
• Interactions between blind optimizations at two
  levels may lead to lose-lose situation
• Future work
• larger topology analysis/experimentation
• overlay routing and inter-domain routing
• interactions between multiple overlays ()
• implications on design overlay routing
• regulation between overlay and underlay ()

20
Second course
On the Interaction of Multiple Overlay
Routings Performance 2005 Joe W.J. Jiang, D.M.
Chiu, John C.S. Lui
21
Questions

• These overlays tend to fully utilize available
  resource.
• So, is there any anarchy?
• How do overlay networks co-exist with each other?
• What is the implication of interactions?
• How to regulate selfish overlay networks via
  mechanism design?
• Can ISPs take advantage of this?

22
Outline

• Motivation
• Mathematical Modeling
• Overlay Routing Game
• Implications of Interaction
• Pricing
• Conclusion

23
Motivation

• Overlays provide a feasibility for users to
  control their own routing.
• Routing, possible multi-path, becomes an
  optimization problem.
• Interaction occurs (due to same underlay)
• Interaction between one overlay and underlay
  traffic engineering, Zhang et al, Infocom05.
• Interaction between co-existing overlays ?

24
Performance Characteristics

• Objective minimize end-to-end delay (e.g., RON)
• Delay of a physical link e
• Performance Characteristics (Underlay)

25
Performance Characteristics

• Objective minimize end-to-end delay
• Delay of a physical link e
• Performance Characteristics (Underlay)

26
Performance Characteristics

• Objective minimize end-to-end delay
• Delay of a physical link e
• Performance Characteristics (Underlay)

27
System Objectives

• Network Operators
• Min average delay in the whole underlay network
• Overlay Users
• Min average delay experienced by the overlay

28
How do Overlays Interact?

• Overlapping physical links.
• Performance dependent on each other.
• Selfish routing optimization.
• Overlays are transparent to each other.
• Lack of information exchange between overlays.

29
Contribution

• What is the form of interaction?
• Is there routing instability (oscillation), or
  there is an equilibrium ?
• Is the routing equilibrium efficient?
• What is the price of anarchy?
• Fairness issues
• Mechanism design can we lead the selfish
  behaviors to an efficient equilibrium?

30
Mathematical Modeling

• Overlay routing An optimization problem

• Decision variable routing policy

s overlay f flow r path
31
Mathematical Modeling

• Overlay routing An optimization problem

• Objective average weighted delay (matrix form)

32
Overlay Routing Optimization

• Convex programming

33
Algorithmic Solution

• Unique optimizer
• Convex programming
• feasible region convex
• delay function continuous, non-decreasing,
  strictly convex
• Solution
• Apply any convex programming techniques.
• Marginal cost network flow (probabilistic routing
  ICNP04).
• This is solved in an independent, and distributed
  fashion by each overlay.

But will independent optimization leads to system
instability (route flop)?
34
Overlay Routing Game
Strategic Game GoverlayltN, (?s), (s)gt

• Nash Routing Game
• Player -- N
• all overlays
• Strategy -- ?s
• feasible routing policy feasible region of
  OVERLAY(s)
• Preference relation -- s
• low delay players utility function is -delay(s)

35
Illustration of Interaction
Aggregate traffic on physical links
Routing decision on logical paths in overlay 1
Delay of logical paths in overlay 1
Overlay 1
Routing decision on logical paths in overlay 2
Delay of logical paths in overlay 2
Overlay 2
Overlay probing
Aggregate overlay traffic

?
Routing Underlay
Underlay (non-overlay) traffic

Overlay n
Routing decision on logical paths in overlay n
Delay of logical paths in overlay n
36
Existence of Nash Equilibrium

• Definition Nash equilibrium point (NE)

A feasible strategy profile y(y(1),, y(s),,
y(n))T is a Nash equilibrium in the overlay
routing game if for every overlay s?N,
delay(s)(y(1),y(s),y(n))
delay(s)(y(1),y(s),y(n))for any other
feasible strategy profile y(s) .
37
Existence of Nash Equilibrium

• Theorem

Good News NO ROUTE FLOP !!!
In the overlay routing game, there exists a Nash
equilibrium if the delay function
delay(s)(y(s) y(-s)) is continuous,
non-decreasing and convex.
38
Fluid Simulation
39
Overlay performance
40
Overlay routing decisions
41
The Price of Anarchy
Global Performance (average delay for all flows)
Efficiency Loss ?

• GOR Global Optimal Routing
• NOR Nash equilibrium for Overlay Routing Game
• NSR Nash equilibrium for Selfish Routing

42
Selfish Routing

• (User) selfish routing a single packets
  selfishness
• Every single packet chooses to route via a
  shortest (delay) path.
• A flow is at Nash equilibrium if no packet can
  improve its delay by changing its route.

43
Selfish Routing

• Also a Nash equilibrium of a mixed strategic game
• Player flow f
• Strategy p ? Pf
• Preference low delay
• System Optimization Problem

44
Performance Comparison
Overlay One Overlay Two Average Delay
Centralized Global Optimal Routing 2.50 2.38 2.44
NE of Overlay Optimal Routing 2.46 2.53 2.50
NE of Selfish Routing 2.63 2.75 2.69
45
Inspiration

• Is the equilibrium point efficient (at least
  Pareto optimal) ?
• Fairness issues of resource competition between
  overlays.

46
Example Network
y1
1-y1
y2
1-y2
47
Sub-Optimality
physical link delay function de(le)
1-5 1l
3-4 l
2-6 2.5l
y1
y2
Non Pareto-optimal !
Routing (y1, y2) Average Delay (overlay1, overlay2 )
NE (0.5, 1.0) (1.5, 1.5)
Pareto Curve (0.4, 0.9) (1.4, 1.4)
48
Fairness Paradox
physical link delay function de(le)
1-5 al
3-4 bl?
2-6 cl
y1
y2

• a, b, c, ? are non-negative parameters
• Everything is symmetric except two private links
  a c

49
Fairness Paradox
physical link delay function de(le)
1-5 al
3-4 bl?
2-6 cl
y1
y2
a lt c
Overlay 1 has a better private link !
50
Fairness Paradox
y1
y2
Unbounded Unfairness
a lt c ? delay1 lt delay2
51
War of Resource Competition
1 unit
1 unit
USA
China
poil(y1y2)
pusa(1-y1)
pchn(1-y2)
pusalt pchn
52
War of Resource Competition
1 unit
1 unit
y1
USA
China
poil(y1y2)
Min Costchn(y2 y1) y2poil(y1y2)(1-y2)pchn(
1-y2)
pusa(1-y1)
pchn(1-y2)
pusalt pchn
53
War of Resource Competition
1 unit
1 unit
USA
poil(y1y2)
China
pusalt pchn ? Costusa gt Costchn
pusa(1-y1)
pchn(1-y2)
54
Pricing (opportunity for ISP)
Mechanism Design
Inefficient Nash equilibrium
Desired equilibrium
payment
new Nash equilibrium
55
Pricing I Improve Delay

• Objective to achieve global optimality

• NE of overlay routing game

• Global optimal

le(s) traffic of overlay s le(-s) traffic
other than overlay s
56
Pricing I Improve Delay

• Objective to achieve global optimality

• New NE of overlay routing game

• Global optimal

Heterogeneous pricing
57
Pricing I Improve Delay

• Global optimal

• New NE of overlay routing game

KKT condition
KKT condition
58
Pricing II improve fairness

• Cause of unfairness
• Over-utilize good common resources
• Unfair resource (bandwidth) allocation
• Pricing Scheme

price p
ISP
Overlay
Improve performance Reduce cost
maximize profit
routing decision
59
Incentive Resource Allocation

• For overlays

new Nash equilibrium ? le
60
Revenue Distribution

• For ISPs (links)

61
Effectiveness of Pricing
62
Conclusion

• Study the interaction between multiple
  co-existing overlays.
• Non-cooperative Nash routing game.
• Prove the existence of NEP.
• Show the anomalies and implications of the NEP.
• Present two distributed pricing schemes to
  address the anomalies.

63
Third Course
Interaction of ISPs Distributed Resource
Allocation and Revenue Maximization Sam C.M.
Lee, Joe W.J. Jiang, D.M Chiu, John C.S. Lui
64
View of ISPs
65
Tier-2 ISP
Local ISP
ISP
Peer
Peer
Peer
Peering link
ISP link
ISP
Peer
Peer
Peer
66
Optimization problem of peers
Issues to consider
Tier-2 ISP (ISP)
Peer i
1. performance of the link
2. charge of the link
Peer k
Peer j
67
Optimization problem of peers
Happiness obtained from sending traffic to peers
Delay cost in ISP link
Payment to ISP
Delay costs in peering links
Payments to peers
68
Constraints of peers
1.
2.
3.
4.
69
Solution to the peers

• Objective function is strictly concave in every
  transmission rate
• The optimal transmission rates and maximum
  utility are unique and can be found by the
  Lagrangian method.

70
Problems for an ISP
71
Information exchange framework
Next period
Bandwidth allocation
ISP
Bid
peer
Compute resource distribution
Compute optimal rates
72
ISP 1 Resource distribution
ISP
Bandwidth 600MBps
?
?
?
peer1
peer2
peer3
Bid 100MBps
Bid 150MBps
Bid 50MBps
73
Proportional share algorithm
ISP
Bandwidth 600MBps
100MBps
200MBps
300MBps
peer1
peer2
peer3
Bid 100MBps
Bid 150MBps
Bid 50MBps
74
Equal share algorithm
ISP
Bandwidth 600MBps
150MBps
200MBps
250MBps
peer1
peer2
peer3
Bid 100MBps
Bid 150MBps
Bid 50MBps
75
Simulations

• When the happiness coefficients of peers are low

PSA
ESA
76
Simulation

• When the happiness coefficients of peers are high

PSA
ESA
77
ISP 2 Maximization of Revenue
Unit price
Demand by peer i
Total revenue from the peers
Determine the optimal price
78
Solution Maximization of revenue

• Estimate the aggregate traffic ( ) from
  all
• peers in term of the price (P)

79
Conclusions

• Utility maximization of a peer
• Resource distribution of ISP
• Revenue maximization of ISP

80
Fourth Course
On the Access Pricing Issues of Wireless Mesh
Networks ICDCS 2006
Ray K. Lam Dah-Ming Chiu John C.S. Lui
81
WMN Paints a Bright Future

• Wireless mesh network (WMN)
• Wireless nodes
• Multi-hop routing
• Form a wireless mesh
• More access to the Internet
• More people, rich or poor
• More ubiquitous, anywhere, anytime
• More opportunities to everyone

Internet
Internet
82
The Critical ThingCooperation

• Multi-hop routing
• Relay packets for each other
• My concerns bandwidth, CPU time, security
• Community network with symmetric traffic
• Help each other gt mutual benefit
• Access network with asymmetric traffic
• Geographically good VS poor
• Why help the poor?
• Incentive system neededpricing

83
When AP Meets a Client

• Simple analysis by Musacchio and Walrand 1
• A game with 2 players
• Access point (AP) provides Internet access
• Client buys the service
• One deal per time slot

AP
Client
p1
accept
slot 1
p2
service duration
accept
slot 2
p3
reject
slot 3
p
AP
Client
84
A Beautiful Equilibrium

• AP and client each maximizesher gain
• AP guess the right price
• Client compare the price p withservice utility
  U
• Web browsing utility function
• A beautiful equilibrium
• AP has the same optimal price in every time slot
• Client connects if her per-slot service utility
  is greater than slot price (U gt p)
• Encourages flat-rate pricing

85
To a Multi-hop Scenario
AP
Client
RS

• Adding a relaying node, or reseller (RS)
• RS tries to mark up APs price to the right
  level
• AP takes note of RSs action when setting her
  price
• Equilibrium is still flat-rate pricing
• Multi-hop gt multiple RSs

c1
p1
p
accept
accept
c
slot 1
service duration
c2
p2
accept
accept
slot 2
AP
Client
RS
c3
p3
reject
reject
slot 3
86
Drawbacks of the Simple Model

• Assuming unlimited network capacity
• 2-player game represents whole system
• Treat every incoming client the same
• Unlimited admission gt unlimited capacity
• Assuming a tree-like network
• 2-hop / multi-hop linear network extension
• Does not consider multiple paths
• Pricing competition may occur

AP
Client
RS
A tree-like network
A graph-like network
87
What If Capacity Limited?

• Cannot admit unlimited clients
• Client demands bandwidth guarantee
• AP admission control
• APs system capacity m
• 2-player game not enough
• AP deals with each client differently
• Client arrival model Poisson process
• Like an M/M/m/m/M queuing system

88
Flat-rate Pricing Fails

• Failure scenario
• AP is full m clients admitted
• An admitted client a is paying 5/slot
• A new client b arrives
• AP asks b for 6/slot
• If b accepts
• AP raises price for a to 6/slot, OR
• Simply kicks a out
• Flat-rate pricing is not optimal!

89
Everybody Loves Flat Rate

• Unrealistic for variable rate
• More practicalfixed-rate, non-interrupted
  service
• AP charges a client a fixed rate p over time
• AP cannot disconnect a client unilaterally
• AP can still charge different clients at
  different fixed rates
• How to set the optimal rate on different
  occasions?

90
Best Strategy in New Service Model

• AP sets price based on remaining capacity
• Raises price when becoming full
• State price at state k, AP charges next
  to-be-admitted client at fixed rate pk
• Policy of AP characterized by price vector
• Clients best strategy
• Connect AP if service utility per unit time gt
  price per unit time (U gt p)

91
System Dynamics

• State transition
• Adding a factor P(U gt pk) to regular arrival rate
  in M/M/m/m/M model
• Reward structure
• Simplification immediate expected profit when a
  client connects

? M P(U gt p0)
? (M-1) P(U gt p1)
? (M-m1) P(U gt pm-1)

0
1
2
m
m-1
?
2?
m?
State transition diagram
92
System Dynamics

• State transition
• Adding a factor P(U gt pk) to regular arrival rate
  in M/M/m/m/M model
• Reward structure
• Simplification immediate expected profit when a
  client connects

p0/?
p1/?
pm-1/?

0
1
2
m
m-1
0
0
0
Reward Structure
93
Finding Optimal Price Vector

• Classical optimization
• Solution for queuing system gives limiting state
  probability for each state k, ?k
• Gain of AP is a function of price vector
• Complicated to optimize with classical techniques
• Policy-iteration method in Markovian decision
  theory
• Reduces computational complexity by iterative
  algorithm
• Guarantees convergence to the best policy

94
Numerical Results

• Capacity m5, population M10, departure rate ?1
• Vary arrival rate ? from 0.2 to 10
• Utility U uniformly distributed on 0,10

• U normally distributed with mean 5, s.d.
  1.67
• Price increases number of clients in AP and with
  ?

95
Limited Capacity in Multi-hop Case

• Simplification
• Traffic merges at AP
• AP is the bottleneck
• Only AP controlsadmission
• APs policy specified by a price matrix
• At each state, different prices for requests from
  different distances
• pki price at state k for a client i-hop away

Internet
bandwidth bottleneck
AP
Client
RS
96
System Dynamics

• Removing finite population
• Complicates state information
• Different arrival rates for clients at different
  distances

?n P(U gt mn(p0,n))
?n P(U gt mn(pm-1,n))
Client n-hop away arrives
Client 2-hop away arrives


Client 1-hop away arrives
?2 P(U gt m2(p0,2))
?2 P(U gt m2(pm-1,2))
?1 P(U gt m1(p0,1))
?1 P(U gt m1(pm-1,1))

0
1
m-1
m
m?
?
State transition diagram
97
System Dynamics

• Removing finite population
• Complicates state information
• Different arrival rates for clients at different
  distances

p0,n/?
pm-1,n/?
Client n-hop away arrives
Client 2-hop away arrives


Client 1-hop away arrives
p0,2/?
pm-1,2/?
p0,1/?
pm-1,1/?

0
1
m-1
m
0
0
Reward Structure
98
Conclusion

• Contributions
• Show that fixed-rate pricing fails with limited
  capacity
• Generalize unlimited capacity model into limited
  capacity model
• Devise optimal pricing for fixed-rate,
  non-interrupted service with Markovian decision
  theory
• References
• 1 J. Musacchio and J. Walrand. WiFi access
  point pricing as a dynamic game. IEEE/ACM Trans.
  Networking. to appear in.