Analyzing the Interactions of Cognitive Radios

1
Analyzing the Interactions of Cognitive Radios

• James Neel
• August 31, 2005

2
Research in a nutshell

• Hypothesis Applying game theory and game models
  (potential and supermodular) to the analysis of
  cognitive radio interactions
• Provides a natural method for modeling cognitive
  radio interactions
• Significantly speeds up and simplifies the
  analysis process (can be performed at the
  undergraduate level Senior EE)
• Permits analysis without well defined decision
  processes (only the goals are needed)
• Can be supplemented with traditional analysis
  techniques
• Can provides valuable insights into how to design
  cognitive radio decision processes
• Has wide applicability
• Focus areas
• Formalizing connection between game theory and
  cognitive radio
• Collecting relevant game model analytic results
• Filling in the gaps in the models
• Model identification (potential games)
• Convergence
• Stability
• Formalizing application methodology
• Developing applications

3
Presentation overview

• A formal analysis model for cognitive radio
  decision rules
• Traditional analysis methods
• Contraction mappings
• Game theory and cognitive radio
• Key game models
• Formal methodology
• Research status and future work

4
Formal Analysis Model
5
Spectrum occupancy

• Perception of bandwidth scarcity, but reality is
  spectrum under-utilization

Adapted from Figure 1 in Published August 15,
2005 M. McHenry in NSF Spectrum Occupancy
Measurements Project Summary, Aug 15,
2005. Available online http//www.sharedspectrum.
com/?sectionnsf_measurements
6
How is spectrum underutilized?

• In time, frequency, space, code,

Figure from F. Jondral, Spectrum Pooling An
Efficient Strategy for Radio Resource Sharing,
Blacksburg, VA June 8, 2004.
7
What is a cognitive radio?
Cognitive radio
Cognition cycle

• An enhancement on the traditional software radio
  concept wherein the radio is aware of its
  environment and its capabilities, is able to
  independently alter its physical layer behavior,
  and is capable of following complex adaptation
  strategies.

Infer from Context
Orient
Infer from Radio Model
Establish Priority
Normal
Pre-process
Select Alternate Goals
Parse Stimuli
Plan
Urgent
Immediate
Learn
Observe
New States
Decide
States
User Driven (Buttons)
Generate Best Waveform
Autonomous
Outside World
Act
Allocate Resources Initiate Processes Negotiate
Protocols
Adapted From Mitola, Cognitive Radio for
Flexible Mobile Multimedia Communications , IEEE
Mobile Multimedia Conference, 1999, pp 3-10.
8
Why cognitive radio?

• Improved spectrum utilization
• Fill in unused spectrum
• Move away from over occupied spectrum
• Improved link performance
• Adapt away from bad channels
• Increase data rate on good channels
• New business propositions
• High speed internet in rural areas
• High data rate application networks (e.g.
  Video-conferencing)
• Significant interest from FCC, DoD
• Possible use in TV band refarming

9
A more realistic depiction

• Outside world is determined by the interaction of
  numerous cognitive radios

Outside World
10
Dynamic cognitive radios in a network

• Many decisions may have to be localized
• Distributed behavior
• Adaptations of one radio can impact adaptations
  of others
• Interactive decisions
• Locally optimal decisions may be globally
  undesirable

11
Locally optimal decisions that lead to globally
undesirable networks

• Scenario Distributed SINR maximizing power
  control in a single cluster
• For each link, it is desirable to increase
  transmit power in response to increased
  interference
• Steady state of network is all nodes transmitting
  at maximum power

Power
SINR
Need way to analyze networks with interactive
decisions
12
Basic analysis model

• N (finite) set of cognitive radios
• Aj adaptations available to cognitive radio
• A space of adaptations
• O space of outcomes (network states)

• Tj set of times when j updates its decision
• f j decision update rule for radio j
• f t network update function at time t

Assumes decision rule is known. Perhaps only the
goals are known
13
Analysis model terminology

• Synchronous system
• Round-robin system
• Random system
• Asynchronous system

14
Analysis objectives

• Establishing Expected Behavior
• Existence
• Identification
• Optimality
• Desirability
• Optimality
• Convergence
• Rate
• Sensitivity to initial conditions
• Stability
• Lyapunov stability
• Attractivity

15
Establishing expected behavior

• Treat existence as a fixed point problem

• Relevant fixed point theorems
• Brouwers
• Kakutanis
• Tarskis
• Glicksberg-Fan
• Zhou
• Banachs
• Identification can be problematic
• NP problem

1
f t(a)
a
1
0
16
Pareto efficiency
Almost Worthless!

• Formal definition An action vector a is Pareto
  efficient if there exists no other action vector
  a, such that every radios valuation of the
  network is at least as good and at least one
  radio assigns a higher valuation
• Informal definition An action tuple is Pareto
  efficient if some radios must be hurt in order to
  improve the payoff of other radios.
• Important note
• Unrelated to the fixed point

17
Being misled by Pareto efficiency

• Scenario Distributed SINR maximizing power
  control in a single cluster.
• Unique fixed point All nodes transmit at maximum
  power.
• Though clearly undesirable, fixed point is Pareto
  efficient.

Power
SINR
Preferable approach demonstrate fixed point
maximizes a design objective function.
18
Stability

• Lyapunov stability
• A fixed point, a, is Lyapunov stable if for
  every ?gt0, there is a dgt0 such that for all t?t0
• Attractivity
• A fixed point, a, is attractive over the region
  S?A if for all a0?S, at converges to a.

19
Relation of stability concepts
Paths for a fixed point that is attractive but
not Lyapunov stable. Reproduced from Figure 1.1
in Hofbauer
Paths for a system that is Lyapunov stable but
not attractive
20
Identifying Lyapunov stable sSystems

• Lyapunovs Direct Method
• Simpler than definition
• Still difficult to find the Lyapunov function
  (Mostly trial and error)

(Pasted in from J. Neel, Potential Games MPRG
Technical Report)
21
Lyapunovs direct method(Discrete Time)
Text from prelim report, Direct Method for
Discrete Time Systems from A. Medio, M. Lines,
Nonlinear Dynamics A Primer, Cambridge
University Press, Cambridge, UK, 2001.
22
Traditional Methods
23
Contraction mappings

• Given a recursion , f is
  said to be a contraction mapping if there is a
  such that

• Approach adopted in D. Bertsekas, J. Tsitsiklis,
  Parallel and Distributed Computation Numerical
  Methods, Athena Scientific, Belmont MA, 1997.

24
Contraction mapping identification

• Derivative conditions (implies Lipschitz
  continuity)
• Blackwells conditions
• 1. Monotonicity Given bounded functions
  
• where
  , f must satisfy
• 2. Discounting There exists a
  such that
• for
  all bounded

25
Theoretical implications

• Unique fixed point exists
• Identify by recursive application of f
• Optimality?
• Converges
• Rate
• Stability
• Lyapunov function
• Similar results for pseudo-contraction

26
General convergence theorem (1/2)

• Suppose
  such that
• Synchronous convergence condition
• If ak is a sequence such that ak?A(k), then
  every limit point of ak is a fixed point of f.
• Box Condition For every k, there exist sets
  Ai(k)?Ai such that

27
General convergence theorem (2/2)

• If the Synchronous and Box Conditions hold, and
  the initial solution estimate a0 belongs to A(0),
  then every limit point of ak is a fixed point
  of f and f converges asynchronously.

28
Standard Interference Function

• Conditions
• Suppose fA?A and f satisfies
• Positivity f(a)gt0
• Monotonicity If a1?a2, then f(a1)?f(a2)
• Scalability For all ?gt1, ?f(a)gtf(? a)
• f is a pseudo-contraction mapping Berggren

R. Yates, A Framework for Uplink Power Control
in Cellular Radio Systems, IEEE JSAC., Vol. 13,
No 7, Sep. 1995, pp. 1341-1347. F. Berggren,
Power Control, Transmission Rate Control and
Scheduling in Cellular Radio Systems, PhD
Dissertation Royal Institute of Technology,
Stockholm, Sweden, May, 2001.
29
Yates power control applications

• Target SINR algorithms
• Fixed assignment - each mobile is assigned to a
  particular base station
• Minimum power assignment - each mobile is
  assigned to the base station in the network where
  its SINR is maximized
• Macro diversity - all base stations in the
  network combine the signals of the mobiles
• Limited diversity - a subset of the base stations
  combine the signals of the mobiles
• Multiple connection reception - the target SINR
  must be maintained at a number of base stations.

30
Shortcomings in traditional techniques

• Fixed point theorems provide little insight into
  convergence or stability
• Lyapunov functions hard to identify
• Contraction mappings rarely encountered
• Doesnt address nondeterministic algorithms
• Genetic algorithms
• Analyze one algorithm at a time little insight
  into related algorithms
• Not very useful for finite action spaces
• No help if all you have is the cognitive radios
  goal and actions

31
Game Theory and Cognitive Radio Interactions
32
Games

• A game is a model (mathematical representation)
  of an interactive decision situation.
• Its purpose is to create a formal framework that
  captures the relevant information in such a way
  that is suitable for analysis.
• Different situations indicate the use of
  different game models.

Normal Form Game Model

• A set of 2 or more players, N
• A set of actions for each player, Ai
• A set of utility functions, ui, that describe
  the players preferences over the outcome space

33
How a normal form game works
Player 1
Player 2
Actions
Actions
Action Space
Decision Rules
Decision Rules
Outcome Space
u1
u2
-1
1
1 WINS!
34
Generalized repeated game
Generalized Repeated Game Model

• A set of 2 or more players, N
• A set of actions for each player, Ai
• A set of utility functions, ui, that describe
  the players preferences over the outcome space
• A set of times for each player when it will
  update its decisions, Tj

Refer to 1-3 as the stage game for the
generalized repeated game.
35
The cognition cycle is a player
Utility Function
Utility function Arguments
Goal
Establish Priority
Immediate
Normal
Urgent
Outcome Space
Decision Rules
\
Action Sets
Negotiate
Adapted From Mitola, Cognitive Radio for
Flexible Mobile Multimedia Communications , IEEE
Mobile Multimedia Conference, 1999, pp 3-10.
36
Cognitive radio network as a game
Radio 1
Radio 2
Actions
Actions
Action Space
Decision Rules
Decision Rules
Informed by Communications Theory
u2
Outcome Space
u1
37
When game theory can be applied

• Level
• 0 SDR
• 1 Goal Driven
• 2 Context Aware
• 3 Radio Aware
• 4 Planning
• 5 Negotiating
• 6 Learns Environment
• 7 Adapts Plans
• 8 Adapts Protocols

Game Theory applies to 1. Adaptive aware radios
2.
Cognitive radios that learn about
their environment
38
How game theory addresses the issues

• Steady-state characterization
• Nash equilibrium existence
• Identification requires side information
• Steady-state optimality
• Pareto optimality (weak concept)
• Convergence
• Learning processes
• Stability
• No general techniques
• Requires side information

39
Nash equilibrium existence

• Frequently shown with the aid of fixed point
  theorems.
• Given
• is nonempty, compact, and
  convex
• ui is continuous in a, and quasi-concave in ai
  (implies BR A?A is upper-semi continuous)
• Then the game has a Nash Equilibrium

40
Nash equilibrium identification

• Time to find all NE can be significant
• Let tu be the time to evaluate a utility
  function.
• Search Time
• Example
• 4 player game, each player has 5 actions.
• NE characterization requires 4x625 2,500 tu
• Desirable to introduce side information.

41
Better response

• Constraint on decision update rule
• if
• and
• Can be many decision update rules that satisfy
  this constraint

• Random Better
• Directional
• Min
• Max

42
Best response (locally optimal)

• Constraint on decision update rule
• Can be many decision update rules that satisfy
  this constraint

• Random Better
• Min
• Max

43
Convergence concepts

• Finite Improvement Path (FIP)
• From any initial starting action vector, every
  sequence of round robin better responses
  converges.
• Weak FIP
• From any initial starting action vector, there
  exists a sequence of round robin better responses
  that converge.
• L-FIP
• From any initial starting action vector, every
  sequence of round robin better responses
  converges within L steps.

44
Better response dynamic

• During each stage game, player(s) choose an
  action that increases their payoff, presuming
  other players actions are fixed.
• Converges if stage game has FIP.

B
A
a
1,-1
0,2
b
-1,1
2,2
45
Best response dynamic

• During each stage game, player(s) choose the
  action that maximizes their payoff, presuming
  other players actions are fixed.
• May converge with weak FIP.

B
A
C
a
-1,1
1,-1
0,2
b
1,-1
-1,1
1,2
c
2,1
2,0
2,2
46
Stability

• Not typically applied.
• Some authors claim NE is stable, e.g.,
  Anshelevich
• However, this obviously doesnt work for elusive
  NE Hicks

E. Anshelevich, A. Dasgupta, J. Kleinberg, E.
Tardos, T. Wexler, T. Roughgarden, The price of
stability for network design with fair cost
allocation, 45th Annual IEEE Symposium on
Foundations of Computer Science, pp. 295 304,
Oct 2004. J. Hicks, A. MacKenzie, A Convergence
Result for Potential Games, 11th Intl Symp. on
Dynamic Games and Applications, Tuscon, Arizona,
Dec. 18-21, 2004.
47
Comments on general game theory approach

• Nice for modeling
• Not particularly insightful for analysis
• NE identification?
• FIP, weakFIP identification?
• Stability?
• Need additional information to make analysis
  efficient (more powerful game models)

48
Game Models for Analysis of Cognitive Radio
Interactions

• Potential Games
• Supermodular Games

49
Potential game model

• Existence of a function (called the potential
  function, V), that reflects the change in utility
  seen by a unilaterally deviating player.

50
Exact potential game identification
51
Better response transformations (new result)

• Better response transformation
• A mapping of ? to ? such that ???
• Example
• Monotonic mapping

52
Potential game fixed point and convergence

• Fixed point
• Existence
• If V is continuous and A is compact, then V has a
  maximum this maximum is a NE
• Identification
• Can be found by solving for maximizers of V.
• Convergence
• Has FIP, L-FIP (GOPG), L-AFIP (generalized
  ?-potential game)
• For continuous V, satisfies conditions for
  Zangwills.
• For ?-better response, no more than
  max(V) min(V)/ ? steps.
• Converges to NE for
• Finite action space all better response
• Infinite action space - Random better, all best
  response

53
Stability

• Previous work
• Almost sure convergence of noisy-best response
  algorithm Hicks
• Lyapunov function for directional better response
  Slade, Anderson
• New result
• Lyapunov function for any better response
  decision algorithm with round-robin or random
  timing (trick is to apply discrete time version
  of Lyapunovs direct method)

M. Slade, "What Does an Oligopoly Maximize?,"
Journal of Industrial Economics, 58, 45-61,
1994. S. Anderson, J. Goeree, and C. Holt,
Stochastic Game Theory Adjustment to
Equilibrium Under Noisy Directional Learning,
Virginia Economics Online Papers, paper no. 327.
54
Optimality

• If V is designed so that its maximizers are
  coincident with your design objective function,
  then NE are also optimal.
• Happens for many desirable cognitive radio
  algorithms
• System interference minimization (and its better
  response equivalents)

55
Supermodular game model

• A game such that
• (1) A is a complete lattice
• (2) ui is supermodular in ai
• (3) ui has increasing differences in (ai, a-i)
• Identification
• A is a Cartesian product of compact subsets of ?

56
Supermodular game best response properties
Lemma 4.2.2 in D. Topkis, Supermodularity and
Complementarity, Princeton University Press,
Princeton, New Jersey, 1998.
57
Fixed point properties

• By Tarskis fixed point theorem, a NE exists.
• Further all NE form a lattice.
• Also works for finite action spaces.

58
Convergence

• Convergence
• Monotonic best response implies convergence to NE
  lattice
• Has weak FIP
• Random sampling better response for finite games
  Friedman (absorbing Markov chain)
• Adaptive dynamics (best response to some past
  weighting of past actions) converges
  (quasi-concave utility functions) Milgrom

59
Stability (new result)

• For finite action spaces, equivalent to a
  Generalized Best Response Potential Game (new
  concept)
• Lyapunov function for finite action space
• Number of best response improvement paths that
  terminate in a
• Have to rely on other techniques for infinite
  action spaces.

60
Potential vs Supermodular
1 For finite action spaces 2 For generalized
?-potential games, converges to a ?-NE
Some games are both potential and supermodular
61
Comments on game models

• Readily applied identification techniques
• Well defined conditions for convergence
• Nice stability conditions
• Possibility for optimality when design objective
  function coincides with system Lyapunov function

62
Analysis Methodology
63
Proposed analysis methodology

• Form generalized repeated game model
• Apply model identification criteria to stage game
  to identify all applicable models
• Use the theoretical implications of the model(s)
  to identify fixed point, establish convergence
  and determine stability
• Determine if network provides sufficient
  performance by evaluating network design
  objective function.

64
Forming the generalized repeated game model

• Players are cognitive radios in network
• Timing taken from network structure
• Actions are each radios available adaptations
• Utility function from cognitive radio goal
• Map utility function to action space expression
  using communications theory

65
Comments on methodology

• Cant always find a model
• However, many algorithms can be used
• When an appropriate model can be found, analysis
  is simple

66
Example Applications

• Ad-hoc Power Control
• Distributed Frequency Selection

67
Ad-hoc power control

• Network description
• Each radio attempts to achieve a target SINR at
  the receiving end of its link.
• System objective is ensuring every radio achieves
  its target SINR

68
Generalized repeated gamestage game

• Players N
• Actions
• Utility function
• Action space formulation

gjk fraction of power transmitted by j that cant
be removed by receiving end of radio js link Nj
noise power at receiving end of radio js link
69
Model identification analysis

• Supermodular game
• Action space is a lattice
• Implications
• NE exists
• Best response converges
• Stable if discrete action space
• Best response is also standard
• Unique NE
• Solvable (see prelim report)
• Stable (pseudo-contraction) for infinite action
  spaces

70
Validation
Implies all radios achieved target SINR
Noiseless Best Response
Noisy Best Response
71
Distributed frequency selection

• Player set N
• Set of independent decision making radio links
• Individual links i, j ? N
• Action sets
• Fi center frequencies available to link i
• F frequency space
• f frequency tuple (vector)
• fi frequency chosen by link i
• Goals
• Minimize inband interference
• Design objective
• Minimize total network interference

72
Specific parameters

• Specific parameters
• Signal bandwidth 1 MHz
• Channel bandwidth 10 Mhz
• 10 (decision making) links
• Frequency discretized with center frequencies
  every 0.1 MHz
• Random initial frequencies

• Game is an exact potential game
• Potential function implies existence of a
  steady-state, convergence, stability

73
Simulation (round-robin) results
Steady-state All frequencies spaced by at least 1
MHz
All links achieve maximum utility (not
generalizable)
74
Simulation (random) results
75
Simulation (random) results
76
Research Status
77
Theoretical Results

• New Concepts
• Multilateral Symmetric Interaction Games
• Generalized ?-potential games
• L-(A)FIP
• Separable action space potential games
• Convergence
• ?-better response for generalized ?-potential
  games
• Stability (Lyapunov functions)
• Discrete time (random, round-robin) better
  response decision update for potential games
• Finite supermodular games with best response
  decision update
• Standard Interference Function
• Equivalence of Lyapunov function for a system and
  best response potential function
• Other
• Relationship between better response contraction
  mappings and FIP
• Better response transformations
• All OPG have a better response transformation to
  a EPG (aids OPG identification)
• Preserves NE
• Preserves FIP and OPF

78
Applications

• Ad-hoc power control (target SINR and its
  better-response equivalents)
• Supermodular game
• Potential game
• Waveform adaptation (Interference minimization)
• Abstract power controlled interference
  minimization (BSI)
• Distributed frequency selection (EPG)
• Single cell OFDM capacity maximization channel
  allocation
• Joint power waveform adaptation
• Multi-cell/cluster target SINR Power SINR
  maximization waveform is standard
• Multi-cell/cluster target SINR Power
  Interference minimization is an OPG when action
  space is separable
• Network formation
• Finite sensor network (costly links, no hop
  penalty, single sink) best response potential
  game
• Qualitative estimates of node complexity from
  game model.

79
Publications (10)

• (Book) to appear J. Neel, J. Reed, A.
  MacKenzie, Analyzing Cognitive Radio Networks
  in Cognitive Radio, ed. B. Fette, Elsevier
  Publications, 2005.
• (Other) J. Neel, Tip of the week Use game
  theory to analyze cognitive radio EE Times,
  August 29, 2005.
• (JNL) accepted V. Srivastava, J. Neel, A.
  MacKenzie, J. Hicks, L.A. DaSilva, J.H. Reed and
  R. Gilles, Using Game Theory to Analyze Wireless
  Ad Hoc Networks, submitted to IEEE
  Communications Surveys and Tutorials. 
• (CNF) J. Neel, R. Menon, A. MacKenzie, J. Reed,
  "Using Game Theory to Aid the Design of Physical
  Layer Cognitive Radio Algorithms," accepted on
  basis of abstract to Conference on Economics,
  Technology and Policy of Unlicensed Spectrum, May
  16-17 2005, Lansing, Michigan.
• (CNF) J. Hicks, A. MacKenzie, J. Neel, J. Reed,
  "A Game Theory Perspective on Interference
  Avoidance," Globecom 2004, November 29 - December
  3, 2004.
• (CNF) J. Neel, J. Reed, R. Gilles, Game Models
  for Cognitive Radio Analysis, SDR Forum 2004
  Technical Conference, November 2004.
• (CNF) J. Neel, J. Reed, and R. Gilles,
  Convergence of Cognitive Radio Networks,
  WCNC2004, March 25, 2004.
• (CNF) S. Ginde, R. Buehrer, and J. Neel, A Game
  Theoretic Analysis of the GPRS Adaptive
  Modulation Schemes, Fall VTC 2003.
• (CNF) J. Neel, J. Reed, R. Gilles, The Role of
  Game Theory in the Analysis of Software Radio
  Networks, SDR Forum Technical Conference
  November, 2002. (Named outstanding paper)
• (CNF) J. Neel, R. Buehrer, J. Reed, and R.
  Gilles, Game Theoretic Analysis of a Network of
  Cognitive Radios, Midwest Symposium on Circuits
  and Systems 2002.

80
Planned Submissions (5)

• (JNL) Potential Games in Wireless Networks
• This paper will summarize the various insights
  that can be gained by applying potential games to
  cognitive radio networks and present some of the
  applications included in this report.
• (JNL) Analyzing Dynamic Frequency Selection
• This paper will provide an extended discussion
  and analysis of a number of dynamic frequency
  selection algorithms.
• (JNL) Ad-hoc Power Control
• This paper will present the analytic power
  control results mentioned in the preceding and
  will also include analysis showing ?-better
  response convergence.
• (JNL) Cognitive Network Selection
• This paper will present an application where the
  cognitive radios choose between several
  coexisting networks.
• (JNL) Distributed Sensor Network Formation
• This paper will present the analytic sensor
  network formation results presented on xxx.

81
Additional Future Work

• Refine example applications
• Build simulation of joint power/waveform
  adaptation
• Explore synchronous potential games

82
Questions?
83
Additional Slides
84
Additional slides(Hyperlinks to files)

• April 2 talk
• Better Response Transformations
• Connections Model
• Convergence
• Convergence2
• Convergence Applications
• Distributed Power Control
• Fixed Point Theorems
• Infinite Better Response
• IREAN_04

• Joint Adaptation
• Network Formation
• Network Formation2
• Noisy Convergence
• Noisy Equilibria
• OFDM
• Power Control
• Qualifier
• SBC
• SDR Forum 02
• SDR Forum 04
• WCNC

85
Simulation Scenario

• Two cluster ad-hoc network
• 11 nodes
• DS-SS N 63
• Path loss exponent n 4
• Power levels -120, 20 dBm
• Step size 0.1 dBm
• Synchronous updating
• Target SINR ? 8.4 dB
• Objective Function

86
Simulation Results
Noiseless Simulation
Noisy Simulation
Identical values for ui Implies fairness
Statistically Identical values for ui Implies
fairness
Cluster heads
Cluster heads
Steady state Steady state exists Attractor and
steady state the same point Attractor is
stable Steady state is fair
Convergence Both scenarios converge Noise has
little impact on convergence rate Implies that
outside of region immediately around NE, PE ltlt PC
87
Power Control Thoughts

• Game has a unique NE
• Distributed ad-hoc power control algorithms
  converge asynchronously if nodes adapt in the
  right direction (making the game a potential
  game)
• Target SINR
• Target BER,FER
• Target Throughput
• Target QoS
• Convergence rate only weakly influenced by noise
  implies convergence rate can be estimated from
  noiseless analysis