Decentralized Cognitive MAC for Opportunistic Spectrum Access in Ad Hoc Networks: A POMDP Framework

1
Decentralized Cognitive MAC for Opportunistic
Spectrum Access in Ad Hoc NetworksA POMDP
Framework

• Qing Zhao, Lang Tong, Ananthram Swami, and Yunzia
  Chen
• IEEE JSAC, April 2007
• Speaker Yu-chun Cheng

2
Outline

• Introduction
• POMDP
• Decision Theoretic approach
• MAC design
• Numerical and Simulation Results
• Conclusion

3
Introduction

• Cognitive Radio MAC Opportunistic Spectrum Access

4
Introduction (cont.)

• OSA network Challenges
• Sensing and access strategies
• SU does not have full knowledge of availability
  of all channels
• Part of spectrum can be sensed at a particular
  time
• Transmitter-receiver synchronization
• Spatially invariant/varying spectrum opportunity
• Central coordinator / dedicated control channel

5
Outline

• Introduction
• POMDP
• Decision Theoretic approach
• MAC design
• Numerical and Simulation Results
• Conclusion

6
POMDP

• Partially Observable Markov Decision Process
• A POMDP models an agent decision process in
  which it is assumed that the system dynamics are
  determined by an MDP, but the agent cannot
  directly observe the underlying state.

7
POMDP

• In OSA network challenge
• Partially sensed spectrum state

L channels are sensed
N channels
?
8
POMDP
Markov Process State (S1,S2,SN)
(1,0,1,0,?)
Partially observation
Sx, x1,2,,N 0(occupied), 1(idle)
9
Network Model in Markov Process

• For example, while N2

10
POMDP Necessary parameters

• Formal Definition (S,A,O,P,R)
• S is a finite set of states,
• A is a finite set of actions,
• P is a finite set of probabilities,
• O is a finite set of observations,
• RA, S?R is the reward function
• Belief vector
• Let b be the vector of state probabilities b(s)
  denotes the probability that the environment is
  in state s.

11
Markovian Dynamic of OSA network
Pi,j transition probability (well known) A1 /
A2 Set of sensed/accessed channel Tj,A1
lt-0,1 Observes availability of each sensed
channel A1 rj, A1, A2 Reward at end of slot
form receiver
12
The slot structure of Cognitive MAC using POMDP
?(t) Belief vector at slot t
13
Outline

• Introduction
• POMDP
• Decision Theoretic approach
• MAC design
• Numerical and Simulation Results
• Conclusion

14
Decision Theoretic approach

• Decision-theoretic approach based on POMDP
• Optimal Channel Sensing and Access Strategy
• Reduced-state Suboptimal Strategy

15
Optimal Channel Sensing and Access Strategy

• Idea find the maximum expected remaining reward
  Vt(?(t)) that can be accrued from slot t and
  current belief vector ?(t)

Maximum remaining reward Vt1(?(t1)) after
taking this action in channel a at next slot(t1)
The immediate reward obtain in slot t given by
Tj,a Ba bandwidth of channel a
16
Optimal Channel Sensing and Access Strategy

• Computationally prohibitive
• ? grows exponentially with the number of N
  channels
• Ex N5
• Markov process states M 25
• numbers of transition probability 25 x 25

Pjj


PjM
Pj1
State j (S1, S2, S3, S4, S5)
17
Reduced-state Suboptimal Strategy

• Independent channels 1, 2, , N
• Let O ?1, ?2, ..., ?N, where ?i is the
  probability that channel i is available at
  begining of slot
• Given state transition probability
• aa probability of busy state to idle state
• ßa probability of idle state to idle state

18
Reduced-state Suboptimal Strategy

• Reward from channel a
• (?a(t)ßa (1-?a(t))aa)Ba

Channel a will be available in slot t
19
Reduced-state Suboptimal Strategy

• The action in slot t is chosen the maximize
  expected immediate reward
• a(t) arg max (?a(t)ßa (1-?a(t))aa)Ba,
• a?1,2,,N
• When a channel is sensed, the probability will
  updated according to the Markov chain.

20
Spectrum Sensing and Access in the Presence of
Sensing Error

• Take sensing errors into consideration
• e False alarm (overlook)
• d Miss detection (misidentification)

21
Spectrum Sensing and Access in the Presence of
Sensing Error

• The selected channel can be given
• a(t) arg max E(UaO)
• a?1,2,,N
• arg max (BaPrSa1, Ta1O
  a?1,2,,N
• arg max (Ba (1- e) ?aßa (1-?a)aa)
• a?1,2,,N

Ua denote the number of bits that can be
successfully delivered if channel a is chosen
22
Outline

• Introduction
• POMDP
• Decision Theoretic approach
• MAC design
• Numerical and Simulation Results
• Conclusion

23
MAC Design

• Consider two network scenarios
• Spatially Invariant Spectrum Opportunity
• The state of channel is the same at transmitter
  and receiver
• Spatially Varying Spectrum Opportunity
• A channel only presents an opportunity to a pair
  of secondary users if it is available at both
  transmitter and receiver

24
Spatially Invariant Spectrum Opportunity

• Main issue transceiver synchronization
• Transmitter and receiver communicate in same
  channel, and hop synchronously
• Initial handshake
• Synchronous hopping

25
Spatially Varying Spectrum Opportunity

• Spectrum opportunity Identification

26
Spatially Varying Spectrum Opportunity

• Hidden and Exposed Terminals

27
Outline

• Introduction
• POMDP
• Decision Theoretic approach
• MAC design
• Numerical and Simulation Results
• Conclusion

28
Numerical and Simulation Results

• Traffic statistics
• Performance of the optimal protocol under
  different spectrum occupancy

29
Numerical and Simulation Results

• Performance of suboptimal greedy approach

Upper plot N3, B1 Transition prob. a0.2,
ß0.8 Lower plot N3 Transition prob. a
0.8, 0.6, 0.4 ß 0.6, 0.4,
0.2 B3/4, 1, 3/2
30
Numerical and Simulation Results

• Spectrum efficiency in the presence of Sensing
  Error

N3 B1 a0.4, ß0.5
31
Numerical and Simulation Results

• Multiple Secondary Users with Random Message
  Arrivals

N10 B1 a0.2, ß0.8 3 Secondary Users
32
Outline

• Introduction
• POMDP
• Decision Theoretic approach
• MAC design
• Numerical and Simulation Results
• Conclusion

33
Conclusion

• Decentralized MAC for ad hoc OSA networks
• MAC operating in POMDP framework
• Proposition
• OSA is the most effective when it is overlayed
  over a primary network with large inter-arrival
  time and message length.

34
Conclusion

• Disadvantages
• POMDP based on Markov Process
• The reduced solution is simple
• Small-scale simulation