Preview of Markov Process

1
Lecture 3
2
Preview of Markov Process

• A sequence of random variables X1 , X2,.,Xn ,..
  such that
• Xi1 is independent of X1,.Xi-1 given Xi
• Pr(Xi1 ai1/ Xi ai, ., X1 a1) Pr(Xi1
  ai1/ Xi ai )
• Discrete time discrete state markov chain
• So a markov chain evolution can be specified by
• Initial states
• Transition probabilities Pr(Xi1 ai1/ Xi ai
  ) pi,i1

3
0.5
1
0
0.5
0.5
0.5
State A communicates with state B if there is a
positive probability path from A to B
A set of states is closed if all states in the
set communicate with each other, and no state in
the set communicates to any state outside the
set, e.g., 0, 1
4
A state has a period d if the process can only
return to this state in intervals which are
multiples of d
More precisely, di is the g.c.d. of k pi ik gt
0, Where pi ik is Pr(Xk i/ X0 i)
d0 1
d0 2
5
Period of any two communicating states are equal
A state is aperiodic if its period is 1
Consider the wireless system
Let the system have random arrivals, i.e., the
number of arrivals for the different sessions is
random
Arrival process for each session is i.i.d
(independent and identically distributed) Number
of arrivals of session i in slot t is
independent of the number of arrivals of session
i or any other session in past or future slots,
and also independent of number of arrivals of any
other session in the same slot
Probability that a packet arrives for session i
in any slot t is qi
6
Under this assumption the queue length process at
the nodes (values of the Biu(t) s for all
sessions i and nodes u) constitute a markov
process
(B1u(t) B1v(t) B1x(t) B2v(t) B2x(t) B2y(t) ) is a
markov chain
This markov chain consists of one closed set, and
some other open states (prove it!)
Periodicity of the closed set is 1 (prove it!)
(under a minor assumption on the values of the
arrival probabilities, qi s)
7
How many states does this markov chain have?
Then the probability distribution of the markov
chain converges to a probability distribution
with finite expectation
This means the system stabilizes!
The function V(x) we choose is ?icixi2
8
Problem Complexity
Most optimization problems can be expressed as
decision problems (yes/no answer)
For example, shortest path problem can be
expressed as, is there a path with weight less
than k in a graph G?
P is the class of decision problems which can be
decided in polynomial time
Language has certain membership (a, b, c
belongs to English)
Language L (G, k) G has spanning tree of
weight at most k
9
A language is in P if its membership can be
tested in polynomial complexity
How about (G, k) G has an independent set of
size at least k?
How about G G has a unique hamiltonian cycle?
10
A language is NP-hard, if it can be solved in
polynomial complexity, then all other languages
in NP can also be solved in polynomial
complexity, i.e., any other language in NP can be
transformed in to this language in polynomial
complexity
Note that a NP-hard language does not necessarily
belong to NP.
Example (G, k) G does not have a path of
weight higher than k.
A language is NP-complete if it is in NP and is
NP-hard
11
NP-complete
NP-hard
NP
P
Is P NP? probably not
12
Technique for proving that a problem is NP-hard
There are already problems proved to be
NP-hard. Show that if you can solve this problem
(problem we are trying to prove NP-hard) in
polynomial complexity, you can solve such a known
problem in polynomial complexity using the
solution of this problem.
We will do such a reduction.
13
Quality of Service Routing

• Quality of Service
• Delay
• Packet loss
• Packet loss
• Transmission error
• Buffer overflow
• Delay consists of
• Queueing delay
• Transmission delay
• Propagation delay
• First two can be related to bandwidth

14

• If we guarantee a certain bandwidth for a session
  on each of its links, its queueing and
  transmission delay can be upper-bounded
• Congestion related packet loss can also be
  upper-bounded for certain guaranteed bandwidth
• Using Shannons channel capacity theorem
  transmission error related packet loss can also
  be upper-bounded

So quality of service requirement translates to
bandwidth requirement in every link
15
Given a network with finite bandwidths on the
links, and a connection with a certain bandwidth
requirement, find a path which gives that
bandwidth to the session in every link, If there
is no such path detect the condition
16
Generalization
There is a reward associated with every link
Reward of a path is the minimum reward on a link
Find a path with required reward
17
Pseudo-code
Delete any link which has bandwidth less than the
required amount
Find any path between the source and the
destination in the remaining link
First step has complexity O(E)
Using depth first search the second step has
complexity O(V E)
Overall complexity O(V E)
Find an efficient algorithm which gives the
path with the highest reward
18
Offline QoS Routing
We have a set of connection requests, each with a
source, destination and a bandwidth requirement
Is it possible to satisfy all the requests? Sum
of the bandwidths used by connections in a link
should not exceed link capacity
NP-complete
Polynomial complexity computable if we are
allowed to split paths for a single connection
If we are not allowed to split paths, then need
to study approximation algorithms
heuristics
19
Widest path heuristic
Initially residual bandwidth of a link is the
entire link bandwidth
Route a connection through the largest reward path
Find the residual bandwidth in each link of the
network Residual bandwidth? Residual bandwidth
bandwidth used by the connection
Route another connection
20
If a connection can not be routed, backtrack.
Re-route the previous connection through the
second best path, and now try routing this
connection.
Limit the number of backtracks
If the number of allowed backtracks
increase, Computation time increases Better
results are expected
21
A more Efficient Heuristic
Start with the most difficult demand Demand is
high All routes for the demand have low bandwidths
Why should we start with the most difficult task?
Choose a good route for the demand
Go to the next most difficult demand
Backtrack a limited number of times, whenever
demand can not be routed.
22
Abstraction and Constraint Satisfaction
techniques for planning bandwidth
allocation Christian Frei (Swiss Federal
Institute of Technology (EPFL)), Boi Faltings
(Swiss Federal Institute of Technology (EPFL)) ,
INFOCOM 2000
A group of nodes constitute a ?-blocking island,
if any node in the set is connected to another in
the set by path of bandwidth at least ?.
23
k
256
a
j
112
96
16
58
g
10
i
22
b
c
20
9
128
76
28
h
f
d
e
118
60
68
24
N7
If two nodes are in a ?-blocking island, then
there is a path of bandwidth ? between them.
25
N7
16
(b, c) most difficult
N6
N5
56
N4
N1
N2
N3
64
k
i
j
e
f
g
h
a
b
c
d
If the lowest common ancestor of two end points
is level ?, then any path for routing the demand
can not have more than level ?1 bandwidth. So
higher the lowest common ancestor, more difficult
is the demand.
However, this does not consider the amount of the
demand
The most difficult demand is the one with the
least difference between the least common
ancestor bandwidth and the demand amount
26
Form the ?-blocking islands for all bandwidth
levels ?
Start with ? highest value of the bandwidth
Choose any vertex, add all vertices connected by
an edge of b.w. ? or higher to this vertex,
further add all vertices similarly connected to
the new vertices and so on. When you can not add
any more, you end up with a ?-blocking
island. Start with a vertex which is not in any
?-blocking island, and repeat the same till all
vertices are exhausted.
Repeat the entire procedure with ? second
highest value of the bandwidth and so on.
27
Choose the most difficult demand first
The best route for the demand is the one that
lies completely in the lowest level blocking
island which consists of the two end points. This
is because connectivity between any two nodes in
a ?-blocking island is at least ?.
Choose the next most difficult demand
Backtrack if necessary
Artificial intelligence based technique.
28
Online Routing of Demands

• Route demands as and when they arrive
• Demands may be blocked
• Previous technique works well when all demands
  are known in advance.
• You can still use this for online routing, but
  you may need to reroute a lot of existing
  connections as future connections come in
• Any good online strategy must use some
  information about future demands
• For example potential source destinations may be
  known

29
Minimum Interference Routing with application to
MPLS traffic engineering Murali Kodialam (Bell
Labs, Lucent Technologies), T. V. Lakshman (Bell
Labs, Lucent Technologies) INFOCOM 2000
There are a few source-destination pairs in the
network.
Demands are multiples of some constants
When a demand arrives, route it through a path
which has a lot of residual bandwidth , and
which Does not overlap a lot with routes between
other source destination pairs
30
Path 1
Source 2
Path 2
Destination 2
The objective is to reduce blockings of potential
future demands as much as possible.
31
Network flows
Given a network and a source and a destination
find the maximum traffic the source can send to
the destination, while not exceeding the link
bandwidths, and using as many routes as it wishes
to.
Max flow 1.5
32
A valid flow is a traffic the source can send to
the destination, while not exceeding the link
bandwidths, and using as many routes as it wishes
to.
Flow ? Max Flow
A cut is a partition of the vertex set into two
sets, A and B such that A and B do not overlap A
contains the source B contains the destination
For blue cut, A source, B destination, a ,
b
For red cut, A source, b, B destination, a

33
Capacity of a cut is the total bandwidth of the
edges who have one end in set A and the other end
in set B (edges which cross the cut)
34
Any flow is upper bounded by the capacity of any
cut
This is because flow is the amount of traffic you
are sending from source to destination, Source is
in one set of the cut Destination is in the other
set of the cut The two sets of the cuts do not
overlap So any flow from the source to
destination must flow across the cut, and
hence can not exceed the cut capacity
Max flow from source to destination ? Min. cut
capacity
Max flow from source to destination Min. cut
capacity
35
This means if we reduce the bandwidths of any
link in any min. cut even slightly, maximum flow
across the source to destination is reduced.
Whereas this does not hold for other edges in the
network. We can always find a small amount such
that reducing the link capacities of the edges
not in any min. cut by that amount, does not hurt
the max. flow
Edges in the min. cuts are critical to
connectivity of the source and the
destination.
Any edge in any min. cut will be denoted as a
critical edge.
36
Incidentally, max flow and all min. cuts are
polynomial complexity computable.
If link bandwidths are integers, then max flow
allocation algorithm assigns integer flows to all
links
So, max flow with the constraint that flow in any
link must be integer is polynomial complexity
computable.
However, given n source destination pairs, and a
flow vector (f1, .fn) (fi is the flow for
session i) , the question whether we can route
this flow vector without violating the capacity
constraints, using only integer flows in the
links, and as many routes as we wish to, is
NP-hard!
37
Interference Measure
Interference caused by a path is measured by
computing the max flow in the residual capacity
graph between all source destination pairs.
Let demand B be routed along path p
Look at the residual capacity graph after routing
the demand along p (subtract bandwidth B from all
edges in p)
Find the maximum flow between each source
destination pair in the residual capacity graph.
Consider the minimum value of these individual
maxflows. This is the interference caused by the
path p
38
Another measure of the interference can be the
lexicographic ordering of these maxflows
Another measure is the weighted sum of these
maxflows.
Choose the path p which maximizes the
interference measure
Higher the interference measure, less is the
interference
The problem is NP-hard heuristics
39
MIRA
While routing a new demand, one should avoid the
critical edges of other source destination pairs
as much as possible.
So when a demand arrives, look at the network
with the residual capacities.
Find the max. flow of each source destination
pair separately (polynomial complexity)
Find the critical edges for every source
destination pair (polynomial complexity)
40
Remove edges with residual capacity lt demand
Give weights to all the critical edges of all the
source destination pairs.
Weights to edges which are not critical for any
source destination will be 0 or very small
positive numbers
Use Dijkstra to find the minimum weight path for
the new demand.
Note that the minimum weight path will have the
requisite bandwidth. Why?
The min. weight path would try to avoid critical
edges as much as possible.
41
Do we gain anything by giving small nonzero
weights to the non-critical edges instead of 0
weights?
Weights for critical edges could be the same or
different for different source destination.
For example, weight could be largest for the most
valued flow. This would avoid using the critical
edges of the most valued flow by other flows as
much as possible, and reduce the blocking for the
most valued flow.
Weight could be the largest for the source
destination pair with the least max. flow in the
residual capacity graph (this has the largest
blocking possibility, so its critical edges
should be preserved the most).
42
Weight could be in lexicographic order of max.
flow values, the one with least value of max flow
gets much larger weight for its critical edges
than others, the one with with second min. value
of max flow gets second largest weight for its
critical edges (much larger than the others),
etc.
How do we compute all the critical edges in the
graph?
43

• Typically, routing decisions are not taken for
  every connection, but at intervals.
• For this case, lexicographic weighing of edges
  performs better.
• Also, we should have non-negligible weights for
  edges which are not critical, but close to
  critical.
• The capacities in these edges allow increasing
  flows, but only by a small amount ?

44
Minimum Interference Routing with application to
MPLS traffic engineering Koushik Kar (University
of Maryland), Murali Kodialam (Bell Labs, Lucent
Technologies), T. V. Lakshman (Bell Labs, Lucent
Technologies) JSAC Nov. 2000