Part 3: Design Principles

1
Part 3 Design Principles

• design principles
• separation of data, control
• hard state versus soft state
• randomization
• indirection
• multiplexing
• network virtualization overlays
• design for scale

• Goals
• identify, study common architectural components,
  protocol mechanisms
• what approaches do we find in network
  architectures?
• synthesis big picture

2
1 Separation of control and data

• PSTN (public switched telephone network)
• SS7 (packets-switched control network) separate
  from (circuit-switched) call trunk lines
• earlier tone-based (in-band signaling)
• ATM
• Q.2931 messages carried on VCI5 VPI-0
• signaling ATM Adaptation Layer (SAAL) resides
  between ATM layer and Q.2931 function reliable
  Q.2931 msgs between ATM switch, host
• Internet
• RSVP (signaling) separate from routing,
  forwarding.
• http in-band signaling ftp out-of-band
  signaling

3
Internet HTTP - inband signaling

• Suppose user enters URL www.someSchool.edu/someDep
  artment/home.index

• 1a. HTTP client initiates TCP connection to HTTP
  server (process) at www.someSchool.edu on port 80

1b. HTTP server at host www.someSchool.edu
waiting for TCP connection at port 80. accepts
connection, notifying client
2. HTTP client sends HTTP request message
(containing URL) into TCP connection socket.
Message indicates that client wants object
someDepartment/home.index
3. HTTP server receives request message, forms
response message containing requested object, and
sends message into its socket
4
Nonpersistent HTTP (cont.)
4. HTTP server closes TCP connection.

• 5. HTTP client receives response message
  containing html file, displays html. Parsing
  html file, finds 10 referenced jpeg objects

time
6. Steps 1-5 repeated for each of 10 jpeg objects
5
HTTP request message
request line (GET, POST, HEAD commands)
GET /somedir/page.html HTTP/1.0 Host
www.someschool.edu User-agent
Mozilla/4.0 Connection close Accept-languagefr
(extra carriage return, line feed)
header lines
Carriage return, line feed indicates end of
message
Note request msg just a signaling msg (no data)
6
HTTP response message
status line (protocol status code status phrase)
HTTP/1.0 200 OK Connection close Date Thu, 06
Aug 1998 120015 GMT Server Apache/1.3.0
(Unix) Last-Modified Mon, 22 Jun 1998 ...
Content-Length 6821 Content-Type text/html
data data data data data ...
header lines
data, e.g., requested HTML file

• Note response msg mixes signaling and data
• request, response msgs exchanged over single TCP
  connection

7
FTP separate control, data connections

• FTP client contacts FTP server at port 21
• client obtains authorization over control
  connection
• client browses remote directory via commands sent
  over control connection.
• when server receives file transfer command server
  opens new TCP data connection to client
• after transferring one file, server closes
  connection.

FTP client
FTP server

• server opens 2nd TCP data connection to transfer
  another file.
• control connection out of band signaling
• FTP server maintains state current directory,
  earlier authentication

8
Separate control, data why (or why not)?

• Why?
• allows concurrent control data
• allows perform authentication at control level
• simplifies processing of data/control streams-
  higher thruput
• provide Qos appropriate for control/data streams
• Why not?
• separate channels complicate management,
  increases resource requirements
• can increase latency, e.g., http two tcp
  connections vs one.

9
Primer Markov chains
10
Markov chains

• system occupies state X(t) at time t
• X(t) takes values from finite state space S
• transitions between states

11
Example stop and wait
12
Examples finite buffer systems

• example transmission link with one buffer
• example transmission link with two buffers

13
Markov chains

• transition rate ?i,j - rate at which system
  transits from i to j given system in i, i?j
• P(X(t ?t) j X(t) i ) ?i,j ?t o(?t)
• P(X(t ?t) i X(t) i ) 1-?j?i?i,j ?t
  o(?t)

14
Q what is P(X(t) j), j?S, t gt 0

• differential equations describing behavior
• dP(X(t) j)/dt ?j?i?i,j P(X(t) i)
• - ?j?i?i,j P(X(t) j) , j ?S
• given initial distribution, P(X(0) j), j ?S

15
Sketch of derivation
16
Steady state behavior

• let ?i(t) P(X(t) i), i?S
• expect system to reach steady state where
  behavior independent of initial conditions, ?i
  limt?? ?i(t)
• satisfy ?j?i?i,j ?i ?j?i?j,i ?j, i ?S

17
Example finite buffer link
18
Steady state behavior

• let ?i(t) P(X(t) i), i?S
• expect system to reach steady state where
  behavior independent of initial conditions, ?i
  limt?? ?i(t)
• satisfy
• ?j?i?i,j ?i ?j?i?j,i ?j, i ?S
• along with
• ?i ?i 1

19
Existence of steady state

• finite state MC
• paths between all pairs of states gt steady state
  solutions exist
• otherwise, MC is transient
• infinite state MC more complicated
• ergodicity

20
Properties

• transition rate out of state i
• ?i ?j?i ?i,j
• Yi time spent in state i
• Yi exponentially distributed with rate ?i
• P(Yi lt y) 1-exp(- ?i y), y gt 0.
• EYi 1/?i

21
Solving for steady state probabilities

• infinitesimal generator Q
• steady state probability distribution
• p satisfies

22
Solving for steady state probabilities

• define A
• p satisfies

23
2 Maintaining network state

• state information stored in network nodes by
  network protocols

• updated when network conditions change
• stored in multiple nodes
• often associated with end-system generated call
  or session
• examples
• RSVP routers maintain lists of upstream sender
  IDs, downstream receiver reservations
• ATM switches maintain lists of VCs bandwidth
  allocations, VCI/VPI input-output mappings
• TCP Sequence numbers, timer values, RTT
  estimates

24
State senders, receivers

• sender network node that (re)generates signaling
  (control) msgs to install, keep-alive, remove
  state from other other nodes
• receiver node that creates, maintains, removes
  state based on signaling msgs received from sender

25
Hard-state

• state installed by receiver on receipt of setup
  msg from sender
• state removed by receiver on receipt of teardown
  msg from sender
• default assumption state valid unless told
  otherwise
• in practice failsafe-mechanisms (to remove
  orphaned state) in case of sender failure e.g.,
  receiver-to-sender heartbeat is this state
  still valid ?
• examples
• Q.2931 (ATM Signaling)
• ST-II (Internet hard-state signaling)
• TCP

26
Hard-state signaling
Sender
Receiver
Signaling plane
Communication plane

• reliable signaling
• state removal by request
• requires additional error handling
• e.g., sender failure

27
Soft-state

• state installed by receiver on receipt of setup
  (trigger) msg from sender (typically, an
  endpoint)
• sender also sends periodic refresh msg
  indicating receiver should continue to maintain
  state
• state removed by receiver via timeout, in absence
  of refresh msg from sender
• default assumption state becomes invalid unless
  refreshed
• in practice explicit state removal (teardown)
  msgs also used
• examples
• RSVP, RTP, IGMP

28
Soft-state signaling
Sender
Receiver
Signaling plane
Communication plane

• best effort signaling

29
Soft-state signaling
Sender
Receiver
Signaling plane
Communication plane

• best effort signaling
• refresh timer, periodic refresh

30
Soft-state signaling
Sender
Receiver
Signaling plane
Communication plane

• best effort signaling
• refresh timer, periodic refresh
• state time-out timer, state removal only by
  time-out

31
Soft-state claims

• Systems built on soft-state are robust Raman
  99
• Soft-state protocols provide .. greater
  robustness to changes in the underlying network
  conditions Sharma 97
• obviates the need for complex error handling
  software Balakrishnan 99

What does this mean?
32
Soft-state easy handling of changes

• Periodic refresh if network conditions change,
  refresh will re-establish state under new
  conditions
• example RSVP/routing interaction if routes
  change (nodes fail) RSVP PATH refresh will
  re-establish state along new path

H3
H2
L8
L3
L2
L7
L6
R1
R2
R3
L4
H4
L1
L5
H1
unused by multicast routing
H5
33
Soft-state easy handling of changes

• L6 goes down, multicast routing reconfigures but
• H1 data no longer reaches H3, H4, H5 (no sender
  or receiver state for L8)
• H1 refreshes PATH, establishes new state for L8
  in R1, R3
• H4 refreshes RESV, propagates upstream to H1,
  establishes new receiver state for H4 in R1, R3

really, L7 state stays in R7 until it times out.
H3
H2
L8
L3
L2
L7
L6
R1
R2
R3
L4
H4
L1
L5
H1
H5
H5
34
Soft-state easy handling of changes

• recovery performed transparently to end-system
  by normal refresh procedures
• no need for network to signal failure/change to
  end system, or end system to respond to specific
  error
• less signaling (volume, types of messages) than
  hard-state from network to end-system but
• more signaling (volume) than hard-state from
  end-system to network for refreshes

35
Soft-state refreshes

• refresh msgs serve many purposes
• trigger first time state-installation
• refresh refresh state known to exist (I am
  still here)
• ltlack of refreshgt remove state (I am gone)
• challenge all refresh msgs unreliable
• would like triggers to result in
  state-installation asap
• enhancement add receiver-to-sender refresh_ACK
  for triggers
• e.g., see Staged Refresh Timers for RSVP

36
How do we model/evaluate?

• Metrics
• inconsistency ratio - fraction time participating
  nodes disagree
• signaling overhead average of messages during
  session lifetime
• measures of robustness?
• convergence time
• complexity?

37
last time

• continuous time Markov chains
• steady state behavior
• soft state, hard state

38
Signaling spectrum
SS reliable trigger/removal ST-II
SS explicit removal IGMPv2/v3

Hard-state
Soft-state
SS reliable trigger RSVP new version

• best effort periodic state
• installation/refresh
• state removal by time out
• RSVP, IGMPv1

• reliable signaling
• explicit state removal
• requires additional mechanism to
• remove orphan state
• Q2931b

39
Evaluation metrics

• inconsistency ratio - fraction time participating
  nodes disagree
• signaling overhead average of messages during
  session lifetime
• robustness?
• will look at Lui, etal

40
Single hop model

• sender, receiver
• single state variable
• state lifetime, exp(m)
• updates Poisson(l)
• timers exponentially distributed
• refresh - 1/T
• state expiration 1/X
• link delay exp(1/D) ,loss prob. p

41
Model for SS (Ji03)
42
Model for SS
43
Model for SS
44
Model for SS
45
Model for SS
46
Model for SS
47
Model for SS
48
Model for SS
49
Performance metrics (SS)

• inconsistency ratio
• d 1 p
• signaling overhead

50
Parameter settings

• mean lifetime 30 min.
• refresh timer, T5sec
• state timer, X 15 sec
• update rate 1/20sec
• signal loss rate 2
• Motivated by Kazaa

51
Impact of state lifetime

• inconsistency, overhead decrease as state
  life-time increases
• explicit removal improves consistency with little
  additional overhead

52
Soft-state setting timer values

• Q How to set refresh/timeout timers
• state-timeout interval n refresh-interval-time
  out
• what value of n to choose?
• will determine amount of signaling traffic,
  responsiveness to change
• small timers fast response to changes, more
  signaling
• long timers slow response to changes, less
  signaling
• ultimately consequence of slow/fast response,
  msg loss probability will dictate appropriate
  timer values

53
Impact of soft-state refresh timer

• as refresh timer increases
• inconsistency increases
• overhead decreases
• tradeoff exists for SS signaling systems

54
Impact of state timeout timer
T5s

• X lt T inconsistency high (premature state
  removal)
• X gt 2T increasing X ? increasing inconsistency
  for SS, SSER, SSRT (due to orphan state)
• X 2T sweet spot

55
Robustness of SSLui, etal
56
Protocol Performance Hypothesis
ambient conditions

• Hard State better under ambient conditions
• Soft State better under abnormal conditions

performance
acceptable performance
network condition
57
Modeling behavior

• what distinguishes the protocols?
• costs?
• which parameters impact robustness?

58
Cost model

• L Lifetime of state
• R Refresh interval
• T Timeout interval (typically 3R)
• p Channel loss probability
• K1 , K2 etc. Various Costs

Note parameters taken from Lui, et al.
59
Refresh Cost
Sender
Receiver
Signaling plane
Communication plane
60
(Re)Initialization Cost
Sender
Receiver
Signaling plane
Communication plane
of drops pL/R, Cost K2 pL/R
61
Stale state cost
Sender
Receiver
Signaling plane
Communication plane
Stale state lifetime R, Cost K3 pR
62
Total Cost
C(R) K2 pL/R K1 L/R K3 pR EC(R) K2
pEL/R K1 EL/R K3 pR
Optimal R
63
Optimal R implications

• K2, K1 -gt Performance emphasis
• K3 -gt Robustness emphasis
• Higher R, Harder the protocol, Lower R,
  Softer the protocol

64
Example Cost Comparison
Refresh intervals optimized emphasizing (i)
Performance for Hard State and (ii) Robustness
for Soft State
65
Hard-state versus soft-state discussion

• Q which is preferable and why?