Faculty recruitment talk

1
Response Time Analysis of a Middleware
Demultiplexing Pattern for Network Services
Aniruddha Gokhale a.gokhale_at_vanderbilt.edu Asst.
Professor of EECS, Vanderbilt University,
Nashville, TN
Swapna Gokhale ssg_at_engr.uconn.edu Asst. Professor
of CSE, University of Connecticut, Storrs, CT
Jeff Gray gray_at_cis.uab.edu Asst. Professor of
CIS Univ. of Alabama at Birmingham Birmingham, AL
Presented at IEEE Globecom 2005 Symposium on
Advances in Networks and Internet St. Louis
MO Nov 28-Dec 1, 2005 Work supported by
collaborative grant from NSF CSR-SMA Program
2
Problem Statement Estimating Performance
Characteristics of Network Services at Design-time

• Network services need support for efficient
  (de)-multiplexing, dispatching and
  routing/forwarding

• .e.g., VPN Service provided by a virtual router
• Provides differentiated services to customers,
  e.g., prioritized service
• VPN setup messages must be efficiently (de)
  multiplexed, serviced and forwarded
• Need to estimate capacity of the system at
  design-time

• Standards middleware is increasingly being used
  to develop network services e.g., J2EE, .NET,
  CORBA, Web services
• Middleware frameworks incorporate elegant
  patterns-based building blocks
• Problem boils down to estimating performance of
  middleware

3
Solution Approach Middleware Performance
Analysis using Stochastic Reward Nets
Transition
Inhibitor arc
Place
Immediate transition
Token

• Stochastic Reward Nets (SRNs) are an extension to
  Generalized Stochastic Petri Nets (GSPNs) which
  are an extension to Petri Nets.
• Extend the modeling power of GSPNs by allowing
• Guard functions
• Marking-dependent arc multiplicities
• General transition probabilities
• Reward rates at the net level
• Allow model specification at a level closer to
  intuition.
• Solved using tools such as SPNP (Stochastic Petri
  Net Package).

4
Goal Performance Analysis of Reactor Pattern in
VR

• Customers send VPN setup messages to router
• VPN setup messages manifest as events at the VR
• VR must service these events (e.g., resource
  allocation) and honor the prioritized service, if
  any
• Accepted messages are forwarded
• Events could be dropped in overload conditions

The Reactor architectural pattern allows
event-driven applications to demultiplex
dispatch service requests that are delivered to
an application from one or more clients.

• Reactor pattern decouples the detection,
  demultiplexing, dispatching of events from the
  handling of events
• Participants include the Reactor, Event handle,
  Event demultiplexer, abstract and concrete event
  handlers

5
Modeling VR Capabilities in a Reactor

• Consider VPN service for two customer classes
• Reactor accepts and handles two types of input
  events
• Differentiated services for two classes
• Events are handled in prioritized order
• Each event type has a separate queue to hold the
  incoming events. Buffer capacity for events of
  type one is N1 and of type two is N2.
• Event arrivals are Poisson for type one and type
  two events with rates l1 and l2, resp.
• Event service time is exponential for type one
  and type two events with rates m1 and m2, resp.

Model of a single-threaded, select-based reactor
implementation
6
Performance Metrics of Interest for VR (i.e.,
Reactor)

• Throughput
• -Number of events that can be processed
• -Applications such as telecommunications call
  processing.
• Queue length
• -Queuing for the event handler queues.
• -Appropriate scheduling policies for
  applications with real-time requirements.
• Total number of events
• -Total number of events in the system.
• -Scheduling decisions.
• -Resource provisioning required to sustain
  system demands.
• Probability of event loss
• -Events discarded due to lack of buffer
  space.
• -Safety-critical systems.
• -Levels of resource provisioning.
• Response time

7
Modeling the Reactor using SRN (1/2)
Event arr.
Drop events on overflow
Service queue
Prioritized service
Servicing the event
Service completion

• Models arrivals, queuing, and prioritized service
  of events.
• Transitions A1 and A2 Event arrivals.
• Places B1 and B2 Buffer/queues.
• Places S1 and S2 Service of the events.
• Transitions Sr1 and Sr2 Service completions.
• Inhibitor arcs Place B1and transition A1 with
  multiplicity N1 (B2, A2, N2)
• - Prevents firing of transition A1 when
  there are N1 tokens in place B1.
• Inhibitor arc from place S1 to transition Sr2
• - Offers prioritized service to an event
  of type one over event of type two.
• - Prevents firing of transition Sr2 when
  there is a token in place S1.

8
Modeling the Reactor using SRN (2/2)

• Process of taking successive snapshots
• Reactor waits for new events when currently
  enabled events are handled
• Sn1 enabled Token in StSnpSht Tokens in B1
  No Token in S1.
• Sn2 enabled Token in StSnpSht Tokens in B2
  No Token in S2.
• T_SrvSnpSht enabled Token in S1 and/or S2.
• T_EndSnpSht enabled No token in S1 and S2.
• Sn1 and Sn2 have same priority
• T_SrvSnpSht lower priority than Sn1 and Sn2

9
VR SRN Performance Estimates

• SRN model solved using Stochastic Petri Net
  Package (SPNP) to obtain estimates of performance
  metrics.
• Parameter values l1 0.5/sec, l2 0.5/sec, m1
  2.0/sec, m2 2.0/sec.
• Two cases N1 N2 1, and N1 N2 5.

• Observations
• Probability of event loss is higher when the
  buffer space is 1
• Total number of events of type two is higher than
  type one.
• Events of type two stay in the system longer than
  events of type one.
• May degrade the response time of event requests
  for class 2 customers compared to requests from
  class 1 customers

10
VR SRN Sensitivity Analysis

• Analyze the sensitivity of performance metrics to
  variations in input parameter values.
• Vary l1 from 0.5/sec to 2.0/sec.
• Values of other parameters l2 0.5/sec, m1
  2.0/sec, m2 2.0/sec, N1 N2 5.
• Compute performance measures for each one of the
  input values.

• Observations
• Throughput of event requests from customer class
  1 increases, but rate of increase declines.
• Throughput of event requests from customer class
  2 remains unchanged.

11
Next Steps Addressing Variability in Middleware
Although middleware provides reusable building
blocks that capture commonalities, these blocks
and their compositions incur variabilities that
impact performance in significant ways.

• Per Building Block Variability
• Incurred due to variations in implementations
  configurations for a patterns-based building
  block
• E.g., single threaded versus thread-pool based
  reactor implementation dimension that crosscuts
  the event demultiplexing strategy (e.g., select,
  poll, WaitForMultipleObjects

• Compositional Variability
• Incurred due to variations in the compositions of
  these building blocks
• Need to address compatibility in the compositions
  and individual configurations
• Dictated by needs of the domain
• E.g., Leader-Follower makes no sense in a single
  threaded Reactor

12
Next Steps Model-driven Performance Analysis of
Middleware-based Network Services
Applying design-time performance analysis
techniques to estimate the impact of variability
in middleware-based DPSS systems

• Build and validate performance models for
  invariant parts of middleware building blocks
• Weaving of variability concerns manifested in a
  building block into the performance models
• Compose and validate performance models of
  building blocks mirroring the anticipated
  software design of DPSS systems
• Estimate end-to-end performance of composed
  system
• Iterate until design meets performance
  requirements

Composed System
Refined model of a pattern
Refined model of a pattern
Invariant model of a pattern
Refined model of a pattern
Refined model of a pattern
Refined model of a pattern
weave
weave
variability
variability
Refined model of a pattern
Refined model of a pattern
system
13
Concluding Remarks

• Network services are implemented using middleware
  building blocks
• Need to estimate performance early in development
  lifecycle
• Stochastic Reward Nets enables scalable
  intuitive performance analysis
• Goal is to use model-driven generative techniques
  to automatically synthesize performance models
  for network services
• Analysis for other dimensions of quality of
  service e.g., trustworthiness, dependability
• www.cse.uconn.edu/ssg (Swapna Gokhale)
• www.dre.vanderbilt.edu/gokhale (Aniruddha
  Gokhale)
• www.gray-area.org (Jeff Gray)

14
Questions?
15
EXTRAS
16
Reactor Dynamics

• Registration Phase
• Event handlers register themselves with the
  Reactor for an event type (e.g., input, output,
  timeout, exception event types)
• Reactor returns a handle it maintains, which it
  uses to associate an event type with the
  registered handler
• Snapshot Phase
• Main program delegates thread of control to
  Reactor, which in turn takes a snapshot of the
  system to determine which events are enabled in
  that snapshot
• For each enabled event, the corresponding event
  handler is invoked, which services the event
• When all events in a snapshot are handled, the
  Reactor proceeds to the next snapshot

17
Reactor SRN Taking a Snapshot

• Part B
• Process of taking successive snapshots
• Sn1 enabled Token in StSnpSht Tokens in B1
  No Token in S1.
• Sn2 enabled Token in StSnpSht Tokens in B2
  No Token in S2.
• T_SrvSnpSht enabled Token in S1 and/or S2.
• T_EndSnpSht enabled No token in S1 and S2.
• Sn1 and Sn2 have same priority
• T_SrvSnpSht lower priority than Sn1 and Sn2

18
Reactor SRN Initial Marking
A1
A2
N2
N1
StSnpSht
B2
B1
Sn1
T_SrvSnpSht
Sn2
T_EndSnpSht
S2
S1
SnpShtInProg
Sr2
Sr1
(a)
(b)

• Initial marking
• StSnpSht 1, B1 2, B2 2, S1 0, S2 0
• Transitions enabled Sn1 and Sn2
• Sn1 fires.

19
Reactor SRN Firing a Transition (1/6)
A1
A2
N2
N1
StSnpSht
B2
B1
Sn1
T_SrvSnpSht
Sn2
T_EndSnpSht
S2
S1
SnpShtInProg
Sr2
Sr1
(a)
(b)

• Upon firing of Sn1
• StSnpSht 1, B1 1, B2 2, S1 1, S2 0
• Transitions enabled Sr1, Sn2, T_SrvSnpSht
• Sn2 and T_SrvSnhpSht are immediate transitions,
  have to fire before Sr1.
• T_SrvSnpSht has a lower priority over Sn2.
• Sn2 fires.

20
Reactor SRN Firing a Transition (2/6)
A1
A2
N2
N1
StSnpSht
B2
B1
Sn1
T_SrvSnpSht
Sn2
T_EndSnpSht
S2
S1
SnpShtInProg
Sr2
Sr1
(a)
(b)

• Upon firing of Sn2
• StSnpSht 1, B1 1, B2 1, S1 1, S2 1
• Transitions enabled Sr1, T_SrvSnpSht
• T_SrvSnhpSht is an immediate transition, has to
  fire before Sr1.
• T_SrvSnpSht fires.

21
Reactor SRN Firing a Transition (3/6)
A1
A2
N2
N1
StSnpSht
B2
B1
Sn1
T_SrvSnpSht
Sn2
T_EndSnpSht
S2
S1
SnpShtInProg
Sr2
Sr1
(a)
(b)

• Upon firing of T_SrvSnpSht
• TSnpShtInProg 1, B1 1, B2 1, S1 1, S2 1
• Transitions enabled Sr1
• Snapshot in progress.
• Sr1 fires.

22
Reactor SRN Firing a Transition (4/6)
A1
A2
N2
N1
StSnpSht
B2
B1
Sn1
T_SrvSnpSht
Sn2
T_EndSnpSht
S2
S1
SnpShtInProg
Sr2
Sr1
(a)
(b)

• Upon firing of Sr1
• TSnpShtInProg 1, B1 1, B2 1, S1 0, S2 1
• Transitions enabled Sr2
• Snapshot in progress
• Sr2 fires.

23
Reactor SRN Firing a Transition (5/6)
A1
A2
N2
N1
StSnpSht
B2
B1
Sn1
T_SrvSnpSht
Sn2
T_EndSnpSht
S2
S1
SnpShtInProg
Sr2
Sr1
(a)
(b)

• Upon firing of Sr2
• TSnpShtInProg 1, B1 1, B2 1, S1 0, S2 0
• Transitions enabled T_EndSnpSht
• End of snapshot
• T_EndSnpSht fires

24
Reactor SRN Firing a Transition (6/6)
A2
A1
N2
N1
StSnpSht
B1
B2
Sn1
Sn2
T_SrvSnpSht
T_EndSnpSht
S2
S1
SnpShtInProg
Sr2
Sr1
(a)
(b)

• Upon firing of T_EndSnpSht
• StSnpSht 1, B1 1, B2 1, S1 0, S2 0
• Transitions enabled Sn1 and Sn2
• Back to initial state

25
Reactor SRN Performance Measures
Reward rate assignments to compute performance
measures
A2
A1
N2
N1
StSnpSht
B1
B2
Sn1
Sn2
T_SrvSnpSht
T_EndSnpSht
S2
S1
SnpShtInProg
Sr2
Sr1
(a)
(b)

• Throughput Rate of firing of transitions Sr1
  (Sr2).
• Queue length Number of tokens in place B1 (B2).
• Total number of events Sum of the tokens in
  places B1 S1 (B2 S2)
• Probability of event loss Number of tokens in
  place B1 N1 (B2 N2)
• Response time Can be obtained using the tagged
  customer approach.
• SRN model solved using Stochastic Petri Net
  Package (SPNP) to obtain
• estimates of performance metrics.

26
Designing Evaluating SRNs for Network Services
A2
A1
N2
N1
StSnpSht
B1
B2
Sn1
Sn2
T_SrvSnpSht
T_EndSnpSht
S2
S1
SnpShtInProg
Sr2
Sr1
(a)
(b)

• Initial Step
• Obtain performance measures for individual
  patterns-based building blocks
• Iterative Algorithm
• Compose systems vertically and horizontally to
  form a DPSS system
• Determine performance measures for specified
  workloads and service times
• Alter the configurations until DPSS performance
  meets specifications.

27
VR SRN Disruption Detection

• Obtain an anomaly score for the Reactor based on
  each one of the
• performance metrics for each event type.
• Correlate the anomaly scores based on each event
  type to obtain an overall
• anomaly score for the Reactor.
• - Anomaly score for the Reactor used at each
  CE to demultiplex events from
• two groups within a single organization.
• Anomaly score for the Reactor in the VR used to
  demultiplex events from the
• two organizations.
• Correlate the anomaly score of the Reactor in the
  VR with the score of the
• Reactor in CE 1 to determine service disruptions
  for organization 1.
• Correlate the anomaly score of the Reactor in the
  VR with the score of the
• Reactor in CE 2 to determine service disruptions
  for organization 2.
• Source of disruption may be identified by
  correlating the scores at various layers.

28
Collaborative Research

• Performance analysis methodology (UConn S.
  Gokhale)
• Develop and validate performance models for
  invariant characteristics of building blocks.
• Compose and validate performance models for
  common building block compositions.
• Develop model decomposition and solution
  strategies to alleviate state-space explosion
  issue.
• Model-driven generative methodology (Vanderbilt
  A. Gokhale)
• Manually developing performance models of each
  block with its variations is cumbersome
• Compositions of building blocks cannot be made in
  ad hoc, arbitrary manner
• Model-driven generative tools use visual modeling
  languages and model interpreters to automate
  tedious tasks and provide correct-by-construction
  development
• Aspect-oriented methodology (Univ of Alabama,
  Birmingham J. Gray)
• Variability in building blocks and compositions
  is a primary candidate for separating the concern
  as an aspect
• Aspect weaving technology can be used to refine
  and enhance the models by weaving in the concerns
  into the performance models

29
VR SRN Expected Behavior

• VPN service has two modes of operation normal
  inclement.
• Normal mode
• - Daily basis, some employees have
  negotiated telecommute plans and
• use VPN for remote access.
• Inclement mode
• - Hazardous driving conditions due to bad
  weather may keep people at home.
• - Large number of telecommuters
• - Increase in the connection set up and tear
  down requests.
• Modes of operation can be defined at a finer
  level of granularity, such as
• a few hours, rather than a day.

30
VR SRN Expected Behavior

• Normal mode
• - l1 0.5/sec, l2 0.5/sec, m1 2.0/sec,
  m2 2.0/sec, N1 N2 5
• - Probability 0.9
• Inclement mode
• - l1 1.0/sec, l2 1.0/sec, m1
  2.0/sec, m2 2.0/sec, N1 N2 5
• - Probability 0.1