EndtoEnd Realtime QoS Guarantees in DOC Middleware

1
End-to-End Real-time QoS Guarantees in DOC
Middleware

• Irfan Pyarali
• irfan_at_oomworks.com
• SCR Presentation

2
Presentation Outline

• Motivation for QoS-enabled middleware
• Limitations of CORBA when applied to real-time
  systems
• RT-CORBA improves end-to-end predictability
• Event and Notification services
• Dynamic Scheduling
• Transport protocol selection for DRE applications
• CORBA Component Model
• Model Driven Architecture
• Concluding remarks

3
Motivation for QoS-enabled Middleware

• Building distributed systems is hard
• Building them on-time under budget is even
  harder

• Many mission-critical distributed applications
  require real-time QoS support
• e.g., combat systems, online trading, telecom
• Building QoS-enabled applications manually is
  tedious, error-prone, expensive
• Conventional middleware does not support
  real-time QoS requirements effectively

4
Motivating Mission-Critical Applications
Large-scale Switching Systems
Total Ship Computing Environments
Industrial Process Control
Theater Missile Defense
5
Overview of CORBA

• Common Object Request Broker Architecture (CORBA)
• A family of specifications
• OMG is the standards body
• Over 800 companies
• CORBA defines interfaces, not implementations
• It simplifies development of distributed
  applications by automating/encapsulating
• Object location
• Connection memory mgmt.
• Parameter (de)marshaling
• Event request demultiplexing
• Error handling fault tolerance
• Object/server activation
• Concurrency
• Security

• CORBA shields applications from heterogeneous
  platform dependencies
• e.g., languages, operating systems, networking
  protocols, hardware

6
Limitations of CORBA for Real-time Systems

• Requirements
• Location transparency
• Performance transparency
• Predictability transparency
• Reliability transparency

• Historical Limitations
• Lack of QoS specifications
• Lack of QoS enforcement
• Lack of real-time programming features
• Lack of performance optimizations

7
Experiment Increasing Workload in CORBA

• Experiment
• Measure the disruption caused by increasing
  workload in CORBA
• Server
• 3 threads
• Client
• 3 rate-based invocation threads
• High ? 75 Hertz
• Medium ? 50 Hertz
• Low ? 25 Hertz

8
Results Increasing Workload in CORBA
9
Real-time CORBA Overview

• RT-CORBA adds QoS control to regular CORBA
  improve the application predictability
• Bounding priority inversions
• Managing resources end-to-end
• Policies mechanisms for resource
  configuration/control in RT-CORBA include
• Processor Resources
• Thread pools
• Priority models
• Portable priorities
• Communication Resources
• Protocol policies
• Explicit binding
• Memory Resources
• Request buffering
• These capabilities address some (but by no means
  all) important real-time application development
  challenges

10
Why is End-to-End Priority Propagation Relevant?

• Preserve priority as activities flow between
  endsystems
• Respect priorities to resolve resource contention
• Bounded request latencies

11
Tracing an Invocation End-to-end
Client ORB
Server ORB
Connection Cache
Memory Pool
Connection Cache
Memory Pool
A
B
S
S
S
X
Y
C
Reply Demultiplexer
Connector
Acceptor
Reactor
POA
Reactor
CV1
CV1
CV1
X
Y
C
C
C
A
B
CV2
S
S
S
1. Lookup connection to Server (S)
2. Lookup failed make new connection to Server
(S)
3. Add new connection to cache
4. Add new connection to Reactor
5. Accept new connection from Client (C)
6. Add new connection to Cache
7. Add new connection to Reactor
8. Wait for incoming events on the Reactor

9. Allocate memory to marshal data
10. Send data to Server marking connection (S)
busy
11. Wait in Reply Demuliplexer some other thread
is already leader, become follower


12. Read request header
13. Allocate buffer for incoming request
14. Read request data
15. Demultiplex request and dispatch upcall
16. Send reply to client
17. Wait for incoming events

18. Leader reads reply from server
19. Leader hands off reply to follower
20. Follower unmarshals reply
12
Identifying Sources of Unbounded Priority
Inversion
Client ORB
Server ORB
Connection Cache
Memory Pool
Connection Cache
Memory Pool
A
B
S
X
Y
C
Reply Demultiplexer
Connector
Acceptor
Reactor
POA
Reactor
CV1
X
Y
C
A
B
CV2
S

• Connection cache
• Time required to send request is dependent on
  availability of network resources and the size of
  the request
• Priority inheritance will help
• Creating new connections can be expensive and
  unpredictable

• Memory Pool
• Time required to allocate new buffer depends on
  pool fragmentation and memory management
  algorithm
• Priority inheritance will help

• Reply Demultiplexer
• If the leader thread is preempted by a thread of
  higher priority before the reply is handed-off
  (i.e., while reading the reply or signaling the
  invocation thread), then unbounded priority
  inversion will occur
• There is no chance of priority inheritance since
  signaling is done through condition variables

• Reactor
• No way to identify high priority client request
  from one of lower priority

• POA
• Time required to demultiplex request may depend
  on server organization
• Time required to dispatch request may depend on
  contention on dispatching table
• Priority inheritance will help

13
RT-CORBA Architecture
ORB A
ORB B
Connector
POA
Connector
POA
Low Priority Lane
Low Priority Lane
Connection Cache
Memory Pool
Connection Cache
Memory Pool
A
B
S
A
B
S
Leader/Followers
Acceptor
Leader/Followers
Acceptor
Reactor
Reactor
CV1
CV1
CV2
CV2
A
B
S
A
B
S
High Priority Lane
High Priority Lane
Connection Cache
Memory Pool
Connection Cache
Memory Pool
A
B
S
A
B
S
Leader/Followers
Acceptor
Leader/Followers
Acceptor
Reactor
Reactor
CV1
CV1
A
B
CV2
S
A
B
CV2
S
14
Experiment Increasing Workload in RT-CORBA with
Lanes

• Experiment
• Measure the disruption caused by increasing
  workload in RT-CORBA with lanes
• Increasing priority ? increasing rate
• Server
• 3 thread lanes
• High / Medium / Low
• Client
• 3 rate-based invocation threads
• High ? 75 Hertz
• Medium ? 50 Hertz
• Low ? 25 Hertz

15
Results Increasing Workload in RT-CORBA with
Lanes
16
Notification Service Component Structure
17
RT-Notification Architecture

• Concurrency options
• Reactive
• ThreadPool
• ThreadLane
• QoS support
• Priority
• Ordering
• Discarding
• Timeouts

18
End-to-end Priority Preservation
19
Comparison of Static and Dynamic Scheduling
20
Challenges Dynamic Scheduling in Distributed
Systems

• Tasks span multiple heterogeneous hosts
• Tasks and hosts they span are not known a priori
• Propagate scheduling requirements of a task
  across hosts it spans
• Allow custom scheduling discipline plug-ins
• Fixed priorities for tasks insufficient for
  scheduling analysis
• Cancel a distributed task and reschedule
  accordingly on hosts it spans
• Local schedulers need to reschedule when tasks
  return from a remote invocation
• Collaboration of schedulers in the system to
  ensure global scheduling

21
Dynamic Scheduling with RT-CORBA 2.0

• Distributable thread distributed concurrency
  abstraction
• Scheduling segment governed by a single
  scheduling policy
• Locus of execution point at which distributable
  thread is currently running
• Scheduling policies determine eligibility of
  different distributable threads
• Dynamic schedulers enforce distributable thread
  eligibility constraints

22
Distributable Threads
23
Pluggable Scheduling

• Scheduling/Dispatching
• Enforce predictable behavior in DRE systems
• Alternative disciplines
• RMS, MUF, EDF, LLF
• Custom scheduling disciplines dictated by system
  requirements
• Queried via resolve_initial_references
  (RTScheduler)
• RTSchedulingScheduler interface
• From which implementations are derived
• Interactions with application and ORB
• Scheduler Managers
• Install/manage schedulers in the ORB

Running DT
Waiting Distributable Threads (DTs)
DT QoS Descriptor
Lane (possibly one of many)
24
Scheduling Points

• Begin scheduling segment or spawn
• Update scheduling segment
• End scheduling segment
• Send request
• Receive request
• Send reply
• Receive reply

25
Transport Protocol Woes in DRE Systems
26
Stream Control Transport Protocol (SCTP)

• IP based transport protocol originally designed
  for telephony signaling
• SCTP supports features found to be useful in TCP
  or UDP
• Reliable data transfer (TCP)
• Congestion control (TCP)
• Message boundary conservation (UDP)
• Path MTU discovery and message fragmentation (TCP)
  
• Ordered (TCP) and unordered (UDP) data delivery
• Additional features in SCTP
• Multi-streaming multiple independent data flows
  within one association
• Multi-homed single association runs across
  multiple network paths
• Security and authentication checksum, tagging
  and a security cookie mechanism to prevent
  SYN-flood attacks
• Multiple types of service
• SOCK_SEQPACKET message oriented, reliable,
  ordered/unordered
• SOCK_STREAM TCP like byte oriented, reliable,
  ordered
• SOCK_RDM UDP like message oriented, reliable,
  unordered
• Control over key parameters like retransmission
  timeout and number of retransmissions

27
Testbed Configurations

• Both links are up at 100 Mbps
• 1st link has 1 packet loss
• Systematic link failures
• Up 4 seconds, down 2 seconds
• One link is always available

28
Throughput Tests

• Emulating applications that want to get bulk data
  from one machine to another as quickly as
  possible
• Two protocols IIOP, SCIOP
• DIOP not included because it is unreliable
• Frame size was varied from 1 to 64k bytes
• Client was sending data continuously
• IDL interface
• one-way void method(in octets payload)
• void twoway_sync()
• Experiment measures
• Time required by client to send large amount of
  data to server

29
Experiment ThroughputNetwork Both links are
up

• IIOP peaks around 94 Mbps
• SCIOP is up to 28 slower for smaller frames
• SCIOP is able to utilize both links for a
  combined throughput up to 122 Mbps

30
Experiment ThroughputNetwork 1 packet loss
on 1st link

• 1 packet loss causes maximum IIOP bandwidth to
  reduce to 87 Mbps (8 drop)
• IIOP outperforms SCIOP for smaller frames
• SCIOP maintains high throughput for larger
  frames, maxing out at 100 Mbps

31
Experiment ThroughputNetwork Systemic link
failure

• Link failure causes maximum IIOP throughput to
  drop to 38 Mbps (60 drop)
• SCIOP outperforms IIOP for all frame sizes
• SCIOP maxes out at 83 Mbps

32
Latency Tests

• Emulating applications that want to send a
  message and get a reply as quickly as possible
• Two protocols IIOP, SCIOP
• DIOP not included because it is unreliable
• Frame size was varied from 0 to 64k bytes
• Client sends data and waits for reply
• IDL interface
• void method(inout octets payload)
• Experiment measures
• Time required by client to send to and receive a
  frame from the server

33
Experiment LatencyNetwork Both links are up

• Mean IIOP latency comparable to SCIOP
• For larger frames, maximum latency for SCIOP is
  15 times maximum latency for IIOP

34
Experiment LatencyNetwork 1 packet loss on
1st link

• 1 packet loss causes maximum IIOP latency to
  reach about 1 second
• SCIOP outperforms IIOP for both average and
  maximum latencies for all frame sizes

35
Experiment LatencyNetwork Systemic link
failure

• Link failure causes maximum IIOP latency to reach
  about 4 seconds
• SCIOP outperforms IIOP for both average and
  maximum latencies for all frame sizes

36
Experiments Summary
37
Shortcomings of CORBA Middleware

• CORBA does not specify how assembly and
  deployment of object implementations should be
  done to create larger applications
• Proprietary infrastructure and scripts are
  usually written to facilitate this

38
Emergence of Component Middleware

• Standard mechanisms for assembly and deployment
  of applications
• Components aggregate related interfaces
• into logical, reusable units
• Containers provide execution environment for
  components
• Containers communicate via a middleware bus

39
Application Development Using CCM

• Separates concerns
• Server configuration
• Object/service configuration
• Application configuration
• Object/service deployment

40
Stages of Development Lifecycle
41
Model Driven Architecture
42
Concluding Remarks

• End-to-end QoS guarantees requires careful
  engineering of subsystems to ensure
• Predictability, scalability, and performance
• Avoid unbounded priority inversion
• End-to-end QoS preservation also require that
  object services also understand and enforce
  end-system QoS requirements
• Dynamic scheduling is essential for complex
  systems that cannot be scheduled a priori and/or
  have changing QoS requirements
• Enforcing network QoS is also necessary in
  achieving end-to-end predictability
• Components and Model Driven Architecture are
  essential for modeling, analyzing, synthesizing
  and provisioning next generation of DRE systems

43
Pattern identification goes beyond CORBA

• Distributed, real-time applications developers
  confront common challenges
• Patterns can capture key design and performance
  characteristics of proven mechanisms
• Reuse successful strategies
• Communicate design alternatives and decisions
• Evaluate proposed designs and implementations
• Careful application of patterns can yield
  efficient, predictable, scalable, and flexible
  middleware
• Create pattern handbook for developing real-time
  DOC middleware