Designing and Optimizing a Scalable CORBA Notification Service

1
Designing and Optimizing a Scalable CORBA
Notification Service

• Pradeep Gore, Ron Cytron pradeep,cytron_at_cs.wustl
  .edu
• Department of Computer Science, Washington
  University, Saint Louis

Doug Schmidt, Carlos ORyan schmidt,coryan_at_uci.e
du Electrical and Computer Engineering,
University of California, Irvine
2
Overview

• CORBA Notification Service
• CORBA Event Service and drawbacks
• Structured Events, Filtering and QoS properties.
• Design Challenges
• Handling multiple event, supplier and consumer
  types uniformly.
• Efficiently propagating different event types to
  different consumers.
• Minimizing interference between channel
  participants.
• Fairness in Event Processing
• Optimizing the performance of Any's.
• Footprint reduction.
• Customizing for particular deployment
  environments.
• Performance Measurements
• Event Channel Scalability (per consumer
  throughput).
• Effect of different thread configurations on
  consumer throughput.
• Filtering overhead.
• Concluding Remarks

3
CORBA Overview

• CORBA is a family of open specification
• It simplifies development of distributed
  applications by automating/encapsulating
• Object location
• Connection memory mgmt.
• Parameter (de)marshaling
• Event request demultiplexing
• Error handling fault tolerance
• Object/server activation
• Concurrency
• Security
• RT CORBA adds QoS control to regular CORBA
  improve the application predictability, e.g.,
• Bounding priority inversions
• Managing resources end-to-end

• CORBA shields applications from heterogeneous
  platform dependencies
• e.g., languages, operating systems, networking
  protocols, hardware

4
Performance-Critical CORBA Applications
Large-scale Switching Systems
Industrial Process Control
Theater Missile Defense
5
CORBA Invocation

• Standard "CORBA" method invocations result in
  synchronous execution of an operation provided by
  an object
• Both requestor (client) and provider (server)
  must be present
• Client blocks until operation returns
• Only supports uni-cast communication

6
CORBA Event Service

• For many applications, a more decoupled
  communication model between objects is required.
  (e.g. publish-subscribe)
• asynchronous communication with multiple
  suppliers and consumers.

7
Common Event Service Collaborations
8
Benefits of the OMG Event Service

• Anonymous consumers/suppliers
• Publish and subscribe model
• Group communication
• Supplier(s) to consumer(s)
• Decoupled communication
• Asynchronous delivery
• Abstraction for distribution
• Can help draw the lines of distribution in the
  system
• Abstraction for concurrency
• Can facilitate concurrent event handling

9
Notification Service

• Extends Event Service
• Filtering
• Consumers can specify arbitrarily complex
  filtering constraints in Extended TCL (Trader
  Constraint Language).
• Structured Events
• carry filtering and QoS parameters that influence
  the delivery of payload data to its destination.

• Sharing subscription information
• Subscriptions and publications are visible to
  participants.
• QoS properties
• Allows suppliers, consumers administrators of
  event channels to configure QoS properties such
  as reliability, priority, buffer ordering and
  timeliness on a per-channel, per-proxy or
  per-event basis.

10
Structure of the CORBA Notification Service
11
Structured Events

• Defines a standard data structure into which a
  wide variety of event messages can be stored.
• Fixed event header contains type information -
• Domain (e.g.Telecom, Health Care, Financial)
• Type ( e.g. Communication Alarm, VitalSigns,
  StockQuote)
• Event Name (e.g. heartrate, ECG reading)
• Variable header has per event QoS properties as
  (name, value) pairs.
• Event Body has filterable body fields that are
  used by filters, and the payload data.

12
Example Use case of Notification Service
13
Design Challenges Solutions
14
Challenge 1 Handling Multiple Event, Supplier
and Consumer types uniformly

• Context
• Events can have different representations - e.g.
  Any, Structured Sequence or custom
• Corresponding Consumers and Suppliers can have
  different representations e.g. StructuredConsumer
• Problem
• Event Channel must propogate events in one form
  to consumers that require it in another form.
• Do not want to convert to a cannonical format or
  make many copies.
• For similar IDL interfaces, do not want to
  duplicate code in implementation.

15
Solution1 Adapter Pattern

• Adapter converts the interface of a class to
  another interface that a client expects.
• Adapter object implements a Target interface and
  delegates operations to the Adaptee.
• Notify_Event target interface
• Any_Event, Structured_Event Adapters
• CORBAAny, CosNotificationStructuredEvent
  Adaptee

16
Challenge 2 Efficiently propagating different
event types to different consumers

• Context
• The event channel can receive events that have to
  be dispatched to consumers that accept another
  type of event. E,g, Any event to Structured
  consumer.
• Problem
• Different event types have to be propagated to
  different types of receiving consumers.

17
Solution 2 Visitor pattern

• Visitor allows to represent an operation to be
  performed on the elements of an object structure
• Double Dispatching
• Channel invokes dispatch_event method that in
  turn invokes push_event on the event.

18
Challenge 3 minimizing interference between
channel participants

• Context
• Event Channel must receive requests, apply
  filtering and dispatch events to consumers.
• Some suppliers might not want to be blocked in a
  CORBA 2-way while event channel delivers the
  event to consumers.
• Filter evaluation might be a lengthy operation
  (e.g. A remote filter that needs to consult a
  database)
• A consumer might be arbitrarily slow hence
  causing the event channel to block in the
  dispatching 2-way
• Problem
• A reasonable implementation should strive to
  minimize this interference between channel
  participants.

19
Solution 3 Active Object pattern

• Active object pattern decouples method execution
  from method invocation.
• Events and operations required on them are
  encapsulated as command objects.
• An enqueuing thread places events in the queue as
  per the buffering policy.
• Worker thread(s) dequeue each event and applies
  the operation.

20
Challenge 4 Ensuring fairness in event
processing

• Context
• At worst, the event processing overhead is a
  maximum of 4 filters and the time to deliver to
  consumer.
• Though events are queued according to priority, a
  long winded event processing phase can starve
  other events in the queue from making any
  progress.
• Problem
• Event processing should be broken up into
  different stages to ensure fairness.

21
Solution 4 Command Object Pattern

• A command object encapsulates a request as an
  object, thereby allowing parameterization of
  different requests.
• Filter processing, subscription lookup and event
  dispatching are encapsulated as command objects.

22
Challenge 5 Optimizing the performance of Anys

• Context
• A CORBA compliant Notification Service must be
  able to process events containing Anys.
• An Any is an expensive data type that can have
  many levels of nesting. E.g. an Any that contains
  a sequence of Anys and so on.
• The ORB demarshalling engine makes copies of the
  entire data buffer used to represent the Any.
• Problem
• Any intensive CORBA services such as Notification
  can benefit from reduced data copies of this
  expensive type.

23
Solution 5 Reference Counting

• A reference counted object can be shared by
  multiple objects and destroyed when it is not in
  use.
• Client applications rather than the ORB can be
  made to release a ref. counted Any.
• Very well suited for Notification as the service
  itself does not modify the event.
• As the event can be logically multicast to
  multiple recipients, the ownership can be shared
  via a reference count.

24
Other design challenges

• Footprint Reduction
• A use case might want a subset of the
  Notification Service.
• The builder pattern can be used to make the
  service composable by users.
• Customizing event channels for particular
  deployment environments.
• A developer might want to extend the service
  implementation.
• Need a way to specify options and properties for
  a configuration
• The Component Configurator pattern decouples the
  behavior of component services from the time the
  service is configured into an application.

25
Performance measurements

• Experimental Testbed
• QuadCPU PC
• 400MHz pentiumII processors
• Linux 2.2.18, GCC 2.95.4
• TAO 1.1.15, all optimizations enabled.

26
Event Channel Scalability
27
Effect of Thread configuration

• 2 consumers, 1 supplier
• slow consumer has a 1 sec. delay in push.
• Case 1 reactive
• Case 2 2 threads, 1 shared queue.
• Case 3 Thread and queue per proxy supplier.

28
Filtering overhead

• Filter attached to Consumer Admin, 2 dispatching
  threads.
• Filtering drops average throughput of 15 and
  picks up slightly by 2 with 2 more dispatching
  threads.

29
Concluding remarks

• Notification Service allows decoupled, anonymous
  communication between consumers and suppliers.
• Filtering and QoS properties
• Suitable Design patterns and reusable framework
  components were utilized to implement TAOs
  Notification Service.
• Critical path of event propagation is optimized.
• Configurable and extensible implementation.
• TAO and its Notification Event Service
  implementations are available at
• www.cs.wustl.edu/schmidt/TAO.html