Messaging Systems and Technology

1
Messaging Systems and Technology
2
Introduction

• Synchronous, CORBA-style middleware doesnt suit
  all applications
• Message Oriented Middleware (MOM) provides
  features like
• asynchronous communications between processes
• store-and-forward capabilities
• publish-subscribe
• transactional messaging

3
Basic Asynchronous Operations

• Send (dest, message)
• Receive ( target, message)

queue
receive
send
4
MOM usage

• Applications often need
• deferred processing of some slow transaction
  (eg printing an invoice)
• support efficient 1-to-many and many-to-many
  communications
• to send messages whether the server is available
  or not
• to provide event notification to a dynanic user
  community
• loosely coupled client-server systems

5
Example - Transactions
6
MOM Technologies

• Broadly two (somewhat intersecting) categories
• Message brokers (IBM, BEA, Microsoft, etc)
• Broadcast-based publish-subscribe (TIBCO, IONA)
• Both categories differ greatly in their features
  and capabilities

7
Some Application Examples

• Stock price update notification
• General workflow systems
• Application integration
• System management

8
Some Example Products

• TIBCO/Rendezvous
• IONAs OrbixTalk
• IBMs MQ Series (Publish-Subscribe)

9
TIB/Rendezvous

• Pioneers of broadcast/multicast publish-subscribe
  technology
• Publishers and subscribers communicate using
  subjects

Register/ Subscribe
Create/ Publish
Subject
Sub
Pub
Sub
Sub
10
Subjects

• Hierarchical names identify a subject of interest
• /CSIRO
• /CSIRO/gossip
• /CSIRO/gossip/ADSaT
• /CSIRO/work/ADSaT
• Wildcards can be used
• /CSIRO/
• /CSIRO//ADSaT

11
Multicast/Broadcast

• A published message is sent across the network
  only once
• All subscribers receive the same message
• IP Multicast or broadcast is used on LANs
• Important scalability issue as bandwidth/processor
  usage is low

12
LAN Architecture
13
WAN Architecture
Filtered, Point-to-Point
14
Quality-of-Service (QOS)

• Reliable
• publisher informed after time-out period if
  message cannot be delivered to a subscriber
• Certified
• guaranteed delivery or both parties informed of
  failure
• to survive process failure messages can be logged
  to disk until specified time-out period expires

15
Programming Options

• C/C/Java
• Self-describing message format
• TIB/ObjectBus layers a standard CORBA-compliant
  ORB upon the TIBCO protocols.
• TIBIOP provides broadcast messages in a CORBA
  environment.

16
OrbixTalk

• Uses IP multicast in CORBA environment
• Also supports CORBA Events Service IDL
• (see lab 4)

17
Topics

• Communication based on topics (read subjects,
  wildcards, etc)
• 3 QOS levels
• otmcp UDP IP multicast. Limits packet size to
  1280 bytes and no guarantee of receipt.
• otrmp Augments IP multicast to ensure message
  delivery to all subscribers until configurable
  time-out expires
• otsfp OrbixTalk MessageStore daemon uses a
  store-and-forward protocol to provide guaranteed
  message delivery.

18
OrbixTalk Architecture
19
Architecture Issues

• Daemons can have hot backups to support failure
  of primary daemon
• Several MessageStore daemons can be used to
  facilitate load balancing
• each OrbixTalk process only use one MessageStore
• Compaction utility must be scheduled to remove
  old messages

20
MQSeries PubSub

• MQSeries is probably most widely deployed MOM
  product
• Basic technology provides a message queue
  architecture
• communication via shared queues managed by Queue
  Managers
• persistent/non-persistent messages
• transactional queue access with XA
• broad platform support

21
MQSeries PubSub
1 PubSub Broker per QM
LAN/WAN support
Topics interchanged by Brokers if needed
22
Architecture

• Brokers can be organized to communicate
  hierarchically

HQ
USA
Asia
West
East
HK
23
Features

• Typical hierachical topic names, wildcards, etc
• Streams can be used as higher level topic
  partitioning scheme
• DEFAULT stream per broker
• optional additional streams for specified topics
• broker allocates a thread to handle each stream

24
Quality-of-Service

• Persistent
• messages written to log file
• survive broker/QM failure
• Non-persistent
• messages lost if broker fails
• fast
• Message priority supported in queues

25
Other features

• MQ can act as an XA transaction manager
• Basic security provided, integrated with native
  operating system
• Security exits support integration with 3rd party
  products
• Base for many other products, eg MQ System
  Integrator

26
Performance Test

• Publishers and subscribers run on different
  machines
• Dual pentium 500Mhz NT 4.0 boxes, 0.5GB memory
• MQ QM/broker and Orbix daemons run on same
  machine as publishers
• Reliable, non-presistent protocols used

27
256 Byte Message
28
512 Byte Message
29
Interpretation

• Multicast technologies are fast!
• TIBCO rvd becomes bottleneck
• Publisher too fast exceptions
• OrbixTalk generates large number of interrupts
• one per message per listener
• MQ scales well, but queue/broker architecture is
  inherently slower

30
Summary

• MOM provides excellent solution to many business
  problems
• Range of technologies available, with very
  different
• architectures/technologies
• features
• performance