Enhancing Realtime CORBA via Realtime Java features

1
Enhancing Real-time CORBAvia Real-time Java
features
Raymond Klefstad Elec Comp. Eng.
Dept University of California, Irvine klefstad_at_uci
.edu
Arvind S. Krishna Douglas C. Schmidt Elec
Comp. Eng. Dept Vanderbilt University arvindk,
schmidt_at_dre.vanderbilt.edu
International Conference on Distributed Computer
Systems Tokyo, Japan Friday, March 19, 2004
2
Talk Outline

• Tech transitions in the DRE domain ? Real-time
  middleware Real-time Java
• RT-Java real-time middleware ? ZEN project
• ZEN RD process
• Design Architecture
• Applying Real-time Java features
• Empirical Results
• Concluding remarks and References

3
DRE Domain Characteristics

• Types
• Wide applicability
• Range from Total ship-board computing systems to
  Industrial process control systems
• Common Requirement
• Right answer delivered too late becomes wrong
  answer
• Characteristics and Requirements
• Distributed Systems ?require capabilities to
  manage connections and message transfer between
  separate machines
• Real-time Systems ? require predictable and
  efficient control over end-to-end system
  resources
• Embedded Systems ? have weight, cost, and power
  constraints that limit their computing and memory
  resources

Industrial Process Control
Increasingly DRE applications are combined to
form systems of systems
4
Current Real-time Middleware Trends

• Current Real-time CORBA ORBs are developed in C
  and C
• ACETAO from ISIS Vanderbilt
• ORBexpress from OIC
• eORB from Prism Technologies
• Real-time CORBA has not been universally adopted
  by DRE application developers
• Complexity of the CORBA C mapping
• Steep learning curve caused by feature rich and
  complex C language
• Increasingly hard to find good C application
  developers and retain them

• Java Programming Language has emerged having less
  complexity than C
• Safety
• Simplicity
• Productivity
• However, Java is not suitable for developing DRE
  applications

5
RTSJ Thread Memory Models
6
Scoped Memory in Action
Obj A created in Heap region
Heap
time
(new RealtimeThread()) .start()
new A()
7
Scoped Memory in Action
Ma1 is an inner scope can hold references only
to outer regions
Note ma1 is reference is in heap,
Heap
time
(new RealtimeThread()) .start()
8
Talk Outline

• Tech transitions in the DRE domain ? Real-time
  middleware Real-time Java
• RT-Java real-time middleware ? ZEN project
• Design Architecture
• Applying Real-time Java features
• Empirical Results
• Concluding remarks and References

9
Motivation for ZEN Real-time ORB

• Integrate best aspects of several key
  technologies
• Java Simple, less error-prone, large user-base
• Real-time Java Real-time support
• CORBA Standards-based distributed applications
• Real-time CORBA CORBA with Real-time QoS
  capabilities
• ZEN project goals
• Make development of distributed, real-time,
  embedded (DRE) systems easier, faster, more
  portable
• Provide open-source Real-time CORBA ORB written
  in Real-time Java to enhance international
  middleware RD efforts

10
Phase I Applying Opt Strategies

• Foot-print Reduction Optimization
• Micro ORB Architecture ? Virtual Component
  Pattern
• Micro POA Architecture ? Pluggable components
• Request Demux/Dispatch Optimizations
• Connection Management ? Acceptor-Connector
  pattern, Reactor (javas nio package)
• Buffer Management Strategies
• Request Demultiplexing ? Active Demultiplexing
  Perfect Hashing
• POA Optimizations
• Object Key Processing Strategies ? Asynchronous
  completion token pattern
• Servant lookup ? Reverse lookup map
• Concurrency Strategies ? Half-Sync/Half-Async

11
Phase II Applying RTSJ

• Phase I ? Optimization patterns and principles
• ORB-Core Optimizations
• Micro ORB Architecture ? Virtual Component
  Pattern
• Connection Management ? Acceptor-Connector
  pattern, Reactor (javas nio package)
• Collocation and Buffer Management Strategies
• POA Optimizations
• Request Demultiplexing ? Active Demultiplexing
  Perfect Hashing
• Object Key Processing Strategies ? Asynchronous
  completion token pattern
• Servant lookup ? Reverse lookup map
• Concurrency Strategies ? Half-Sync/Half-Async

• Phase II ? Enhance Predictability by applying
  RTSJ features
• Associate Scoped Memory with Key ORB Components
• I/O Layer Acceptor-Connector, Transports
• ORB Layer CDR Streams, Message Parsers
• POA Layer Thread-Pools and Upcall Objects
• Using NoHeapRealtimeThreads
• Ultimately use NHRT Threads for request/response
  processing
• Reduce priority inversions from Garbage Collector

12
Talk Outline

• Tech transitions in the DRE domain ? Real-time
  middleware Real-time Java
• RT-Java real-time middleware ? ZEN project
• Design Architecture
• Applying Real-time Java features
• Empirical Results
• Open challenges
• Concluding remarks and References

13
Applying RTSJ features Motivation

• Original design of ZEN
• All components allocated in heap
• Request processing thread may be preempted by
  GC (demand garbage collection)

• Goals
• Compliance with CORBA specification
• Interoperability with classic CORBA
• Reduce overhead for applications not using
  real-time features
• End-user transparent

14
Analyzing Request Processing Steps
Server Side Connection Acceptance 4. An acceptor
accepts the new incoming connection. 6. A new
connection handler T1 is created to service
requests 7. The Transport's event loop waits for
data events from the client Server Side Request
Processing Steps 11. The request header on
connection is read to determine the size of the
request. 12. A buffer of the corresponding size
is obtained from the buffer manager to hold
request and read data. 13. The request is the
demultiplexed to obtain the target POA, servant,
and skeleton servicing the request. The upcall is
dispatched to the servant after demarshaling the
request. 14. The reply is marshaled using the
corresponding GIOP message writer Transport
sends reply to the client.
Independent Steps for two different clients do
not share context
Repetitive Ephemeral Carried out for each
client request Typically objects live for one
cycle
Thread Bound Steps executed by I/O threads
Thread-Pool threads
15
Application of Scoped Memory ZEN
Ephemeral
Create all objects in a scoped region

• Two requests can be mapped to two separate scope
  regions
• Temporary objects may be cleared after request
  processing

Independent
Threadbound

• Encapsulate steps as logic class associate this
  logic class with real-time threads
• Threads enter the scoped region processes
  request exits region enabling objects to be
  finalized

16
Scope Memory in ZEN Action
Associate a start scope with real-time thread
POA Scope
1 (new RealtimeThread(default Scope))
ORB Scope
The three scopes are created during ORB
initialization time
I/O Scope
time
NETWORK
Following Slides are adapted from Angelo Corsaro
17
I/O Stage Processing

• I/O Scope
• Participants The participants for this phase
  include, acceptors, and transports
• RTSJ application
• Each of these components are thread-bound
  components and are designed based on inner logic
  class
• Corresponds to the logic run by the thread
• Instead of creating the entire component in
  scoped memory, we create the inner logic class in
  a scoped memory region, mio
• This logic class is associated with the thread at
  creation time

I/O Scope
time
NETWORK
1Data arrival
18
ORB Stage processing

• ORB Scope
• Participants Message parsers, CDR Streams
• RTSJ application
• The appropriate message parser associated to
  parse request
• The message parser and buffer created in a nested
  memory region, morb.
• Using RTSJ memory rules, references from the ORB
  to the I/O space are valid

The thread enters ORB scope to parse request
Nested inner scope all refs from ORB - I/O are
valid
ORB Scope
I/O Scope
new GIOPMessage ()
time
NETWORK
19
POA Stage processing
Enter POA scope to process request and send
response

• Steps
• Demux request to get target POA, servant and
  skeleton
• Perform upcall on the servant marshal reply back
  to client
• RTSJ Application
• Message parser parses the request to find
  target servant and skeleton
• Set up context for the upcall
• Upcall Object holds info necessary to perform
  upcall
• Output buffer holds response

POA Scope
Up-call related objects created in this scope
ORB Scope
I/O Scope
parseAndProcessRequest()
new GIOPMessage ()
time
NETWORK
20
Talk Outline

• Tech transitions in the DRE domain ? Real-time
  middleware Real-time Java
• RT-Java real-time middleware ? ZEN project
• Design Architecture
• Applying Real-time Java features
• Empirical Results
• Open challenges
• Concluding remarks and References

21
Predictability Enhancement

• Overview
• POA Demultiplexing experiment conducted to
  measure improvement in predictability
• Result Synopsis
• Average Measures
• Scoped Memory does have some overhead 3 ?s
• Dispersion Measures
• Considerable improvement in predictability
• Dispersion improves by a factor of 4
• Worst-Case Measures
• Scoped memory bounds worst case
• Heap shows marked variability

• Associating scoped memory
• Does not compromise performance
• Significantly enhances predictability
• Bounds worst case latency

22
Enter Exit Analysis

• Overview
• Quantify overhead incurred by using Scope
  Memory
• enter () entering scope region
• exit () leave the scope region
• Result Synopsis
• Average Measures
• Constant enter () time across all message sizes.
• exit () time increases with message size
• Dispersion Measures
• exit () methods incur considerable variability
  when compared to enter ()
• Worst-Case Measures
• Similar behavior to both enter and exit () time

• On exit finalizers of objects run, hence larger
  messages have higher average latency
• enter () time uses constant time O(1) algorithm
  to validate illegal entry

23
Roundtrip Latency Analysis

• Overview
• Influence of Scoped memory in the Roundtrip
  latency measures
• Result Synopsis
• Average Measures
• For smaller clients, scoped memory incurs greater
  overhead
• As requests increase, Scoped memory outperforms
  heap
• Dispersion Measures
• Considerable improvement in predictability
• Dispersion improves as much as 50
• Worst-Case Measures
• Scoped memory bounds worst case
• Though mean values are greater 99 and Worst case
  measures are bounded

• As number of requests increase, GC activity
  increases for Heap Memory.
• Scope memory kicks in to reduce GC activity
  thereby improving processing time

24
Concluding Remarks Future Work

• Concluding Remarks
• We presented RD efforts on integration of RTSJ
  and RT-CORBA
• Our efforts focus towards effective use of RTSJ
  and Real-time CORBA to improve QoS for Java based
  real-time systems

• Future Real-Time CORBA Research
• Policies at the POA level for RTSJ aware users
• Use NHRT threads for request/response processing
• Threading Models for RTSJ
• Modeling RTSJ exceptions e.g. ScopedCycleException
  
• Complete implementation of Real-time CORBA
  specification

• Downloading ZEN
• www.zen.uci.edu

25
References

• ZEN open-source download web page
• http//www.zen.uci.edu
• Real-time Java (JSR-1)
• http//java.sun.com/aboutJava/communityprocess/jsr
  /
• jsr_001_real_time.html
• Dynamic scheduling RFP
• http//www.omg.org/techprocess/meetings/schedule/
  Dynamic_Scheduling_RFP.html
• Distributed Real-time Java (JSR-50)
• http//java.sun.com/aboutJava/communityprocess/jsr
  / jsr_050_drt.html
• AspectJ web page
• http//www.aspectJ.org
• JRate
• http//tao.doc.wustl.edu/corsaro/jRate/