Asynchronous Completion Token

1
Asynchronous Completion Token

• Eric Galyon
• April 18, 2006

2
Outline

• Overview
• Background
• Example Uses
• Detailed Problem Description
• Alternative Solutions
• ACT Solution
• Solution Structure
• Implementation Activities
• Overview of Variations
• Benefits and Liabilities

3
Overview 1 of 2

• The Asynchronous Completion Token design pattern
• Specialization of the Proactor pattern. Can be
  applied to the Reactor pattern but becomes a
  Synchronous Completion Token.
• Used for event handling for asynchronous service
  calls.
• Efficiently demultiplexes asynchronous event
  calls made to other services to the correct event
  handler.

4
Overview 2 of 2

• The Asynchronous Completion Token design pattern
• Allows for the preservation of state information
  needed to process the event.
• Minimizes state communication overhead between
  the client and service.

5
Background 1 of 4

• Definitions
• ACT Asynchronous Completion Token.
• Completion Event the response sent back from a
  service to a client when an asynchronous event
  finishes.
• Demultiplexing determining which event
  handler(s) will process a completion event.
• Blocking Synchronous I/O- An I/O operation which
  stops the application from processing until the
  I/O completes.

6
Background 2 of 4

• Definitions
• Non-blocking Synchronous I/O- An I/O operation
  that, prior to executing, checks if it will
  block. If it would not block, the I/O runs to
  completion and returns. If it would block, it
  returns immediately and indicates to the
  application that the I/O operation would block.
  The application must try the I/O operation again
  later.
• Non-blocking Asynchronous I/O An I/O operation
  that begins and returns immediately. The
  application continues processing as the I/O
  operation processes asynchronously and is
  notified when the I/O operation completes. Must
  be supported by Operating System level
  asynchronous I/O calls.

7
Background 3 of 4

• Related Patterns
• Reactor
• Synchronous calls
• Example
• Application wants to read a resource.
• Reactor watches for resource availability.
• Reactor notifies application when resource
  available.
• Application begins read of resource, processes
  read, and may resubscribe to another read on a
  resource.

8
Background 4 of 4

• Related Patterns
• Proactor
• Asynchronous calls
• Example
• Application begins asynchronous read call on a
  resource.
• Proactor watches for read of resource to
  complete.
• Proactor notifies the application when the read
  is done.
• Application processes read information and may
  begin another asynchronous read.
• ACT is useful for optimizing the demultiplexing
  process of a Proactor.

9
Example Uses 1 of 2

• HTTP-Cookies
• Web servers can use ACTs for processing responses
  from web browsers.
• Windows NT (2000, 2003, XP)
• Overlapped I/O (non-blocking asynchronous I/O)
  for reading and writing.
• POSIX
• Asynchronous I/O API for calling aio_read() or
  aio_write().

10
Example Uses 2 of 2

• CORBA Demultiplexing
• Object Request Broker uses the ACT pattern for
  demultiplexing General Inter-ORB Protocol
  requests and responses.
• FedEx Inventory Tracking
• FedEx invoices provide a field that can be filled
  in by the sender.
• When the package is delivered the invoice will be
  sent including the filled in field.
• The sender can use the returned field as an ACT
  to reference a completion action.

11
Detailed Problem Description 1 of 2

• A client issues an asynchronous call to a
  service.
• A service returns a completion event to the
  client when the asynchronous call completes.
• The application must demultiplex the completion
  event to determine how to appropriately handle it.

12
Detailed Problem Description 2 of 2

• Complicating Factors
• The application may require the specific state
  information in effect when the service was called
  to process the result.
• There should be as little communication overhead
  as possible between the client and service.
• The application should spend as little time as
  possible demultiplexing and forwarding the
  completion event.

13
Alternative Solutions 1 of 2

• Reactor with synchronous calls
• Long running event handlers can starve the
  Reactor and prevent timely reaction to other
  events.
• Asynchronous calls are more efficient than
  synchronous calls.
• More efficient use of system resources.
• Minimizes process/thread wait time.
• Provides immediate service to client requests.

14
Alternative Solutions 2 of 2

• Create a thread for each service call.
• Doesnt scale well.
• Context-switching overhead between threads
  impedes performance.
• Requires difficult concurrency control for
  threads accessing shared resources .
• Thread optimization usually attempts to match
  thread per processor not thread per client.

15
ACT Solution

• When a client invokes an asynchronous operation,
  create and send an asynchronous completion token
  to uniquely identify the completion handler and
  references the state needed to process the event.
• Have the service pass the unmodified ACT back to
  the client when the operation is complete.
• Use the ACT to identify the correct completion
  handler and restore and necessary state for
  processing the completed operation.

16
Solution Structure 1 of 5
17
Solution Structure 2 of 5

• Service
• Provides asynchronous service operation.
• Returns unaltered ACT to initiator when operation
  completes.

18
Solution Structure 3 of 5

• Initiator
• Invokes asynchronous service operations.
• Demultiplexes completion events and calls correct
  completion handler.

19
Solution Structure 4 of 5

• Asynchronous Completion Token
• Created by client that uses Initiator to make the
  asynchronous operation call.
• Passed from the Initiator to the Service.
• Passed, unaltered, back from the Service to the
  Initiator.
• Initiator demultiplexes based on the ACT and
  returns the ACT to the client to process the
  response.

20
Solution Structure 5 of 5

• Completion Handler
• Interface client implements to define the results
  of a completed asynchronous operation.
• Invoked by the Initiator after demultiplexing the
  asynchronous operation completion event.

21
Implementation Activities 1 of 7

• Six activities for implementing the Asynchronous
  Completion Token pattern
• 1. Define the ACT representation.
• 2. Select a strategy for holding the ACT at the
  Initiator.
• 3. Determine how to pass the ACT from the
  Initiator to the Service.
• 4. Determine a strategy for holding the ACT in
  the Service.
• 5. Determine the number of times the ACT can be
  used.
• 6. Determine the Initiator strategy for
  demultiplexing ACTs to completion handler hook
  methods.

22
Implementation Activities 2 of 7

• 1. ACT Representation
• Should be meaningful to the client and Initiator
  but opaque to the Service.
• Pointer ACT Point to methods or objects that
  implement the Completion Handler interface.
• Object Reference ACT Middleware references such
  as a CORBA object pointer.
• Index ACT A unique index into a table maintained
  by the Initiator to associate the unique index
  with the appropriate Completion Handler.

23
Implementation Activities 3 of 7

• 2. Initiator strategy for holding ACT
• The Initiator may invoke more than one service
  from more than one client at the same time and
  must remember the ACT for demultiplexing.
• Implicit ACT Initiator holds the address of the
  Completion Handler.
• Explicit ACT Initiator holds a data structure
  that contains references back to the Completion
  Handler.

24
Implementation Activities 4 of 7

• 3. Passing ACT from Initiator to Service
• Implicit parameters ACT is passed transparently
  to the Service as part of the context or
  environment of invoking the operation.
• Explicit parameters ACT is passed as a specific
  parameter of the operation.

25
Implementation Activities 5 of 7

• 4. Holding the ACT at the Service
• Different strategies depending on if the service
  executes the operation synchronously or
  asynchronously.
• Synchronously ACT can remain on the runtime
  stack during the execution of the operation.
• Asynchronously ACT must be stored in a data
  structure and retrieved at the end of the
  operation.

26
Implementation Activities 6 of 7

• 5. ACT Reuse
• The Initiator and Service must agree upon
  how/when ACTs can be reused.
• An Initiator can either create a new ACT for
  every action or, if the operation will be
  repeated and handled by the same Completion
  Handler, the ACT can be reused to reduce the cost
  of creating and managing ACTs.
• A Service may send a single response back to an
  event or may send multiple responses each time a
  repeatable event occurs.

27
Implementation Activities 7 of 7

• 6. Demultiplexing ACTs to Completion Handlers
• When an Initiator receives a Completion Event it
  must determine which Completion Handler(s) to
  invoke.
• Queued Completions If numerous events will all
  return and be processed by a single Completion
  Handler, they can be added to a queue and the
  client can remove and process them.
• Callbacks If different types of events will be
  processed by different types of Completion
  Handlers, callbacks can be used to join types of
  events to handlers.

28
Overview of Variations

• Chain of Service ACTs
• If Services become initiators to other services,
  ACTs must be preserved.
• Can be passed statically along entire path.
• New ACTs can be generated between each service.
• Non-opaque ACTs
• Services may be designed to make modifications to
  ACTs.
• Synchronous ACTs
• ACTs can be used to demultiplex synchronous calls
  resulting in a Synchronous Completion Token.

29
Benefits and Liabilities 1 of 3

• Benefits
• Simplified Initiator data structures to
  efficiently link a Completion Event to its
  Completion Handler.
• Efficient state acquisition by using a token to
  retrieve state rather than fully recording and
  passing the state around.
• Space efficiency for memory and communication
  bandwidth used.
• Flexibility since clients define ACTs instead of
  inheriting specific ACT structures from services.
• Better concurrency since long lasting operations
  can run asynchronously.

30
Benefits and Liabilities 2 of 3

• Liabilities
• Memory leaks can occur if an ACT tracking state
  never returns from a call and never garbage
  collected.
• Authentication that the returned ACT from a
  service has not been altered.
• If ACTs point to direct memory references for
  Completion Handlers and an application is
  re-mapped in memory, errors can occur.

31
Benefits and Liabilities 3 of 3

• Inherited liabilities from Proactor
• Restricted applicability since not all operating
  systems support asynchronous calls.
• Complex programming and debugging.
• Can be difficult for Initiators to schedule,
  control, and cancel asynchronous operations.

32
References

• D. C. Schmidt, M.Stal, H.Rhnert, and F.
  Buschmann. Pattern-Oriented Software
  Architecture Patterns for Concurrent and
  Networked Objects, Volume 2. Wiley Sons, New
  York, 2000.
• Reactor, Proactor, Asynchronous Completion Token,
  and Active Object patterns.
• A Libman, and V Gilbourd. Comparing Two
  High-Performance I/O Design Patterns. Artima
  Developer web site http//www.artima.com/articles
  /io_design_patterns.html.
• Reactor and Proactor background information.
• IBM eServer iSeries Information Center Version 5
  Release 3 documentation. Socket communication
  programming synchronous versus asynchronous
  system calls. http//publib.boulder.ibm.com/infoce
  nter/iseries/v5r3/index.jsp?topic/rzab6/rzab6cmul
  tiplex.htm.