Distributed-OMAR: Reconfiguring a Lisp System as a Hybrid Lisp/(Java) Component

1
Distributed-OMARReconfiguring a Lisp System as
a Hybrid Lisp/(Java) Component
Nichael Cramer 17 Nov 1998
GTE-I / BBN Technologies
2
The OMAR System

• Additional information
• Stephen Deutsch (sdeutsch_at_bbn.com).
• http//www.sover.net/nichael/misc/
• (Papers available at front.)

GTE-I / BBN Technologies
Distributed-OMAR
3
The OMAR System

• Simulation Development Environment.
• Editing/Browsing tools.
• Displays for runtime monitoring.
• Application-specific displays (e.g. radars).
• Procedure language (SCORE).
• Simulation engine.
• Human in the Loop experiments.

GTE-I / BBN Technologies
Distributed-OMAR
4
The OMAR System

• OMAR has been primarily used in the modeling of
  human performance.
• Multi-tasking capabilities of human-operators.
• Teamwork activities.
• Air Traffic Control and Flight Deck simulation.

GTE-I / BBN Technologies
Distributed-OMAR
5
The OMAR System

• For the present talk
• OMAR is a large, simulation application, written
  in Lisp (ACL) and CLIM, running on Unix (Solaris
  and SGI) platforms.
• Signal/Event-based semantics for inter-procedure
  communication.

GTE-I / BBN Technologies
Distributed-OMAR
6
Distributed-OMAR

• OMAR was successful.
• (I.e. the customer was very happy.)
• Interest in distributing OMAR.
• OMAR as intelligent agent in existing systems.
• Network of OMAR agents.
• Interact with other agents/components.
• Web-based operation.

GTE-I / BBN Technologies
Distributed-OMAR
7
Distributed-OMAR Architecture Design Goals

• Distributed
• OMAR should be able to participate as an equal in
  distributed systems consisting of multiple
  remote, independent Agents or other Components.
• OMARs internal modules should be able to run in
  a distributed manner as needed.

GTE-I / BBN Technologies
Distributed-OMAR
8
Distributed-OMAR Architecture Design Goals

• Flexible.
• Usable with
• Broad range of systems.
• Multiple external languages/platforms.
• Assortment of middleware choices.
• Stable as standards change.

GTE-I / BBN Technologies
Distributed-OMAR
9
Distributed-OMAR Architecture Design Goals

• Robust and Maintainable.
• To the extent possible, there should be a single
  D-OMAR
• I.e. not a Corba-OMAR a RMI-OMAR etc.
• Main System Functionality in Core-OMAR (Lisp)
  system.
• Efficient.

GTE-I / BBN Technologies
Distributed-OMAR
10
Distributed-OMAR Architecture Design Goals

• Maintain the central Lisp functionality.
• Maintain the advantages that Lisp provides.
• Dynamic class inheritance.
• Rapid-prototyping capabilities.
• Etc.
• Specifically Want to be able to continue writing
  the core simulation engine in Lisp.

GTE-I / BBN Technologies
Distributed-OMAR
11
Distributed-OMAR Architecture Overview

• Distributed-OMAR presents itself to the external
  components as being a single unified component in
  the target language.
• Examples
• An OMAR Bean in Java.
• An OMAR Federate in HLA.

GTE-I / BBN Technologies
Distributed-OMAR
12
Distributed-OMAR Architecture High-Level
Structure

• Core-OMAR module...
• (Compact version of Lisp simulation system.)
• through inter-module communication
• (Socket-based serialization protocol.)
• to Connection in External language.
• (Presence of D-OMAR in foreign language.)

GTE-I / BBN Technologies
Distributed-OMAR
13
Distributed-OMAR Architecture High-Level
Structure (cont.)

• In external Language, D-OMAR presents a basic
  External Connection object.
• External Connection behaves as a source and sink
  of D-OMAR events in the External language.

GTE-I / BBN Technologies
Distributed-OMAR
14
Distributed-OMAR Architecture

• Can run with assortment of external languages.
• Java is language of choice.
• But, other languages available.
• Piggy-back off features of external or embedded
  language.
• Can use standard networking tools.
• E.g. RMI in Java CORBA HLA
• Graphics packages, etc.

GTE-I / BBN Technologies
Distributed-OMAR
15
Distributed-OMAR Architecture

• The Core-OMAR system remains compact and intact.
• Remains in pure CommonLisp.
• It continues to be able to do what it does well.
• Only a small layer (Cap) needs to changed to work
  in different systems.

GTE-I / BBN Technologies
Distributed-OMAR
16
Core OMAR Module

• Original Lisp Kernel.
• Score
• SFL
• Rule-based language
• Pure Common Lisp
• Plus socket package.
• Direct graphics/editing support removed.
• Primarily Signal (Event) based semantics.

Core OMAR
GTE-I / BBN Technologies
Distributed-OMAR
17
Core OMAR API

• Well-defined, cleaned-up functional API protocol.
• Commander
• Incoming Commands
• Listener
• Outgoing Commands

Core OMAR
GTE-I / BBN Technologies
Distributed-OMAR
18
OMAR Master

• Implementation of the Core-OMAR API.
• Governs communication with outside world.
• More than one possible implementation.
• Command-Line.
• OMAR Classic

Master
Core OMAR
GTE-I / BBN Technologies
Distributed-OMAR
19
Distributed Master

• Communicates to external Connections
• Connection written in language of host system
  (e.g. Java, C).
• Communication is a socket-based, serialization,
  message-passing protocol.

GTE-I / BBN Technologies
Distributed-OMAR
20
Sockets

• Sockets
• Standard mechanism for inter-process
  communication.
• TCP/IP based communication layer.
• Common to virtually all platforms.

GTE-I / BBN Technologies
Distributed-OMAR
21
Serialization

• Serialization
• Mechanism for moving objects (Strings, Events,
  Structures, etc) between processes.
• (Encoding representation of Objects into a
  byte-stream representation, to be decoded by an
  external process.)

GTE-I / BBN Technologies
Distributed-OMAR
22
Message-Passing

• Message-passing
• High-level specification of a request for a
  state-change.
• Event, method call, .
• In the stream, an object with a state-flag and
  a set of arguments.

GTE-I / BBN Technologies
Distributed-OMAR
23
Distributed Master (continued)

• External Connections can run on
• Same machine (standard mechanism).
• Different machine.
• Event-based communication.
• Independent of language of Connection.
• Single protocol for all Connection types.

GTE-I / BBN Technologies
Distributed-OMAR
24
Distributed Master (continued)

• Current Connection types
• Control GUI Connection.
• OMAR editing and simulation-control panels.
• Application GUI Connection.
• Panels specific to Current application.
• External Connection.
• Communicates with other Agents.

GTE-I / BBN Technologies
Distributed-OMAR
25
Distributed-OMAR Component
Control GUI Connection
Distributed Master
Core OMAR
Java External Connection
Application GUI Connection
GTE-I / BBN Technologies
Distributed-OMAR
26
Control GUI Connection
Scenario Control Panel
Distributed Master
Core OMAR
Procedure Browser
Control GUI Connection
LISP
JAVA
Concept Editor
GTE-I / BBN Technologies
Distributed-OMAR
27
Control GUI Connection -Java

• In the Java world, OMAR (as viewed through the
  external Connection) is now just another Java
  component.
• To which other Java component connect in a
  standard way.

GTE-I / BBN Technologies
Distributed-OMAR
28
Control GUI Connection -Java(continued)

• Can use any standard Java mechanism for
  inter-component communication.
• Regular Object-Oriented method calls.
• RMI.
• For example, publish and subscribe to Events in
  standard way (Listener).

GTE-I / BBN Technologies
Distributed-OMAR
29
Control GUI Connection -Java (continued)

• In short, foreign-language Connection behaves as
  conduit into Core-OMAR.
• Connections serve as sources and sinks of
  Events in the Java world.
• Finally, nothing special about the fact that the
  connection is written in Java.

GTE-I / BBN Technologies
Distributed-OMAR
30
Control GUI Connection
Scenario Control Panel
Distributed Master
Core OMAR
Procedure Browser
Control GUI Connection
Concept Editor
GTE-I / BBN Technologies
Distributed-OMAR
31
Distributed-OMAR Architecture
External D-OMAR Agent 1
Control GUI Connection
Distributed Master
Core OMAR
External D-OMAR Agent 2
Application GUI Connection
External Agent 3
GTE-I / BBN Technologies
Distributed-OMAR
32
Communication with External Agents

• Standard External Connection Object.
• Common to all module types.
• D-OMAR presents a thin Cap to External
  language.
• Wrapper on External-Connection object.
• Specific to communication/middleware type.
• Inner structure hidden to outside world.

GTE-I / BBN Technologies
Distributed-OMAR
33
Communication with External Agents
External Agent 1
Communication Specific Cap
External Agent 2
External Agent 3
GTE-I / BBN Technologies
Distributed-OMAR
34
Communication with External Agents

• Java Inter-Agent Communication
• Standard O-O method invocation.
• RMI (Remote Method Invocation).
• Corba
• Voyager
• HLA

GTE-I / BBN Technologies
Distributed-OMAR
35
Standard O-O Method Invocation
External Agent 1
Standard Cap
External Agent 2
GTE-I / BBN Technologies
Distributed-OMAR
36
Remote Method Invocation (RMI)
External Agent 1
RMI Cap
Central Remote Server
External Agent 2
GTE-I / BBN Technologies
Distributed-OMAR
37
ORB Connection (e.g. Corba)
External Component 1
Corba Cap
O R B
External Component 2
GTE-I / BBN Technologies
Distributed-OMAR
38
Distributed-OMAR Architecture
External D-Omar Agent 1
Control GUI Connection
Distributed Master
Core OMAR
External D-Omar Agent 2
Application GUI Connection
External Agent 3
Distributed Omar Component
Remote Systems
GTE-I / BBN Technologies
Distributed-OMAR
39
OMAR Communication with Other Implementation
Languages

• Event-communication doesnt depend on details of
  Event-Listener.
• Connection only needs to support
• Sockets.
• Serialization protocol.

GTE-I / BBN Technologies
Distributed-OMAR
40
OMAR Communication with Other Implementation
Languages

• Connection need not be written in Lisp or Java.
• Nothing in Core-OMAR module changes.
• An example HLA.

GTE-I / BBN Technologies
Distributed-OMAR
41
D-OMAR/HLA Architecture
External Federate 1
Control GUI Connection (Java)
Federate Cap
Distributed Master (Lisp)
Core OMAR (Lisp)
R T I
C External Connection
External Federate 2
Application GUI Connection (Java)
GTE-I / BBN Technologies
Distributed-OMAR
42
D-OMAR/HLA Architecture

• C based External Connection and HLA Cap have
  been implemented and initially tested.
• Currently converting to vendor supplied Java/C
  Federate interface/wrapper.

GTE-I / BBN Technologies
Distributed-OMAR
43
Distributed-OMAR Architecture Summary/Status

• Consolidated Core-OMAR kernel.
• In Common-Lisp/CLOS
• Processes.
• Socket-code.
• (Again, no graphics code.)
• Can run in any Lisp that supports these.
• Currently using Allegro Common-Lisp.
• (MCL all but sockets/Open Transport).
• (Harlequin)

GTE-I / BBN Technologies
Distributed-OMAR
44
Distributed-OMAR Architecture Summary/Status
(continued)

• Standard Protocol for Connection to External
  Languages.
• Current implementations in
• Java.
• C (HLA).
• Lisp.

GTE-I / BBN Technologies
Distributed-OMAR
45
Distributed-OMAR Architecture Summary/Status
(continued)

• GUI implementation in Java.
• Implementation of Java-based communication among
  collection of D-OMAR agents.
• In Java Runs anywhere.

GTE-I / BBN Technologies
Distributed-OMAR
46
Distributed-OMAR Architecture Summary/Status
(continued)

• Current Platforms
• Windows NT/98.
• Sun/Sparc.
• (SGI)
• (MacIntosh)

GTE-I / BBN Technologies
Distributed-OMAR
47
Distributed-OMAR Architecture Summary/Status
(continued)

• Middlewares used.
• Corba.
• HLA.
• RMI.
• (Nexus object for collection of D-OMAR agents.)
• (Simple Java Method calls.)

GTE-I / BBN Technologies
Distributed-OMAR
48
Distributed-OMAR Architecture Summary/Status
Recap

• Single Common-Lisp source code for all system
  configurations
• At most, involves a recompile.
• Single Java source code (exclusive of
  application-specific Cap).
• Class files (binaries) Run anywhere.

GTE-I / BBN Technologies
Distributed-OMAR
49
Distributed-OMARReconfiguring a Lisp System as
a Hybrid Lisp/(Java) Component
Nichael Cramer 17 Nov 1998
GTE-I / BBN Technologies
50
Appendix 1ObjectStream Structure

• ObjectStream class
• Supports socket-based read/write bytes
  operations.
• Supports serialization API.
• (read-object ltobjstreamgt)
• (write-object ltobjstreamgt ltobjectgt)

GTE-I / BBN Technologies
Distributed-OMAR
51
Appendix 1ObjectStream Structure

• ObjStreamParser object
• Passed as argument to ObjStream.
• Governs details of translation in reads/writes in
  serialization.

GTE-I / BBN Technologies
Distributed-OMAR
52
Appendix 1OMAR Parser Class Structure

• base-parser
• (Error-handling basic read/write operations.)
• core-parser
• (strings, numbers, arrays, etc.)
• lisp-parser
• (Cons cells lists etc.)
• omar-event-parser
• (omar-events concept-structures etc.)

GTE-I / BBN Technologies
Distributed-OMAR
53
Appendix 2ObjectStream as Eval-Server

• Message-passing not the only protocol
  ObjectStream.
• Java calls to Lisp can be written as
  (stringified) Lisp form.

GTE-I / BBN Technologies
Distributed-OMAR
54
Appendix 2ObjectStream as Eval-Server

• Eval-Server
• Use existing serialization protocol.
• Java submits request as string into Lisp.
• submitEval((setq mainval ( 3 substate)))
• Lisp performs a read-from-string.
• Returns (serialized) result value to Java.

GTE-I / BBN Technologies
Distributed-OMAR
55
Appendix 3DrawStream/Procedure Browser

• Tool for graphically viewing structures of and
  communications between procedures in D-OMAR
  models.
• Multiple Views
• Super-/sub-procedures (who calls, etc.)
• Signals generated/waited-for
• Procedure view.

GTE-I / BBN Technologies
Distributed-OMAR
56
Appendix 3DrawStream/Procedure Browser

• Super/Sub-procedures and Signal views are
  graph-based.
• Java Gui requests data-structure to view.
• Lisp prepares the data-structure and ships it off
  (via serialization) to Java.
• Java uses standard graphing display code to
  display the results.

GTE-I / BBN Technologies
Distributed-OMAR
57
Appendix 3DrawStream/Procedure Browser

• Procedure view (uses DrawStream)
• View is laid-out and drawn in Lisp.
• Uses pre-existing Lisp display code.
• Display code draws to a DrawStream object.
• DrawStream encodes drawing commands as a sequence
  of bytes.

GTE-I / BBN Technologies
Distributed-OMAR
58
Appendix 3DrawStream/Procedure Browser

• Representation (sequence of bytes) is shipped to
  Java.
• Currently as an Array (in an Event).
• Alternatively Live byte-stream.
• Bytes are rendered in Java.
• DrawStreamCanvas.
• Decodes byte-sequence representation.

GTE-I / BBN Technologies
Distributed-OMAR
59
Appendix 3DrawStream/Procedure Browser

• Gestures are handled
• Transmit screen location back to Lisp.
• Lisp returns any appropriate object.
• Note that the DrawStream object and the Java
  DrawStreamCanvas class are write once
  constructs which can be re-used with other Lisp
  displays.

GTE-I / BBN Technologies
Distributed-OMAR
60
Appendix 3DrawStream/Procedure Browser

• Demo.

GTE-I / BBN Technologies
Distributed-OMAR
61
Distributed-OMAR
Additional information http//www.sover.net/nich
ael/misc/ Nichael Cramer 17 Nov 1998
GTE-I / BBN Technologies
62
Distributed-OMARReconfiguring a Lisp System as
a Hybrid Lisp/(Java) Component
Nichael Cramer 17 Nov 1998
GTE-I / BBN Technologies
63
Why not simply Corba? (foreign-function calls?,
etc?)

• When we started, no Lisp Corba binding.
• This has changed/is changing.
• But, for example, no signal servicing.
• Dont want to be Corba-specific.
• I.e no multiple versions to run with
• HLA, DCOM, RMI

GTE-I / BBN Technologies
Distributed-OMAR
64
Why not simply Corba? (foreign-function calls?,
etc?)

• Lightweight
• Dont need an ORB to (e.g.) just run a GUI.
• D-OMAR has a single protocol to communicate with
  all external processes.
• Works with any CommonLisp
• (Provided it supports sockets.)

GTE-I / BBN Technologies
Distributed-OMAR
65
Why not simply Corba? (foreign-function calls?,
etc?)

• By moving the burden into (Java), we can
  automatically leverage any Java-based middle-ware
  support.
• In short, we are betting that Java will always
  have better support for middleware, GUIs, etc.

GTE-I / BBN Technologies
Distributed-OMAR
66
Why not simply Corba? (foreign-function calls?,
etc?)

• Finally, if the decision is made to convert
  D-OMAR to a Corba base, we only have to re-write
  the Connection protocol.
• Modular design ensures a (relatively -) painless
  transition.

GTE-I / BBN Technologies
Distributed-OMAR
67
Distributed-OMARReconfiguring a Lisp System as
a Hybrid Lisp/(Java) Component
Nichael Cramer 17 Nov 1998
GTE-I / BBN Technologies
68
Distributed-OMARReconfiguring a Lisp System as
a Hybrid Lisp/(Java) Component
Nichael Cramer 17 Nov 1998
GTE-I / BBN Technologies
69
Example Command-Line Master
gt Scenario select ATC gt Scenario run gt
Report time 0. gt Report time 5. gt Report
Event (Comm AC-11 ATC-23) gt Report time 10. gt
Report Event (ATC-23 Acknowledge)

Command Line Master
Core Omar
GTE-I / BBN Technologies
70
Application GUI Connection
Radar Display
Control GUI Connection
Distributed Master
Core Omar
ATC Control Panel
Application GUI Connection
Cockpit Display
GTE-I / BBN Technologies
71
Control GUI Connection -Java (continued)

• Events function as high-level request for
  state-change.
• Independent of nature of Subscriber.
• No restrictions on how Subscriber uses events.
• I.e. it can do what it wants with events.

GTE-I / BBN Technologies
Distributed-OMAR
72
D-OMAR/HLA Architecture

• Federate-Cap functions as conduit into/from
  Core-OMAR module.
• Because Cap has no application-specific
  information, it can be reused.
• All modeling behavior is handled within Core-OMAR.

GTE-I / BBN Technologies
Distributed-OMAR
73
D-OMAR/HLA Architecture
External Federate 1
Federate Cap
Distributed Master
Core OMAR
R T I
C External Connection
External Federate 2
LISP
C
GTE-I / BBN Technologies
Distributed-OMAR