STELLA Lispstyle Symbolic Programming with Delivery in CommonLisp, C and Java

1
STELLA-Lisp-style Symbolic Programming
withDelivery in Common-Lisp, C and Java
Hans Chalupsky Robert M. MacGregor Loom/PowerLoom
Group USC Information Sciences Institute
2
Bio

• Member of Loom/PowerLoom group at USC?s
  Information Sciences Institute
• Involved in development of knowledge
  representation systems and related technology
• Research Interests
• KR systems
• Ontology construction, maintenance and
  translation
• Belief reasoning, cognitive modeling
• Programming languages
• 15 years of Inter/Common/Emacs-Lisp programming
  experience

3
What is STELLA?

• STELLA is an object-oriented programming language
  that
• resembles a simplified Common Lisp
• adds strong typing
• adds modules, iterators, triggers/demons
• STELLA programs translate into
• Common Lisp
• C
• Java
• Benefits
• STELLA combines fast prototyping of Lisp with
  efficiency of C.
• Programmer-friendly environment for developing
  (intelligent) symbol processing systems.
• Compatibility with commercial tools and
  applications.

4
The STELLA - Lisp Connection

• Looks like Lisp
• Strongly influenced by Lisp
• Translates into Common-Lisp
• Leverages Lisp development environments
• Provides an alternative to Lisp
• Facilitates transition of Lisp code into C or
  Java
• Is designed to support what Lisp is good at
• etc.

5
The STELLA Team

• Bob MacGregor
• invented STELLA around 1994 to implement
  PowerLoom KRS
• wrote most of the kernel STELLA system
• Hans Chalupsky
• mostly occupied with ?getting things to work?
• completed the first full bootstrap in Lisp
  sometime in Spring 96
• various fire fighting to handle the many C and
  Java idiosyncrasies
• maintainer of STELLA code walker/analyzer
• Eric Melz
• wrote C translator
• Tom Russ
• wrote Java translator

6
STELLA Experience World-Wide as of 10/99

• Amount of available STELLA Code
• we?ve written about 80000 lines of code (3 Meg)
• about 50 for STELLA kernel and 50 for PowerLoom
• STUG (STELLA Users Group)
• 3-4 expert-level programmers
• 2-3 apprentices

7
Requirements Guiding STELLA?s Design

• Emphasis on translation, not compilation
• Close mapping between STELLA and target language
  (C)
• C translator output must be
• readable, conventional and efficient
• Support programming down to the ?bare metal?
• Fast prototyping, minimal redundancy, late
  binding
• Minimal restrictions on placement of declarations
• Type inference (to minimize need for explicit
  type declarations)
• Automatic casting and coercion
• Uniform syntax for function call, method call,
  and slot access
• Full support for symbolic programming
• Symbols, dynamic data structures, backquote
• Interpreted function/method call, slot access,
  object creation
• Meta-object protocol (MOP)

8
STELLA Language Overview
9
STELLA Overview Syntax

• Parenthesized, uniform expression syntax similar
  to Lisp
• Definitional constructs similar to Common Lisp
  analogues (add types)

10
STELLA Overview Type System

• Strongly typed
• Static (compile-time) type checking - efficient
  compilation
• Explicit types required for
• Function/method parameters and return values
• Global variables
• Slot definitions
• Implicit typing of local variables (supported by
  type inference)

11
STELLA Overview Object System

• Single inheritance class/type hierarchy
• Restricted multiple inheritance via mixins
• Dynamic method dispatch via runtime type of first
  argument (similar to C and Java)
• Static (native) and dynamic slots
• Initial and default values for slots
• Slot access can trigger demons
• Meta-objects for classes, functions, methods,
  slots, variables, modules
• Simple MOP allows control of object
• creation
• initialization
• termination
• destruction

12
STELLA Overview Control Structure

• Functions and methods are distinguished
• Variable number of arguments possible
• Zero or more return values
• Lisp-style macros to support syntax extensions
• Expressions and statements are distinguished
• Lexical scoping for local variables
• Dynamic scoping via special variables
• Conditionals similar to Common-Lisp (if, when,
  unless, cond)
• Elegant, uniform and efficient iteration
• Built-in protocol for iterators
• Exception mechanism for error handling and
  non-local exits

13
STELLA Overview Symbolic Programming

• Symbols, keywords, surrogates are first-class
  objects
• First-class literals (via wrappers)
• Extensive support for dynamic datatypes
• cons-trees, lists, sets
• association lists, hash tables
• vectors, extensible vectors
• Backquote mechanism facilitates macros and code
  generation
• Interpreted function call, method call, slot
  access, object creation
• Restricted evaluator

14
STELLA Overview Name Spaces

• Functions/methods, variables and classes occupy
  separate name spaces
• Slot and method polymorphism
• Hierarchical module system partitions symbol
  tables
• facilitates large-scale programming

15
STELLA Overview Memory Management

• Automatic via garbage collector
• native for CL and Java
• Boehm/Weiser collector for C
• Some built-in support for explicit memory
  management

16
Influences from Non-Lisp Languages

• C
• Strong typing
• First-argument polymorphism
• Statements vs. expressions
• Java
• single-inheritance
• Eiffel
• Anchored and parametric types
• Sather
• Iterators
• Dylan
• ???

17
A Few Features Aren?t Finished

• STELLA still needs
• Better logical pathnames
• Formatted output similar to format, printf
• Manual (a very rudimentary one exists)

18
STELLA System Architecture
19
Translation Instead of Compilation

• Important STELLA Goals
• Simple application generation on various
  platforms and languages with conventional
  compiler technology
• Simple interoperability with non-STELLA
  applications and tools
• Maintenance by non-STELLA programmers
• Efficiency
• Solution
• Translation into readable, conventional, and
  efficient target code
• Minimal indirection
• Direct mapping between STELLA and native language
  (classes to native classes, functions to native
  functions, strings to native strings, etc.)
• Use target language as a high-level programming
  language instead of as a kind of assembly
  language

20
STELLA Code Examples
21
Method Example 1

• STELLA
• (defmethod (length INTEGER) ((self CONS))
• documentation "Return the length of the CONS
  list 'self'."
• (let ((cons self)
• (i 0))
• (while (non-empty? cons)
• ( i)
• (setq cons (rest cons)))
• (return i) ))
• C
  Common Lisp
• int Conslength () (CLdefmethod
  length ((self cons))
• // Return the length???? ?Return the
  length ???? ?
• Cons cons this (CLlet
  ((cons self)
• int i 0
  (i 0))
• (CLloop
  while (non-empty? cons)
• while (cons-non_emptyP()) do

22
Method Example 1 - Efficiency via Inlining

• STELLA
• (defmethod (length INTEGER) ((self CONS))
• documentation "Return the length of the CONS
  list 'self'."
• (let ((cons self)
• (i 0))
• (while (non-empty? cons)
• ( i)
• (setq cons (rest cons)))
• (return i) ))
• C
  Common Lisp
• int Conslength () (CLdefmethod
  length ((self cons))
• // Return the length???? ?Return the
  length ???? ?
• Cons cons this (CLlet
  ((cons self)
• int i 0
  (i 0))
• (CLloop
  while (CLnot (CLeq cons NIL))
• while (!(cons NIL)) do

23
Method Example 2 - Iteration

• STELLA
• C

(defmethod (member? BOOLEAN) ((self CONS) (object
OBJECT)) documentation "Return TRUE iff
'object' is a member of 'self'.? (foreach
element in self where (eql? element object)
do (return TRUE)) (return FALSE) )
boolean ConsmemberP (Object object) //
Return TRUE iff 'object' is a member of 'self?.
Object element NULL Cons iter_001
this while (!nilP(iter_001))
element iter_001-value iter_001
iter_001-rest if (element object)
return (TRUE) return
(FALSE)
24
Method Example 3 - Statement as Expression

• STELLA
• C

(defmethod (member? BOOLEAN) ((self CONS) (object
OBJECT)) documentation "Return TRUE iff
'object' is a member of 'self'.? (return
(exists element in self where (eql?
element object))))
boolean ConsmemberP (Object object) //
Return TRUE iff 'object' is a member of 'self'.
boolean foundP_001 FALSE Object
element NULL Cons iter_001 this
while (!nilP(iter_001))
element iter_001-value iter_001
iter_001-rest if
(eqlP(element, object)) foundP_001
TRUE break
boolean value_001 foundP_001 return
(value_001)
25
Class Example 1

• STELLA
• C

(defclass CONS (STANDARD-OBJECT) parameters
((any-value type OBJECT)) slots ((value
type (LIKE (any-value self)) public? TRUE)
(rest type (CONS OF (LIKE (any-value self)))
public? TRUE initially NIL)))
class Cons public Standard_Object public
Object value Cons rest public virtual
int length() virtual Cons reverse()
virtual Object first() virtual Object
second() virtual Object third() ....
virtual boolean memberP(Object object)
26
Class Example 2

• STELLA
• C

(defclass PERSON (STANDARD-OBJECT
DYNAMIC-SLOTS-MIXIN) documentation ?The class
of people." slots ((name type STRING)
(age type INTEGER) (father type
PERSON) (mother type PERSON)
(siblings type (LIST OF PERSON))
(friends type (LIST OF PERSON) allocation
dynamic)))
class Person public Standard_Object,
public Dynamic_Slots_Mixin public char
name int age Person father Person
mother List siblings public virtual
Surrogate primary_type()
27
Programming to the ?Bare Metal?
STELLA
(defun (logor INTEGER) ((arg1 INTEGER) (arg2
INTEGER)) globally-inline? TRUE (return
(verbatim common-lisp (CLlogior arg1
arg2) cpp "(arg1 arg2)")))
java "(arg1 arg2)")))
C
int logor(int arg1, int arg2) return (arg1
arg2)
28
Type Inference
29
Explicit vs. Implicit Typing

• Globally visible signatures are explicitly typed
• Function/method parameters and return types
• Slot types
• Global variables
• Locally bound let variables are implicitly typed
• Type is derived from return type of initializer
• Type inference facilitated by parametric and
  anchored types
• Implicit types can be strengthened/weakend via
  explicit types
• NULL initializers must be explicitly typed
• Implicit typing supports automatic maintenance

30
Type Inference Example

• STELLA
• (defmethod (local-slots (LIST OF SLOT)) ((self
  CLASS))
• (return (class-local-slots self)))
• (defun (yield-attachments CONS) ((class CLASS)
  (classRef SYMBOL))
• (let ((attachmentTrees NIL))
• (foreach slot in (local-slots class)
• where (and (storage-slot? slot)
• (not (standard-dynamic-slot-a
  ccess? slot))
• (and (system-defined-slot-rea
  der? slot)
• (system-defined-slot-wri
  ter? slot)))
• collect (help-yield-attachments slot
  classRef)
• into attachmentTrees)
• (return attachmentTrees)))

31
Type Inference Example (cont).

• C
• Cons yield_attachments (Class r_Class, Symbol
  classref)
• Cons attachmenttrees NIL
• Slot slot NULL
• Cons iter_001 r_Class-local_slots()-the
  _cons_list
• Cons collect_001 NULL
• while (!nilP(iter_001))
• slot ((Slot)(iter_001-value))
• iter_001 iter_001-rest
• if (storage_slotP(slot)
• !standard_dynamic_slot_accessP((Storag
  e_Slot)slot)
• system_defined_slot_readerP((Storage_S
  lot)slot)
• system_defined_slot_writerP((Storage_S
  lot)slot))
• ...........

32
Type Inference Information Flow
(defclass CONS (STANDARD-OBJECT) parameters
((any-value type OBJECT)) slots ((value
type (LIKE (any-value self))) (rest type
(CONS OF (LIKE (any-value self))))) ...)
(defclass LIST (SEQUENCE) parameters
((any-value type OBJECT)) slots ((cons-list
type (CONS OF (LIKE (any-value self))))) ....)
(local-slots class)
(LIST OF SLOT)
(cons-list )
(CONS OF SLOT)
(value )
SLOT
33
Runtime Type Inference via typecase

• C?s v-tables make methods on OBJECT space
  prohibitive
• typecase can be used to avoid methods at the top
  of the hierarchy
• typecase can be used to implement multi-methods
• typecase can be used to write ?methods? without
  touching a class
• Supports working with heterogeneous collections
• (defun (evaluate-term OBJECT) ((self OBJECT))
• (typecase self
• (LITERAL-WRAPPER (return (evaluate-WRAPPED-LIT
  ERAL-term self)))
• (SURROGATE (return (evaluate-SURROGATE-term
  self)))
• (SYMBOL (return (evaluate-SYMBOL-term self)))
• (CONS (return (evaluate-CONS-term self)))
• (otherwise
• (error "Missing 'evaluate-term' method on "
  self))))

34
Program as Data
35
Backquote Example 1

• (defun (help-yield-attachments CONS) ((theSlot
  STORAGE-SLOT)
  (classRef SYMBOL))
• (return
• (bquote
• (let ((slot (lookup-slot classRef
  (quote (slot-name theSlot)))))
• (setf (slot-get-value-code slot)
• (the-code method
• (slot-owner theSlot)
• (yield-objectified-read-sl
  ot-name theSlot)))
• (setf (slot-set-value-code slot)
• (the-code method
• (slot-owner theSlot)
• (yield-setter-method-name
• (yield-objectified-read-s
  lot-name
• theSlot))))))))

36
Backquote Example 1 (cont.)

• Cons help_yield_attachments(Storage_Slot
  theslot, Symbol classref)
• return (listO(5, SYM_STELLA_LET,
• cons(listO(3, SYM_STELLA_SLOT,
• listO(3,
  SYM_STELLA_LOOKUP_SLOT,
• 
  classref,
• 
  cons(listO(3, SYM_STELLA_QUOTE,
• 
  theslot-slot_name,
• 
  NIL),
• 
  NIL)),
• NIL),
• NIL),
• listO(4, SYM_STELLA_SETF,
• listO(3,
  SYM_STELLA_SLOT_GET_VALUE_CODE,
• 
  SYM_STELLA_SLOT,
• NIL),
• listO(4,
  SYM_STELLA_THE_CODE,
• KWD_METHOD,
• 
  theslot-slot_owner,
• 
  cons(yield_objectified_read_slot_name

37
Backquote Example 2

• (defun (canonicalize-negation-tree OBJECT) ((tree
  CONS))
• Return a fully canonical tree in
  negation-normal-form.
• (let ((argument CONS (second tree))
• (nestedOperator SYMBOL (first argument)))
• (case nestedOperator
• (NOT double
  negation
• (free-cons-list tree)
• (return (canonicalize-proposition-tree
  (second argument))))
• ((AND OR)
  deMorgan's law
• (foreach it on (rest argument)
• do (setf (value it) (bquote (NOT
  (value it)))))
• (case nestedOperator
• (AND (setf (first argument) (quote OR)))
• (OR (setf (first argument) (quote
  AND)))))
• (EXISTS
• (let ((whereClause (extract-where-clause
  argument)))
• (setf (first argument) (quote FORALL))
• (setf (rest (last-cons argument))
• (bquote ((ALWAYS (NOT
  whereClause)))))))

38
Demons
39
STELLA?s Triggers (??Demons??)

• A demon can monitor
• Updates to a particular slot
• Updates to any (active) slot
• Instantiation of a particular class
• Instantiation of any (active) class.
• Interpreted slot accessors (get-value, put-value,
  drop-value) are used to program a demon.
• Only an ??active?? slot/class can have demons
• Non-active slots incur no overhead.
• Demons can be activated and deactivated at
  run-time.

40
Example ?Inverse Slot? Demon

• (defdemon inverse-slot-demon
• ((self STANDARD-OBJECT) (slot
  STORAGE-SLOT)
• (oldValue STANDARD-OBJECT) (newValue
  STANDARD-OBJECT))
• (let ((inverseSlot (inverse slot)))
• (when (defined? oldValue)
• (drop-slot-value oldValue inverseSlot
  self))
• (when (defined? newValue)
• (put-slot-value newValue inverseSlot
  self))))
• (defun (put-slot-value OBJECT)
• ((self STANDARD-OBJECT) (slot
  STORAGE-SLOT) (value OBJECT))
• (let ((code METHOD-CODE NULL)
• (oldValues LIST NULL))
• (cond ((single-valued? slot)
• (setq code (slot-set-value-code slot))
• (funcall code self value))
• (otherwise
• (setq code (slot-get-value-code slot))
• (setq oldValues (funcall code self))

41
Discussion
42
STELLA vs. Common Lisp/CLOS

• STELLA variations (mostly due to limitations of
  C and Java)
• Formal parameters are explicitly typed
• Uninitialized local variables are explicitly
  typed
• Definition of methods on top-level classes (e.g.,
  on OBJECT) is discouraged (use typecase instead)
• No multiple inheritance (except for mixin
  classes)
• Some STELLA statements do not return values
• case, cond, progn, let, foreach
• NULL (undefined), NIL (empty cons) and (quote
  NIL) are distinct
• Sometimes (not often) explicit casting is needed
• STELLA does not have
• full eval, lexical closure, multi-methods,
  call-next-method, optional and keyword arguments
• STELLA does not yet have
• format, logical pathnames, handling of native
  exceptions

43
STELLA vs. C

• Type system
• Less explicit typing due to automatic type
  inference
• Casts are seldom needed (translated C code
  contains lots of casts)
• Coercion of literals to/from objects is automatic
  and transparent
• Parameterized and anchored types (superior to C
  templates)
• Run-time type inference via typecase
• Control structure
• Versatile foreach loop (mimics Common Lisp?s
  loop)
• in, on, as, collect into, forall, exists, some
• Uniform syntax for function call, method call,
  and slot access
• Dynamically scoped variables (?specials?)
• Common-Lisp-style macros and backquote syntax
• Multiple return values (instead of reference
  parameters)
• startup-time-progn

44
STELLA vs. C (cont.)

• Object system
• Classes, slots, methods, globals, and modules are
  first-class objects
• Initial and default values for slots
• Dynamically-allocated slots (transparent accessor
  syntax)
• Narrowing of slot and method return types
• Constructor, initializer, terminator, destructor
• Triggers (?demons?)
• Class-specific print methods use dynamic dispatch
  ( ??
• NULL values for integer, float, and character
  types
• Program structure
• No header files
• Free placement of class and method declarations

45
Why a new language, why not extend C?

• Some STELLA features (e.g., iterators, symbols)
  could be implemented using libraries and C
  macros.
• Many STELLA features (critical for rapid
  prototyping) could not be implemented even with a
  powerful macro facility. Their implementation
  requires type inference and/or two-pass
  compilation
• Automatic casting and coercion
• No header files
• Free placement of method declarations.
• Many STELLA features (e.g., funcallable slots,
  default values, dynamic slot allocation, foreach
  loops) are implemented using the program-as-data
  paradigm (backquote). Implementing them directly
  in C is not impossible, just inordinately
  difficult.

46
Alternative Approaches

• Lisp in client/server architecture
• CORBA
• Lisp-to-C translation (e.g., Chestnut)
• Use C directly
• Use Java directly
• etc.

47
Rapid Prototyping

• Using the Lisp-based version of STELLA for
  program development has the following benefits
• Per function edit/compile/test cycle
• Dynamic class redefinition
• Dynamic method re/definition
• Can develop with a ?half-baked? system
• undefined functions
• tolerate some type conflicts
• Availability of powerful Lisp development
  environments
• Window-based inspector, debugger
• On the fly error correction
• Crossreferencer, ?edit-definition?,
  ?edit-callers?, etc.
• Source level STELLA stepper
• Strong typing seems to be a help most of the time

48
Conclusions

• STELLA is almost as easy to program in as Lisp
  (subjective).
• C output is efficient (executes 10-15 times
  faster than CLOS, 3-5 times faster than
  CL-struct object system)
• C applications are reasonably small (PowerLoom
  executable is about 3Meg - includes all of
  STELLA)
• STELLA is portable
• STELLA and PowerLoom have been compiled into
  Centerline C, g, JDK 1.2, Allegro CL,
  Harlequin LispWorks, and MCL.
• STELLA is written in STELLA (except for one C
  file and one Common Lisp file).

49
Conclusions, cont.

• The STELLA translator was worth building
• Implementing a STELLA-to-C translator has
  consumed about 3 person years.
• Upgrading C to support the implementation of
  PowerLoom would have taken more than 1.5 person
  years.
• We expect that, over the long term, our
  investment in translation technology will have
  paid for itself several times over.
• The notion of a strongly-typed Lisp makes sense.
• Automatic generation of efficient, readable C
  from STELLA is feasible.
• Designing a new programming language is hard!