The Invasive Software Composition System COMPOST

1
The Invasive Software Composition System COMPOST

• Dr. Uwe Aßmann
• Institut für Programmstrukturen
• Universität Karlsruhe
• Andreas Ludwig Rainer Neumann
• with contributions of
• Thomas Genssler Dirk Heuzeroth Ralf Reussner
  Volker Kuttruf

2
Overview

• Introduction
• Well-known Problems in System Construction
• Invasive Software Composition
• Component Model Boxes and Hooks
• Composition Technique Program Transformation
• Composition Language
• Results of Invasive Software Composition
• The COMPOST system
• Advantages
• Visions

3
Decomposition and SawMill Linux

• Decomposition is the most important design
  principle
• Mechanical Engineering (VDI 2221)
• Universal Design Theory
• Decomposition builds on components/modules

4
Where are We Today?

• Component systems (CORBA, DCOM, Beans,
• OO-frameworks) provide a communication and
  standard service infrastructure.
• Todays component systems support modularity only
  in one dimension
• No Extension
• No Aspect Separation
• No Scalable and Efficient Communication

5
1.1. Extensibility (here Compilers)

• CoSy is a modular component framework for
  compiler construction Alt/Aßmann/vanSomeren94
• Built in 90-95 in Esprit Project COMPARE
• Sucessfully marketed by ACE bV, Amsterdam
• Goal extensible, easily configurable compilers
• Extensions without changing other components
• Plugging from binary components without
  recompilations
• New compilers within half an hour

6
Optimizer I
Optimizer II
Parser
Generated Factory
Logical view
Generated access layer
7
CoSy Repository-Architecture
Transformation
Semantics
Parser
Optimizer
Lexer
Codegen
Blackboard
8
Extension with Views
Optimizer I
Parser
Optimizer II
9
O-O Technology doesnt fit
Optimizer I
Optimizer II
K
K
Parser
K''
K
K'''
K
Optimizer III
K'
K''''
K K' K''..
Objects have to be allocated by the parser in
base class format Fragile Base Class Extension
Problem IBM-SOM Schema Evolution in OODB OBST
10
CoSy Solution Extension Operators for Classes
K
Physical Layout is a merge of the logical views
K

K

Every component keeps its logical view
11
View Mapping Layer
Optimizer I
Optimizer II
Parser
Generated Factory
Logical view
Generated access layer
12
Advantages of CoSy

• Access level MUST be efficient yes
• Due to views, Cosy compilers can be extended
  easily
• Companies reduce costs (e.g. when migrating to a
  new chip) by improved reuse

Is there a general solution to the extensibility
problem?
13
1.2. Mixing in Code
m () abc.. cde.. return
m () print("enter m") abc..
cde.. print("exit m") return
Method.entry
Method.entry
Method.exit
Method.exit
How to automize?
14
Aspect Separation

• Aspekt-orientierted Programming (AOP)
• Kiczales et al, Xerox Parc. ECOOP 97
• Aspect separation is a fundamental
  interdisciplinary design principle
• should contribute to a Universal Design Theory
  Goos/Aßmann98

15
Structure
Electricity
Water
Gas
Integrated building
16
Debugging aspect
Algorithm
Persistence aspect
Weaver-Tool
Debugging aspect
Persistence aspect
Debugging aspect
Persistence aspect
Debugging aspect
17
Advantages of Aspects
"Es wächst zusammen, was zusammen gehört" Willy
Brandt

• Locally concentrated specifications allow to
  group correlated things (concern concentration)
• Weavers generate efficient programs
• Easy extension

Problem too many aspect languages
18
Subject-Oriented Programming (IBM)

• Was earlier
• Composition rules go further than AOP
• very general composition semantics
• based on C/Java
• simple
• Instead on aspect languages, SOP concentrates on
  composition languages
• Weavers can be written yourself
• Hyperspaces are even more systematic, and model
  cross-cutting/extension very well

19
1.3. Scalable Efficient Communication
m () // Events notifyObservers(d)
e listen_to()
m () // Call e p(d)
??
20
Architecture Systems

• Architectural systems propose separation of
  architectural aspect by
• decoupling architectural and application code.

Interface
Port
Component
Connector
Component
21
Ports and Connectors
22
Evaluation of Architecture Languages

• Architecture Languages are a research niche
• Interfaces slow down systems
• Delegation/Inheritance/Genericity
• Leads to indirections
• Macropreprocessing insecure

How do we combine abstract architecture and
efficient connections?
23
Questions for Improved Composition

• How to extend components flexibly (without
  expensive adapter levels)?
• How to mix aspects into components without
  special systems?
• How to model architecture but make efficient
  systems?

24
2. Invasive Software Composition
25
Software Composition
Component Model
Composition Technique
Composition Language
26
Invasive Software Composition
Invasive Composition adapts and extends box
components at hooks by program transformation

• View-oriented development
• Aspect-oriented development

27
2.1 Component Model of Invasive Composition

• The basic element is a box, a set of program
  elements
• may be the representation
• of a component
• of an aspect
• of a part of a component
• a class
• a package
• a method

28
Boxes have Hooks
Hooks are arbitrary program elements of the
representation of a box which are subject to
change

• Examples
• Method Entries
• Communications/calls
• Generic Parameters
• Generic Statements

29
2.1.1. Implicit Hooks

• Example Method Entry/Exit

m () abc.. cde..
Method.entry
Method.exit
30
2.1.2. Declared Hooks
Declared Hooks are declared by the box writer as
variables in the hooks code. All declared hooks
form the Composition Interface of the box.
Declarations
31
Declaration of hooks

• Language Extensions (keywords..)
• Standardized Names
• Inheritance Relations
• Comment Tags

Class Set extends genericXSuperClass class
Set / _at_superClass /
X SuperClass
32
2.3 Composition Technique

• Invasive Composition
• transforms boxes
• during the embedding into a reuse context
• by program transformations of hooks.
• A composer is a program transformer from unbound
  to bound hooks
• composer box with hooks ? box with code

33
Method Entry
Method Entry
Method Exit
Method Exit
m () abc.. cde..
m () print(enter m) abc.. cde..
Print(exit m)
Method Exit
methodEntryHook.extend(print(\enter
m\)) methodExitHook.extend(print(\enter
m\))
34
Invasive Composition as Program Transformations
Composer
Invasively transformed code
35
Library
Client
Invasive composition
Black box composition
Subsystem
Client
Library
Client
Library
Invasive connection
Black box connection with glue code
36
Invasive Connection with CORBA
import org.omg.CORBA. import Library public
class Client extends CORBA.client public
order(String name) // Initialize CORBA
Broker ORB orb ORB.init(args,new
Properties()) // Get the seller Library
farAway orb.string_to_object(name)
// Order farAway.schaueNach()
farAway.order()
import Library public class Client public
order(String serverName) // Get the
seller Library library
getLibrary() // Order
library.selectIt() library.buy()
hooks
37
Invasive Composition Removing Composition
Interfaces
Functional Interface
Composition Interface with Hooks
Invasive Composition
38
2.3 The Composition Language
39
The Global Picture of the Invasive Composition
Process
Box components and hooks
Composed Architecture
Composition programs
40
Compositions as Static Meta-programs
Client.coc (Javahooks)
Server.coc (Javahooks)
Composition Program in Java and COMPOST
Composition/ Configuration
Client.java
Server.java
Glue.java
Compiler
Compilation
41
All hooks of one component bound
System with components and hooks
More things bound and connected
Composed system
42
3. Results of the Invasive Composition Technique
43
Principle of Invasive Composition

• Every box encapsulates a design decision of the
  application, hiding it behind a
• composition interface which contains well-defined
  hooks which can be recognized and manipulated by
  composers
• Parnas principle driven to the max!

44
Information Hiding

• If only the composition interface is used for
  composition,
• encapsulates this interface the box such that it
  can be exchanged to variants.

Much more flexible as modules, frameworks,
aspects!
45
Other Use Cases for Invasive Composers

• Multiple inheritance
• Mixin-based inheritance
• Intrusive data functors
• Generic type parameters
• Generic program elements
• Refactorings
• Contract-checkers
• Automated design patterns
• Connectors

46
Composers Generalize Connectors

• boxes composers hooks
• boxes connectors ports

47
Hooks for Communications (Ports)

• Can be declared by calls to standard methods (as
  in Linda)

m () // Aufruf e p(d)
m () out(d) in(e)
Output port
Input port
m () // Ereignis notifyObservers(d)
e listen_to()
48
Inheritance vs. Delegation
Library
Client
CORBA-Connector
DCOM-Connector
Client
Library
Client
Library
CORBA-Connection
DCOM-Connection
49
Invasive Extension for CORBA
import org.omg.CORBA. import Library public
class Client extends CORBA.client public
order(String name) // Initialize CORBA
Broker ORB orb ORB.init(args,new
Properties()) // Get the seller Library
farAway orb.string_to_object(name)
// Order farAway.schaueNach()
farAway.order()
import Library public class Client public
order(String serverName) // Get the
seller Library library
getLibrary() // Order
library.selectIt() library.buy()
hooks
50
Inheritance and Delegation are Composers
K
Extend invasively
Inherit
Delegate
K-private
K
K
K-subclass
Invasive Komposition kann invasive Einbettung,
Vererbung oder Delegation einsetzen
51
When Use What?

• Deploy Inheritance
• for consistent side-effect free composition
• Deploy Delegation
• für dynamischen Austausch
• Deploy Invasive Extension
• for non-foreseen extensions (Views, Problem 1)
• to develop aspect-orientedly (Invasive Mixing,
  Problem 2)
• to adapt without delegation (Invasive Adaptation
  for Communication, Problem 3)

52
Example Automaton Extensions
processorFree
Ready
Running
deviceReady
requestDevice
Waiting
paketExpected
paketComesIn
HttpWaiting
53
Example Automaton Extensions
processorFree
Ready
Running
deviceReady
requestDevice
Waiting
paketExpected
paketComesIn
HttpWaiting
54
Schedule () int state loop () switch
(state) / _at_stateSetHook / case
Running if (event requestDevice)
state Waiting continue /
_at_RunningStateHook / if (event
PaketExpected) state HttpWaiting
continue case Waiting if (event
deviceReady) state Ready
continue / _at_WaitingStateHook /
case HttpWaiting if (event
paketComesIn) state Ready
continue / _at_HttpWaitingStateHook
/ ...
Schedule () int state loop () switch
(state) / _at_stateSetHook / case Running
if (event requestDevice)
state Waiting continue /
_at_RunningStateHook / case Waiting
if (event deviceReady) state
Ready continue / _at_WaitingStateHook
/ ......
S findHook(stateSetHook) R
findHook(RunningStateHook) R.extend(if
(...) S.extend(case ....)
55
ReadyStateHook
WaitingStateHook
RunningStateHook
ReadyStateHook
WaitingStateHook
RunningStateHook
HttpWaitingStateHook
56
The COMPOST System
57
The COMPOsition SyTem

• COMPOST is the first system to support invasive
  software composition
• Library of meta-programs, also connectors
• Further meta-level information
• Boxes provide very general components
• Hooks generalize ports
• Names Types (for globally correct replacements)
• Design Patterns
• Refactorings
• Contracts
• Inheritance operators

58
Implementation of Composers in COMPOST

• Components are sets of Java program elements
• Composers are
• standard Java methods
• and reified command objects
• Meta-programming (reflection und transformation)
  serves to
• match hooks, manipulate them, and generate glue
  code (internal and external adaptation)
• statically with a meta-model
• removal of superfluous interfaces
• no runtime overhead
• Components, composers, hooks, ports exist as
  meta-objects, i.e. classes, in COMPOST, i.e.
  there are classes Hook, Component, Composer.

59
The COMPOST Architecture
Wizards
AspectLanguages
Reengineering
Connectors
Language Add-Ons
Design Patterns
Inheritance
Refactorings
Boxes and Hooks
Prettyprinter
Type Information
Components
Database
Meta-Object Protocol (AST)
60
COMPOST for Everybody

• 0.5alpha is out
• parsers pretty printers, name analysis service
• transformations basic patterns, views,
  inheritance
• papers
• http//i44www.info.uni-karlsruhe.de/compost.html
• What to do
• write language adapters (C, C,..)
• extend COMPOST with new composers, boxes and
  hooks
• apply it to adapt your code!

61
Results and Progress
62
Results Answers to the Questions

• View-based Extensions
• Aspect-based development
• Scalable and efficient architecture von
  Komponenten
• Modularity on higher level

63
Advantages

• Composers are standard programs
• Standard tooling available
• Standard software process management
• Open systems
• Scalable systems by simple configuration
• Composers are boxes
• can be composed themselves
• uniformity
• connector reuse from libraries
• Composition programs
• are component-oriented
• can be composed themselves

64
Composition Language Level
Composition Language
Composition Technique for Composition Recipes
Composition Language for Composition Recipes
Component Model of Composition Language
Composition System Level
Composition System
Composition Technique
Composition Recipe
Component Model
65
Progress

• Method is language independent
• Apply techniques from Compilers
• Supports reengineering
• Supports product families
• Invasive composition programs are the assembler,
  what is the HLL?

66
Improvements...

• ...object-oriented systems
• Composers generalize inheritance, delegations,
  and generics
• Invasive Composition removes superfluous
  interfaces
• Views
• ...aspect-oriented programming
• Invasive Composition does not need special
  languages, but supports them
• Invasive Composition systematizes weaving
• Graph rewritings describe weavings

67
(No Transcript)
68
Subject-orientierted Programming
EJB
Aspect-oriented Programming
Invasive Composition
Architectural languages
Corba DCOM Beans
SOP
lN-calculus
Template MP
Standard Composition
69
Sound Composition

• Compositions are sound if
• they are side-effect free
• they extend feature groups Stata/Guttag95
• mix orthogonal code (without dependencies)
  Aßmann98 LambdaN-Kalkül, Dami 97
• mix in code which is only flow dependent on the
  old code
• All these criteria can be checked (by program
  analysis)

70
Visions
71
Visions

• Box components are modules (classes) plus a
  notion of composition interfaces that facilitate
  adaption and composition.
• Hooks generalize Ports for a uniform composition
  mechanism
• Composition libraries will become the foundation
  of a new software industrial branch.
• Meta-models replace languages?!

72
At the Transition to the COMPOSiTion Age

• XML will be the ASCII of the 21st century
• concrete syntax will vanish
• We need composition, not components
• Composition will substitute programming
• Invasive composition is required
• Forget about the language wars everything can be
  composed which has abstract syntax
• Become a developer of composition techniqes!

73
THE END

• More tomorrow on COMPOSTs architecture

74
Relation to Other Works

• Composers are higher-order functions, expanded at
  compile time (functional programming)
• Composers are program transformers (theory on
  program transformation)
• Composers can be derived from design patterns
  (software engineering)
• Composers offer more merge operators than the
  LambdaN calculus of Dami

75
Publications

• http//i44www.info.uni-karlsruhe.de/ compost
• Alt/Aßmann/vanSomeren94 Alt, M., Aßmann, U.,
  van Someren, H. Cosy compiler phase embedding
  with the CoSy compiler system. CC94, LNCS 786
• Aßmann95a Aßmann, U. On edge eddition rewrite
  systems and their relevance to program analysis.
  Graph-grammar Conference 1994, LNCS 1073
• Aßmann95b Aßmann, U. Generierung von
  Programmoptimierungen mit Graphersetzungssystemen.
  Dissertation. Universität Karlsruhe,
  GMD-Berichte 262, Oldenbourg.
• Aßmann96 Aßmann, U. How To Uniformly Specify
  Program Analysis and Transformation. CC96, LNCS
  1060
• Goos/Aßmann98 Goos, G., Aßmann, U. Systematic
  Software Construction. Workshop Universal Design
  Theory, Karlsruhe, Shaker Verlag.
• Aßmann98 Aßmann, U. Meta-programming composers
  in 2nd generation component systems. IFIP WG 2.4
  Systems Implementation, Feb. 98, Berlin.
• Aßmann99 Aßmann, Ludwig How to introduce
  connections into classes with static
  metaprogramming. Coordination 99, Amsterdam.
  LNCS.

76
Publications

• Aßmann00 Aßmann, U., Genssler, T., Bär, H.
  Meta-programming grey-box connectors. TOOLS
  Europe 2000.
• Aßmann99b Aßmann, U. How to transform and
  optimize programs with OPTIMIX. Graph-grammar
  handbook, Vol. II, ed Rozenberg, Kreowski 1999
• Aßmann99c Aßmann, U. Graph rewrite systems for
  program optimization. Under revision by TOPLAS.
• Aßmann99d Aßmann, U, Ludwig, A. Aspect weaving
  with Graph Rewriting. 1st International Symposium
  on Generative and Component-based Programming
  (GCSE). LNCS.

77
Literature

• Other works at our institute
• Zimmer, W. Frameworks und Entwurfsmuster.
  Dissertation. Jan. 1997, Universität Karlsruhe.
• Schulz, B., Genßler, T., Mohr, B., Zimmer, W. On
  the Computer Aided Introduction of Design
  Patterns into Object-Oriented Systems,
  Proceedings of the 27th TOOLS, Sept. 1998, IEEE
  CS Press.
• Ludwig, A. Behandlung von partieller Konformität
  bei polymorphen Methodenaufrufen. Diplomarbeit
  Universität Karlsruhe, 1997.
• Frigo, J., Neumann, R., Zimmermann, W. On the
  Construction of Robust Class Hierarchies. IFE 96.
• Other works
• Berzins94 Berzins, V. Software Merge Semantics
  of Combining Changes to Programs. TOPLAS Nov. 94
• Dami97 Dami, L. Software Composition.
  Dissertation Universität Genf. 1997