JTS: Tools for Implementing DomainSpecific Languages

1
JTS Tools for Implementing Domain-Specific
Languages

• D. Batory, B. Lofaso, Y. Smaradgakis
• Department of Computer Sciences
• University of Texas at Austin

2
Jakarta Tool Suite (JTS)

• Next generation tool suite for
• building scalable domain-specific component
  technologies and their generators
• GenVoca generators
• building extensible programming languages
• extensible Java
• automate software development
• OO design pattern transformations

3
The Jak Language

• open, extensible superset of Java
• examine base extension

4
Metaprogramming in Jak

• Extends Java with
• AST constructors
• represent and manipulate programs as data
• adds LISP backquote/comma to Java
• package for AST traversals, manipulation
• arbitrary program transformations
• plus other goodies...

5
AST Constructors

• Added constructors for code fragments of
  different types (expression, statement, )
• similar to LISP backquote

AST_Exp t exp 7 x8 exp AST_Stm s
stm if (ygt4) return r stm t.unparse( )
// outputs 7 x8 s.unparse( ) // outputs
if (ygt4) return r
6
Composing Code Fragments

• Explicit escapes
• LISP comma

AST_Exp t exp 7 gt x8 exp AST_Exp s
stm if (exp(t)) return r stm t.unparse( )
// outputs if (7gtx8)
return r
7
AST Constructors

• Focusing on AST constructors for Java/Jak now
• Add AST constructors for other languages
• CORBA IDL
• Embedded SQL
• (subsets of) C, C

8
AST Traversals

• Package/classes for depth-first searches,
  manipulations
• support arbitrary program transformations

AstCursor c new AstCursor( ) Ast_Node root
// root of AST to search for (c.First(root)
c.More( ) c.PlusPlus( ) ) if (c.node
instanceof AstInterface) c.Delete( )
9
How JTS Works
10
SSTs versus ASTs

• Surface syntax tree (SST) is a syntactically
  correct parse tree
• tree constructors, modifiers guarantee syntactic
  correctness
• SSTs may not be semantically correct
• Abstract syntax tree (AST) is a type-checked SST
• with annotations to symbol table
• invoked by typecheck( ) method to root of SST

11
Extensibility of Jak

• Microsoft IP Intentions
• new grammar rules introduce new AST nodes with
  domain-specific semantics
• P2 cursor, container types, operations
• at reduction time, intention nodes are replaced
  with host language implementations
• P2 replaced abstract cursor, container AST nodes
  with their concrete C implementations
• Jak/Java inherently open compiler

12
JTS Follows DSL Paradigm
13
How to...

• Produce lexers and parsers?
• Produce transform program?
• Where does GenVoca fit in?answers in remainder
  of talk...

14
Bali

• GenVoca Generator of
• DSL Precompilers

15
Family Languages

• Jak is a family of related languages
• versions with/without AST constructors
• with/without CORBA IDL extensions
• Classic library scalability problem
• n features with 2n combinations
• cannot build all 2n combinations by hand
• can generate them

16
Bali is a GenVoca Generator

• Assembles variants of Jak from components
• Components encapsulate primitive extensions
• AST constructors
• domain-specific generators like P3
• CORBA IDL
• Compositions of components specifies particular
  variant

17
Two Aspects

• Tool for writing compilers for Domain-Specific
  Languages (DSL)
• looks similar to other DSL-compiler tools
• specify syntax of DSL or language extension using
  annotated, extended BNF grammars
• GenVoca generator
• software component technology
• architectural, extensibility aspects

18
Bali Grammars

• Extended BNF grammar
• extended to handle repetitions
• e.g., POPART

StatementList ( Statement )
ArgumentList Argument ( , Argument )

19
Bali Grammars are Annotated

• by class to instantiate when production is
  recognized
• POPART, DIALECT, common OO design techniques

SelectionStmt IF ( Expr ) Statement
IfStm SWITCH ( Expr ) Block
SwitchStm
20
Bali Specifications

• Jak grammar
• 160 tokens
• 240 productions
• 800 lines
• Bali grammar
• 15 tokens
• 20 productions
• 73 lines

// bali spec lexical patterns grammar rules
21
What can be generated...

• Lexical analyzer
• Jlex (java version of lex)
• Bernie Lofaso (Jakarta Team Member)
• Parser
• Java_Cup (java version of yacc)
• Scott Hudson_at_Georgia Tech
• Inheritance hierarchy and AST classes
• constructors, unparsing, editing methods

22
What cant be generated...

• Type checking, reduction, optimization methods
• AST node specific
• hand code subclasses to Bali-generated class

23
Big Picture

• Traditional approach to building compilers for
  domain-specific languages
• clean model of proven technology
• Problem dont want to hand-code and
  hand-assemble each variant
• too expensive
• Solution GenVoca component technology
• components encapsulate primitive extensions
• compose components to form variants of Jakarta

24
GenVoca

• Model of parameterized programming
• addresses the architectural aspects of software
• building blocks of domains are refinements of
  fundamental domain abstractions
• components are implementations of refinements
• components composed by parameter instantiation
• visualize component composition as stacking layers

25
Quick Tutorial

• Software system is named composition of
  components called a type equation

System
System
Question what is relationship of components to
classes?
26
Key Idea Layering and Inheritance

• Inheritance hierarchy is an inverted layering
  hierarchy
• calling super classes same as calling lower layers

27
Relationship to GenVoca

• GenVoca components are large scale
• consistent refinements of multiple classes

28
Relationship to Jak
AstNode
T Ast With Java Kernel
29
Conclusions

• JTS integrates different technologies
• Java
• metaprogramming
• domain-specific language compiler tools
• architectures components/generators
• Powerful tool suite for
• building generators/precompilers for embedded,
  non-embedded languages
• used it to build P3 (data structures generator)
• JTS itself (bootstrapping)

30
The End
31

• intro to Bali 7.5 minutes
• remaining 13 minutes