DiSTiL : A Transformation Library for Data Structures

1
DiSTiL A Transformation Library for Data
Structures

• Yannis Smaragdakis

2
Overview

• DiSTiL is a project to add GenVoca Components to
  Microsofts IP
• Domain container data structures

3
Overview (Continued)

• Goal increase productivity by programming in
  high-level data structure abstractions

cursor
container
4
Overview (Continued)

• Create complex container data structures by
  composing DiSTiL components
• Design rule checking to ensure validity of
  component compositions
• Automatic selection of data structures according
  to retrieval predicates

5
Implementation Platform

• IP Transformation System (Microsoft
  Research)
• Handling of code with tree primitives
• User can define new language primitives
  (intentions) and write code to transform them at
  compilation (reduction) time

6
DiSTiL Library (Realms)

• Data Structures (arrays, linked lists, red-black
  trees, hash-tables)
• Storage (Persistent and Transient storage)
• Architectural (create code in functions)
• Special purpose layers (LRUTree)
• Various data-structure related layers (garbage
  collection, bound checks)
• Hidden layers (out-of-bounds, predicate, order)

7
DiSTiL Type Expressions

• Typex construct the composition of components
  (layers) that defines the target container data
  structure
• Huge number of distinct container data structures
  can be generated from compositions of DiSTiL
  components

8
Cursors

• Cursors define retrieval predicates over
  containers
• cursor(emp_cont, name lt An name gt Akers,
  no_order)
• Predicates used to generate efficient code for
  retrievals
• The smart part of the library

9
A Composite Data Structure
typex Example malloctransient
Keen 4789711
Jones 4719711
Lam 4709711
Mann 4799711
Guy 4769711
Koch 4749711
Ajit 4759711
Land 4739711
Smith 4729711
10
A Composite Data Structure
typex Example treemalloctransient
Keen 4789711
Jones 4719711
Lam 4709711
Mann 4799711
Guy 4769711
Koch 4749711
Ajit 4759711
Land 4739711
Smith 4729711
11
A Composite Data Structure
keyname
typex Example hash tree malloc
transient
Keen 4789711
Jones 4719711
Lam 4709711
Mann 4799711
Guy 4769711
Koch 4749711
Ajit 4759711
Land 4739711
Smith 4729711
12
How to Use
13
What to Notice

• Programming in domain-specific abstractions
• makes code easier to understand
• makes code more reliable (simpler and checked at
  a higher level)
• makes code easier to evolve (can modify typex
  without modifying program)
• should result in very good code

14
What to Notice

• DiSTiL actually
• can increase programmer productivity
  (simpler/shorter programs)
• removes burden of coding debugging data
  structures
• can accommodate architectural styles

15
Example Application

• LRU memory policy simulation

16
What is different about DiSTil?

• Different from static libraries (e.g. STL)
• meta-level optimizations (retrieval structure
  selection, updates)
• more advanced static checking (design-rule
  checking)
• level of programming significantly raised (what
  vs how)
• more general, powerful approach
• larger selection of structures/mechanisms

17
What is different about DiSTil?

• Even at the usual level of abstraction
• static libraries suffer from performance loss
• not easy to integrate such a variety of
  components
• example templates and memory allocation
• would have the added cost of a virtual dispatch
• static consistency checking is a domain-specific
  mechanism (type checking)

18
Internal organization
Static Library
Quote Package
Components
Predicate engine
Generated Application
Exported Symbols
User Application
19
Next Steps

• Improve design-rule checking
• Clarify design, possibly develop tools for
  writing generators
• Explore ideas (explicit, generation-time scoping)
  examined in the Quote package
• Broaden applications

20
Contributions of DiSTiL

• Example of factored libraries (Biggerstaff 1994)
• Primitive data structures factored in new ways
  (out-of-bounds checking, predicates on
  retrievals, inlined/functional implementation
  etc.)
• Extensive background for software generator
  development in non-scoped environments (Quote
  package)

21
Contributions (contd)

• Layered generator
• analysis of layer interaction
• possible mechanisms to facilitate construction
• determining pattern of information flow among
  layers