Existential Types and Module Systems

1
Existential Types and Module Systems

• Xiaoheng Ji
• Department of Computing and Software
• March 19, 2004

2
Data Abstraction

• The fundamental idea the separation of the
  client from the implementor of the abstraction by
  an interface.
• The central ideas
• Define a representation type together with
  operations that manipulate values of that type.
• Hold the type representation type abstract from
  client of the ADT to ensure representation
  independence.
• Abstract types have existential type''
• Mitchell and
  Plotkin 84
• Existential types provide the fundamental
  linguistic mechanisms for defining interfaces,
  implementing them, and using the implementation
  in client code.
• Existential types are a kind of type originally
  developed in constructive logic.

3
Existential Types

• Two different ways of looking at an existential
  type, written ?X, T or ?X.T
• logical intuition an element of ?X, T is a
  value of type XgtST, for some type S.
• Operational intuition an element of ?X, T is a
  pair, written S, t, of a type S and a term t
  of type X?ST.
• Instead of ?X.T, the nonstandard notation ?X, T
  suggests that the existential value is a mixed
  type-value tuple.

4
Existential Types - Overview

• Typing and evaluation rules
• Introducing ADTs
• Introducing ADTs
• Introducing objects
• Objects vs. ADTs
• Encoding existential types

5
Existential introduction

• an existentially typed value introduced by
  pairing a type with a term S,t
• intuition value S,t of type ?X,T is a
  module with a type component S and a term
  component t, where S/XT.
• The type S is often called the hidden
  representation type, or sometimes the witness
  type of the package.

6
Example

• p Nat, a 0, f ?x Int.succ(x) as
  ?X.a X, f X-gtX
• p ?X.a X, f X-gtX
• q Nat, a 0, f ?x Int.succ(x) as
  ?X.a X, f X-gtNat
• q ?X.a X, f X-gtX
• Nat type component
• a 0, f ?x Int.succ(x) term component
• as ?X.a X, f X?Nat type annotation

7
Typing rule for existential introduction

• A value of type ?X.T is a package with a witness
  type T for X and a value term t X -gt T'T.
• pack X T' with t as T ?X.T
  (conventional notation)
• T', t as ?X, T
  (Pierce's notation)
• The Introduction typing rule for existential
  types

? - t1 X gt UT1
(T- PACK)
? - U, t1 as ?X, T1 ?X, T1
8
Examples of Existential types

• Nat, 0 as ?X, X ?X, X
• Bool, true as ?X, X ?X, X
• p Nat, a 0, f ?x Nat. succ x as ?X,
  a X, f X -gt Nat
• p ?X, a X, f X -gt Nat
• q Bool, a true, f ?x Bool. 0 as ?X,
  a X, f X -gt Nat
• q ?X, a X, f X -gt Nat
• The type part is hidden (opaque, abstract), and
  the value
• part provides an interface for interpreting the
  hidden type.

9
Unpacking existential values

• Unpacking an existential let X,x t1 in t2

? - t1 ?X, T1
?, X, x T1 - t2 T2
(T- UNPACK)
? - let X,x t1 in t2 T2

• Type variable X cannot occur in T2 -- it is not
  in scope (i.e. doesn't appear in the context ?).
  This means that the name X of the existential
  witness type cannot "escape from the let
  expression.
• Also, within the body t2, the type X is abstract
  and can only be used through the interface
  provided by x T1.

10
Example

• p Nat, a 0, f ?x Nat. succ x as ?X,
  a X, f X -gt Nat
• p ?X, a X, f X -gt Nat
• The elimination expression
• let X, x p in (x.f x.a)
• ? 1 Nat
• opens p and uses the fields of its body (x.f and
  x.a) to compute a numeric result.

11
Example

• p Nat, a 0, f ?x Nat. succ x as ?X,
  a X, f X -gt Nat
• p ?X, a X, f X -gt Nat
• The body of the elimination form can also involve
  the type variable x
• let X, x p in (?yX. (x.f y)) x.a
• ? 1 Nat
• The fact that the packages representation type
  is held abstract during the typechecking of the
  body means that the only operations allowed on x
  are those warranted by its abstract type' a
  X, f X-gtNat.

12
Example

• p Nat, a 0, f ?x Nat. succ x as ?X,
  a X, f X -gt Nat
• p ?X, a X, f X -gt Nat
• all operations on term x must be warranted by its
  abstract type, e.g. we cannot use x.a concretely
  as a number (since the concrete type of the
  module is hidden)
• let X, x p in succ(x.a)
• ? Error argument of succ is not a number

13
Example

• p Nat, a 0, f ?x Nat. succ x as ?X,
  a X, f X -gt Nat
• p ?X, a X, f X -gt Nat
• In the rule T-UNPACK, the type variable X appears
  in the context in which t2s type is calculated,
  but does not appear in the context of the rules
  conclusion. This means that the result type T2
  cannot contain X free, since any free occurrences
  of X will be out of scope in the conclusion.
• let X, x p in x.a
• ? Error Scoping error!
• must add side condition to typing rule for
  existential elimination X may not occur in the
  result type

14
Evaluation rule for existentials

• let X, x (T11, v12 as T1) in t2
• ? X-gtT11 x-gtv12 t2

(E- UNPACKPACK)

• If the first subexpression of the let has already
  been reduced to a concrete package, then we may
  substitute the components of this package for the
  variables X and x in the body t2.
• In terms of analogy with modules, this rule can
  be viewed as a linking step, in which symbolic
  names (X and x) referring to the components of a
  separately compiled module are replaced by the
  actual contents of the module.
• Since the type variable X is substituted away by
  this rule, the resulting program actually has
  concrete access to the packages internals.

15
Existential Types - Overview

• Typing and evaluation rules
• Introducing ADTs
• Introducing ADTs
• Introducing objects
• Objects vs. ADTs
• Encoding existential types

16
Parametricity

• consider
• p Nat, a 0, f ?x Nat. 0 as ?X,
  a X, f X -gt Nat
• q Bool, a false, f ?x Bool. 0 as
  ?X, a X, f X -gt Nat
• evaluation does not depend on the specific type
  of p and q it is parametric in X
• let X, x p in (x.f x.a)
• ? 0
• let X, x q in (x.f x.a)
• ? 0
• Idea use parametricity to construct two kinds of
  programmer defined abstractions abstract data
  types (ADTs) and objects

17
Existential Types - Overview

• Typing and evaluation rules
• Introducing ADTs
• Introducing ADTs
• Introducing objects
• Objects vs. ADTs
• Encoding existential types

18
Abstract Data Types

• A conventional abstract data type (or ADT)
  consists of
• A type name A
• A concrete representation type T
• Implementations of some operations for creating,
  querying, and manipulating values of type T
• An abstraction boundary enclosing the
  representation and operations

Inside this boundary, elements of the type are
viewed concretely (with type T). Outside, they
are viewed abstractly, with type A. Values of
type A may be passed around, stored in data
structures, etc., but not directly examined or
changed the only operations allowed on A are
those provided by the ADT.
19
Example

• signature COUNTER
• sig
• type counter
• val new counter
• val get counter -gt Nat
• val inc counter -gt counter
• end
• abstract representation type
• concrete representation type
• interface
• implementation

structure Counter gt COUNTER struct type
counter Nat val new 1 fun get(n) n
fun inc(n) n 1 end
20
Example

• signature COUNTER
• sig
• type counter
• val new counter
• val get counter -gt Nat
• val inc counter -gt counter
• end

structure Counter gt COUNTER struct type
counter Nat val new 1 fun get(n) n
fun inc(n) n 1 end

- counter.get ( counter.inc counter.new) val it
2 Nat
21
ADTs as existentials

• signature COUNTER
• sig
• type counter
• val new counter
• val get counter -gt Nat
• val inc counter -gt counter
• end

structure Counter gt COUNTER struct type
counter Nat val new 1 fun get(n) n
fun inc(n) n 1 end
counterADT Nat, new 1, get(n)
?n Nat. n inc(n) ? n Nat. succ(n) As
COUNTER
COUNTER ? Counter. new Counter, get
Counter -gt Nat, inc Counter -gt Counter

• Abstract types have existential type'
  Mitchell and Plotkin 84

22
ADTs as existentials
CounterADT Nat, new 1, get(n)
?n Nat. n inc(n) ? n Nat. succ(n) As
COUNTER
COUNTER ? Counter. new Counter, get
Counter -gt Nat, inc Counter -gt Counter
let Counter, counter CounterADT
in counter.get ( counter.inc counter.new) ? 2
Nat

• type name Counter can be used just like a new
  base type
• e.g. we can define new ADTs with representation
  type Counter,
• e.g. a FlipFlop

23
Flip-Flop

• Let Counter, counter counterADT in
• Let FlipFlop, flipflop
• Counter,
• new counter.new,
• read ?c Counter. iseven (counter.get c),
• toggle ?c Counter. counter.inc c,
• reset ?c Counter. counter.new
• As ?FlipFlop,
• new FlipFlop, read FlipFlop-gtBool,
• toggle FlipFlop-gtFlipFlop, reset
  FlipFlop-gtFlipFlop in
• flipflop.read (flipflop.toggle (flipflop.toggle
  flipflop.new))
• ? false Bool

24
Representation independence

• Alternative implementation of the CounterADT
• counterADT
• xNat,
• new x1,
• get ?i xNat. i.x,
• inc ?i xNat. xsucc(i, x)
• as ?Counter,
• new Counter, get Counter-gtNat, inc
  Counter-gtCounter
• ?counterADT ?Counter,
• new Counter, get Counter-gtNat, inc
  Counter-gtCounter
• Representation independence follows from
  parametricity the whole program remains typesafe
  since the counter instances cannot be accessed
  except using ADT operations

25
ADT-style of programming

• yields huge improvements in robustness and
  maintainability of large systems
• limits the scope of changes to the program
• encourages the programmer to limit the
  dependencies between parts of the program (by
  making the signatures of the ADTs as small as
  possible)
• forces programmers to think about designing
  abstractions

26
Existential Types - Overview

• Typing and evaluation rules
• Introducing ADTs
• Introducing ADTs
• Introducing objects
• Objects vs. ADTs
• Encoding existential types

27
Existential objects

• two basic components internal state, methods to
  manipulate the state.
• e.g., a counter object holding the value 5 might
  be written
• c Nat,
• state 5,
• methods get ?x Nat. x,
• inc ?x Nat. succ(x)
• as Counter
• where
• Counter ?X, stateX, methods get
  X-gtNat, inc X-gtX

28
Invoking the get method

• c Nat,
• state 5,
• methods get ?x Nat. x,
• inc ?x Nat. succ(x)
• as ?X,
• state X,
• methods
• get X-gtNat,
• inc X-gtX
• let X, body c in body.methods.get
  (body.state)
• ? 5 Nat

29
Encapsulating the get method

• C ?X,
• state X,
• methods
• get X-gtNat,
• inc X-gtX
• sendget ?c Counter.
• let X, body c in
• body.methods.get (body.state)
• ? sendget Counter -gt Nat

30
Invoking the inc method

• c Nat,
• state 5,
• methods get ?x Nat. x,
• inc ?x Nat. succ(x)
• as ?X.
• state X,
• methods
• get X-gtNat,
• inc X-gtX
• let X, body c in
  body.methods.inc (body.state)
• ? Error scoping error
• Why? X appears free in the body of the let

31
Encapsulating the inc method

• in order to properly invoke the inc method, we
  must repackage the fresh internal state as a
  counter object
• c1 let X, body c in
• X,
• state body.methods.inc (body.state),
• methods body.methods
• as Counter
• sendinc ?c Counter.
• let X, body c in
• X,
• state body.methods.inc (body.state),
• methods body.methods
• as Counter
• ? sendinc Counter -gt Counter

32
Example

• More complex operations on counters can be
  implemented in terms of these two basic
  operations
• add ?c Counter. sendinc (sendinc
  (sendinc c))
• add Counter -gt Counter

33
Existential Types - Overview

• Typing and evaluation rules
• Introducing ADTs
• Introducing ADTs
• Introducing objects
• Objects vs. ADTs
• Encoding existential types

34
Abstract type of counters

• ADT-style counter values are elements of the
  underlying representation (i.e. simple numbers of
  type Nat)
• object-style each counter is a whole module,
  including not only the internal representation
  but also the methods. Type Counter stands for the
  whole existential type
• ?X.
• state X,
• methods
• get X-gtNat,
• inc X-gtX

35
Stylistic advantages

• advantage of the object-style since each object
  chooses its own representation and operations,
  different implementations of the same object can
  be freely intermixed
• advantage of the ADT-style binary operations
  (i.e. operations that accept gt 2 arguments of
  the abstract type) can be implemented, contrary
  to objects

36
Binary operations and the object-style

• e.g. set objects type
• NatSet ?X, state X, methods empty X,
• singleton Nat-gtX,
• member X-gtNat-gtBool,
• union X-gtNatSet-gtX
• cannot implement the method since it can have no
  access to the concrete representation of the
  second argument
• in reality, mainstream OO languages such as C
  and Java have a hybrid object model that allows
  binary operations (with the cost of restricting
  type equivalence)

37
Existential Types - Overview

• Typing and evaluation rules
• Introducing ADTs
• Introducing ADTs
• Introducing objects
• Objects vs. ADTs
• Encoding existential types

38
Duality

• universal types ?X.T is a value of type S/XT
  for all types S.
• existential types ?X.T is a value of type S/XT
  for some type S.
• idea exploit duality to encode existential types
  using universal types, using the equality
• ?X.T ?X.T

def
39
Encoding

• encoding existential types using universal types
• ?X,T ?Y. (?X. T-gtY) -gt Y.
• operational view a module is a value that gets a
  result type and a continuation, then calls the
  continuation to yield the final result
• A continuation is something that we can call and
  forget about because it does not return control
  to the caller.

def
40
Encoding existential elimination

• given
• let X, x t1 in t2
• where t1 ?Y. (?X. T-gtY) -gt Y
• first apply to result type T2 to get type (?X.
  T-gt gtT2) -gt T2
• let X, x t1 in t2 t1 T2
• then apply to continuation of type ?X. T-gtT2 to
  get result type T2
• let X, x t1 in t2 t1 T2 (?X. ?x
  T.t2)

def
def
41
Encoding existential introduction

• given
• S, t as ?X, T
• we must use S and t to build a value of type ?Y.
  (?X. T -gt Y) -gt Y
• begin with two abstractions
• S, t as ?X, T ?Y. ?f
  (?X. T-gtY)
• apply f to appropriate arguments first, supply
  S
• S, t as ?X, T ?Y. ?f (?X. T-gtY).
  f S
• then supply t of type S to get result type Y
• S, t as ?X, T ?Y. ?f
  (?X. T-gtY). f S t

def
def
def
42
Existential types - summary

• existential types are another form of
  polymorphism
• parametricity of existentials leads to
  representation independence
• trade-offs between ADTs and objects
• ADTs support binary operations, objects do not
• objects support free intermixing of different
  implementations, ADTs do not
• existentials can be encoded using universal types

43
Module systems - overview

• The OCaml Module System
• Haskell Modules

44
The OCaml Module System

• three key parts
• structures are packaged environments. They
  consist of a group of core ML objects (values,
  types, exceptions) and can be manipulated as a
  single unit.
• module Name struct implementation end
• signatures are the types of structures.
  Corresponding to each structure one can derive a
  signature which consists of the names of the
  objects in the structure and type information for
  the objects.
• module type Name sig signature end
• functors are mappings taking structures to
  structures. These are useful in specifying
  generic packages that given any structure
  with a given signature can generate a new
  structure with some other signature.
• module Name functor (ArgName ArgSig) -gt
  struct implementation end
• module Name (Arg ArgSig)
  struct implementation end

45
OCaml Modules Values

• Collection of named (mutually-recursive) values
• module IntListSet struct
• let empty
• let add fun i is -gt i
• List.filter (fun j -gt i ! j) is
• let asList fun is -gt is
• end
• Access using dot-notation
• val ok int list
• let ok IntListSet.add 1 IntListSet.empty
• Modules (structures) have types (signatures)
• module type INTLISTSET sig
• val empty int list
• val add int -gt int list -gt int list
• val asList int list -gt int list
• end

46
OCaml Modules Types

• May also include named (data) types
• module IntListSet struct
• type t int list
• let empty
• let add fun i is -gt i
• List.filter (fun j -gt i ! j) is
• let asList fun is -gt is
• end
• val ok IntListSet.t
• let ok IntListSet.add 1 IntListSet.empty
• And named (nested) modules.

47
OCaml Modules Abstract Types

• May also include abstract types
• module type INTSET sig
• type t --- abstract
• val empty t
• val add int -gt t -gt t
• val asList t -gt int list
• end
• module IntSet INTSET struct
• type t int list --- concrete
• let empty
• let add fun i is -gt i
• List.filter (fun j -gt i ! j) is
• let asList fun is -gt is
• end
• val ok IntListSet.t
• let ok IntListSet.add 1 IntListSet.empty
• val fail int list
• let fail IntListSet.add 1 2
• -- type error incompatible types int list and
  IntListSet.t

48
OCaml Modules Separate Compilation

• Top-level modules may correspond with compilation
  units.
• True separate compilation is possible if each
  top-level module is accompanied by a top-level
  signature.
• intSet.mli
• type t
• val empty t
• val add int -gt t -gt t
• val asList t -gt int list
• intSet.ml
• type t int list
• let empty
• let add fun i is -gt i
• List.filter (fun j -gt i ! j) is
• let asList fun is -gt is
• Compiler never needs to look at intSet.ml when
  compiling other modules only intSet.mli.

49
OCaml Modules Value Parameterisation

• May be parameterised by the values within other
  modules(functors)
• module type INTORD sig
• val less int -gt int -gt bool
• end
• module MkIntBinTreeSet functor (IntOrd
  INTORD) -gt
• struct
• type t Leaf Node of bintree int
  bintree
• let empty Leaf
• let rec add fun i t -gt
• match t with
• Leaf -gt Node (Leaf, i, Leaf)
• Node (l, j, r) -gt
• if IntOrd.less i j then
• Node (add i l, j, r)
• else ...
• let rec asList
• end
• module IntOrd struct let less (lt) end

50
OCaml Modules Type Parameterisation

• May be parameterised by the types within other
  modules
• module type ORD sig
• type t
• val less t -gt t -gt bool
• end
• module MkBinTreeSet functor (Ord ORD) -gt
  struct
• type t Leaf Node of bintree Ord.t
  bintree
• let empty Leaf
• let rec add fun x t -gt
• let rec asList
• end
• Module IntOrd struct
• type t int
• let less (lt)
• end
• Module IntBinTreeSet MkBinTreeSet IntOrd

51
OCaml Modules Type Sharing

• Interaction of
• Separate compilation, and
• Parameterisation by types
• Requires special support
• m.mli
• module type SET sig
• type elt
• type t
• val empty t
• val add elt -gt t -gt t
• val asList t -gt elt list
• end
• module type MKBINTREESET
• functor (Ord ORD) -gt SET
• module MkBinTreeSet MKBINTREESET

52
OCaml - Modules Type Sharing

• Interaction of
• Separate compilation, and
• Parameterisation by types
• Requires special support
• x.ml
• module IntOrd struct
• type t int
• let less (lt)
• end
• module IntBinTreeSet M.MkBinTreeSet IntOrd
• val fail IntBinTreeSet.t
• let fail IntBinTreeSet.add 1
  IntBinTreeSet.empty
• -- type error incompatible types int
  and IntBinTreeSet.elt

53
OCaml Modules Type Sharing (2)

• We need to capture the sharing of types between
  the argument and result of the functor
  MkBinTreeSet.
• m.mli
• module type SET sig
• type elt
• type t
• val empty t
• val add elt -gt t -gt t
• val asList t -gt elt list
• end
• module type MKBINTREESET
• functor (Ord ORD) -gt (SET with type elt
  Ord.t)
• module MkBinTreeSet MKBINTREESET

54
OCaml Modules Type Sharing (2)

• We need to capture the sharing of types between
  the argument and result of the functor
  MkBinTreeSet.
• x.ml
• module IntOrd struct
• type t int
• let less (lt)
• end
• module IntBinTreeSet M.MkBinTreeSet IntOrd
• val fail IntBinTreeSet.t
• let fail IntBinTreeSet.add 1
  IntBinTreeSet.empty
• -- ok, since int IntOrd.t
  IntBinTreeSet.elt

55
The Haskell module system

• Haskell has a very simple module system -- a
  flat module namespace with the ability to import
  and export various entities, hide names, and
  specify that module qualification (M.x) is
  required. It also provides support for abstract
  data types by allowing one to export a data type
  without its constructors.
• -- Paul Hudak, 1998

56
The Haskell Module System

• Module headers
• module Ant where
• data Ants
• anteater x
• The convention for file names is that a module
  Ant resides in the Haskell file Ant.hs or Ant.lhs.

57
The Haskell Module System

• Importing a module
• module Bee where
• import Ant
• beeKeeper
• This means that the visible definitions from Ant
  can be used in Bee. By default the visible
  definitions in a module are those which appear in
  the module itself
• module Cow where
• import Bee

58
The Haskell Module System

• The main module
• Each system of modules should contain a
  top-level module called Main, which gives a
  definition to the name main.
• Note that a module with no explicit name is
  treated as Main.

59
The Haskell Module System

• Export controls
• we might wish not to export some auxiliary
  functions.
• We perhaps want to export some of the definitions
  we imported from other modules.
• We can control what is exported by following the
  name of the module with a list of what is to be
  exported
• module Bee ( beeKeeper, Ants(..), anteater )
  where
• We follow the type name with (..) to indicate
  that the constructors of the type are exported
  with the type itself if this is omitted, then
  the type acts like an abstract data type.

60
The Haskell Module System

• We can also state that all the definitions in a
  module are to be exported
• module Bee ( beeKeeper, module Ant ) where
• or equivalently
• module Bee ( module Bee, module Ant ) where
• The simpler header
• module Fish where
• is equivalent to
• module Fish ( module Fish ) where

61
The Haskell Module System

• Import controls
• we can control how objects are to be imported
• import Ant ( Ants(..) )
• stating that we want just the type Ants
• we can alternatively say which names we wish to
  hide
• import Ant hiding ( anteater )

62
The Haskell Module System

• Suppose that in our module we have a definition
  of bear, and also there is an object named bear
  in the module Ant.
• Use the qualified name Ant.bear for the imported
  object, reserving bear for the locally defined
  one.
• To use qualified names we should make the import
  thus
• import qualified Ant
• Give a local name Insect to a imported module
  Ant
• import Insect as Ant

63
The Haskell Module System

• The standard prelude
• Prelude.hs is implicitly imported into every
  module.
• Modify this import, perhaps hiding one or more
  bindings thus
• module Eagle where
• import Prelude hiding (words)
• A re-definition of a prelude function cannot be
  done invisibly. We have explicitly to hide
  the import of the name that we want to re-define.
• If we also wish to have access to the original
  definition of words we can make a qualified
  import of the prelude,
• import qualified Prelude
• and use the original words by writing its
  qualified name Prelude.words.

64
Example - Stacks

• A stack implemented as an ADT module can be
  defined as follows
• module Stack(Stack,push,pop,top,emptyStack,stack
  Empty) where
• push a -gt stack a -gt Stack a
• pop Stack a -gt Stack a
• top Stack a -gt a
• emptyStack Stack a
• stackEmpty Stack a -gt Bool

65
Example - Stacks

• A first implementation can be done using a
  user-defined type
• data Stack a EmptyStk
• Stk a (Stack a)
• push x s Stk x s
• pop EmptyStk error pop from an empty
  stack
• pop (Stk _ s) s
• top EmptyStk error top from an empty
  stack
• top (Stk x _) x
• emptyStack EmptyStk
• stackEmpty EmptyStk True
• stackEmpty _ False

66
Example - Stacks

• Another possible implementation can be obtained
  using the predefined list data structure because
  push and pop are similar to the () and tail
  operations.
• newtype Stack a Stk a
• push x (Stk xs) Stk (xxs)
• pop (Stk ) error pop from an empty
  stack
• pop (Stk (_xs)) Stk xs
• top (Stk ) error top from an empty
  stack
• top (Stk (x_)) x
• emptyStack Stk
• stackEmpty (Stk ) True
• stackEmpty (Stk _ ) False

67

• References
• Types and Programming Languages
• Abstract Types Have Existential Type
• First-Class Modules for Haskell

• Thank You Very Much!