Conductor

1
Conductor

• A Framework for Distributed, Type-checked
  Computing
• Matthew Kehrt

2
Distributed Computing

• Today, people are dealing with larger and larger
  computational problems

3
Distributed Computing

• Today, people are dealing with larger and larger
  computational problems
• Scientific computations often analyze enormous
  amounts of data
• Radio telescope data

4
Distributed Computing

• Today, people are dealing with larger and larger
  computational problems
• Scientific computations often analyze enormous
  amounts of data
• Radio telescope data
• Some problems have large search spaces
• Factoring
• Password cracking

5
Distributed Computation

• A solution split large computations across
  multiple computers

6
Distributed Computation

• A solution split large computations across
  multiple computers
• Some projects already do this
• SETI_at_home
• folding_at_home (protein folding)
• evolution_at_home (genetics)

7
Distributed Computation

• A solution split large computations across
  multiple computers
• Some projects already do this
• SETI_at_home
• folding_at_home (protein folding)
• evolution_at_home (genetics)
• However, these examples are all for very specific
  problems

8
Grid Computing

• We want something more generic

9
Grid Computing

• We want something more generic
• A general framework for distributing computation
• A way of submitting work to a collection of
  machines

10
Grid Computing

• We want something more generic
• A general framework for distributing computation
• A way of submitting work to a collection of
  machines
• We want a peer-to-peer network to distribute code

11
Safety?

• We want to run code on remote hosts

12
Safety?

• We want to run code on remote hosts
• How do we guarantee code is benign?
• Running code in a safe VM is expensive

13
Safety?

• We want to run code on remote hosts
• How do we guarantee code is benign?
• Running code in a safe VM is expensive
• Native code is faster

14
Safety?

• We want to run code on remote hosts
• How do we guarantee our code is benign?
• Running code in a safe VM is expensive
• Native code is faster
• Can we do static checks for safety on code to
  run?

15
TAL

• Type systems can make static guarantees about
  program behavior

16
TAL

• Type systems can make static guarantees about
  program behavior
• However, we want to send object code over the
  network

17
TAL

• Type systems can make static guarantees about
  program behavior
• However, we want to send object code over the
  network
• Reduces work that needs to be done at runtime
• How about a typed assembly language which need be
  only linked?

18
TAL

• A good type system allows us to distribute
  guaranteed safe code

19
TAL

• A good type system allows us to distribute
  guaranteed safe code
• Assembly language means little work needs to be
  done to prepare distributed code for running.

20
The Conductor

• Each machine on the grid runs a Conductor
• The Conductor
• Maintains a queue of work to be done
• Locates peers through a gossip protocol
• Queries known peers for others on the network
• Downloads, checks and runs work
• Returns results

21
The Conductor

• Allows for multiple safety policies (different
  type systems) selected by the user
• Publishes code and results in a distributed
  hashtable
• Caches results for quick recomputation
• Is tolerant to machines joining and leaving the
  network

22
Summer Goals

• A network running on machines at CMU

23
Summer Goals

• A network running on machines at CMU
• A larger network running on machines owned by the
  Pittsburgh public school system

24
Summer Goals

• A network running on machines at CMU
• A larger network running on machines owned by the
  Pittsburgh public school system
• Adding a new type system to the conductor (TAL
  Two, or TALT)

25
Summer Goals

• A network running on machines at CMU
• A larger network running on machines owned by the
  Pittsburgh public school system
• Adding a new type system to the conductor (TAL
  Two, or TALT)
• Ability to merge two grids