Self-Adjusting Computation

1
Self-Adjusting Computation

• Robert Harper
• Carnegie Mellon University
• (With Umut Acar and Guy Blelloch)

2
The Problem

• Given a static algorithm, obtain a dynamic, or
  incremental, version.
• Maintain a sorted list under insertions and
  deletions
• Maintain a convex hull under motions of the
  points.
• Maintain semantics of a program under edits.

3
Example Sorting

• Input
• Output

4
Dynamic Algorithms

• There is a large body of work on dynamic /
  incremental algorithms.
• Specific techniques for specific problems.
• Our interest is in general methods, rather than
  ad hoc solutions.
• Applying them to a variety of problems.
• Understanding when these methods apply.

5
Self-Adjusting Computation

• Self-adjusting computation is a method for
  dynamizing a static algorithm.
• Start with a static algorithm for a problem.
• Make it robust under specified changes.
• Goal fast response to small change.
• Fast and small are problem-specific!
• As ever, the analysis can be difficult.

6
Self-Adjusting Computation

• Generalizes incremental computation.
• Attribute grammars, circuit models assume static
  control dependencies.
• SAC permits dynamic dependencies.
• Combines algorithmic and programming language
  techniques.
• Linguistic tools to ensure correctness relative
  to static algorithm.
• Algorithmic techniques for efficient
  implementation.

7
Self-Adjusting Computation

• Adaptivity Propagate the effects on the output
  of a change to the input.
• Selective Memoization Reuse old results,
  provided they are valid after change.
• Adaptive Memoization Reuse old results, even
  though they may not be valid after change.

8
Model of Computation

• Purely functional programming model.
• Data structures are persistent.
• No implicit side effects or mutation.
• Imperative model of change.
• Run on initial input to obtain output.
• Make modifications to the input.
• Propagate changes to the output.

9
Model of Computation
10
A Simple Example Map

• data cell nil cons of int list
• and list cell
• fun map (llist) map(l)
• and map(c) case c of nil ) nil
• cons (h, t) ) cons (h10, map t)

11
Static Version of Map

• Running map on the input list
• yields the new output list

12
Dynamic Version of Map

• To permit insertions and deletions, lists are
  made modifiabledata cell nil cons of int
  listand list cell mod

13
Dynamic Version of Map

• Insertion changes a modifiable

14
Dynamic Version of Map

• Wed like to obtain the result

15
Dynamic Version of Map

• Can we update the result in O(1) time?
• Make one new call to map.
• Splice new cell into old result.
• Yes, using self-adjusting computation!
• Adaptivity call map on the new node.
• Memoization re-synchronize with suffix.

16
Adaptivity Overview

• To make map adaptive, ensure that
• Changes invalidate results that depend on the
  modified value.
• Computations dependent on a change are re-run
  with the new value.
• Two key ideas
• Make access to modifiables explicit.
• Maintain dependencies dynamically.

17
Adaptive Map

• data cell nil cons of int list
• and list cell mod
• fun map (llist) mod (let mod c l in
  write(map c))
• and map c case c of nil ) nil cons (h, t) )
  cons (h10, map t)

Allocate new modifiable
Read old modifiable
Write new modifiable
18
Adaptive Map

• Modification to input

Modified cell is argument of map, which must be
re-run.
19
Adaptive Map

• Associated output is invalidated, and suffix is
  re-created.

Result of map written here.
20
Adaptive Programming

• Crux dependencies among modifiables.
• Writing a modifiable invalidates any computation
  that reads it.
• One read can be contained within another.
• Dependencies are fully dynamic!
• Cells are allocated dynamically.
• Reads affect control flow.

21
Adaptive Programming

• Change propagation consists of
• Re-running readers of changed cells.
• Updating dependencies during re-run.
• To ensure correctness,
• All dependencies must be accurately tracked.
• Containment ordering must be maintained.
• Linguistic tools enforce these requirements!

22
Type System for Adaptivity

• The type ? mod is a modality.
• From lax modal logic.
• And therefore forms a monad.
• Two modes of expression
• Stable ordinary functional code, not affected by
  changes.
• Changeable affected by change, will be written
  to another modifiable.

23
Type System for Adaptivity

• Elimination form for ? modlet mod x? s in c
  end
• Read modifiable given by s.
• Bind value to x?, evaluate c.
• Makes dependencies explicit
• Records read of given modifiable.
• Re-run c with new x if changed.
• Reads within c are contained in this read.

24
Type System for Adaptivity

• Execution maintains a trace of adaptive events.
• Creation of a modifiable.
• Writes to a modifiable.
• Reads of a modifiable.
• Containment is recorded using Sleator-Dietz order
  maintenance algorithm.
• Associate time intervals with events.
• Re-run reader within the old time interval.
• Requires arbitrary fractionation of time steps.

25
Adaptive Map

• Responds to insertion in linear time

26
Memoizing Map

• For constant-time update, we must re-synchronize
  with the old result.
• Results after insertion point remain valid
  despite change.
• Re-use, rather than recompute, to save time.
• Selective memoization is a general technique for
  achieving this.

27
Selective Memoization

• Standard memoization is data-driven.
• Associate with a function f a finite set of
  ordered pairs (x, f(x)).
• Consult memo table before call, update memo table
  after call.
• Cannot handle partial dependencies.
• Eg, read only first 10 elements of an array.
• Eg, use an approximation of input.

28
Selective Memoization

• Selective memoization is control-driven.
• Guided by the exploration of the input.
• Sensitive to approximations.
• Associate results with control paths.
• Have I been here before?
• Control path records dependencies of output on
  input.

29
Memoized Adaptive Map

• fun map (llist) mod (let mod c l in
  write(map c))
• and memo map c mcase c of nil ) return
  (nil) cons (h, t) ) let !hh and !tt in
  return(cons (h10, map t))

Depends only on nil/cons.
Depends on nil/cons, head, and tail.
30
Memoized Adaptive Map

• With selective memoization we obtain
• Constant-time update after insertion.

31
Memoized Programming

• Selective memoization requires an accurate record
  of I/O dependencies.
• Which aspects of the input are relevant to the
  output?
• Sensitive to dynamic control flow.
• Linguistic support provides
• Specification of approximations.
• Accurate maintenance of dependencies.

32
Type System for Memoization

• Based on S4 modal logic for necessity.
• Truth assumptions restricted variables.
• Validity assumptions ordinary variables.
• Modality !? means value is necessary.
• !(int int) both components necessary
• !int !int either, neither, or both parts
  needed, depending on control flow.

33
Type System for Memoization

• Key idea variables are classified as restricted
  or unrestricted.
• Arguments to memoized functions are restricted.
• Results of memoized functions may not involve
  restricted variables.
• Elimination form for !? binds an unrestricted
  variable.
• Ensures that relevant portion of input must be
  explicitly touched to record dependency.

34
Selective Memoization

• fun map (llist)
• and memo map c mcase c of nil ) return
  (nil) cons (h, t) ) let !hh and !tt in
  return(cons (h10, map t))

Restricted
Unrestricted
35
Adaptive Memoization

• Effectiveness of memoization depends on
  preserving identity.
• Modifiables compare by reference.
• Copying a modifiable impedes re-use.
• This conflicts with the functional programming
  model.
• Eg, functional insertions copy structure.
• Undermines effectiveness of memoization.

36
Adaptive Memoization

• Consider again applying map to
• We obtain the new modifiable list

37
Adaptive Memoization

• Now functionally insert an element

38
Adaptive Memoization

• Running map on result yields

39
Adaptive Memoization

• Subsequent runs propagate the effect

40
Adaptive Memoization

• Ideally, wed re-use the old prefix!
• But what justifies this?

41
Memoization and Adaptivity

• Consider a memoized copy functionfun copy
  (!m int mod) return (mod (let mod x m in
  x))
• What happens if we modify m?
• Value might change under our feet.
• But adaptivity restores correctness!

42
Memoization and Adaptivity

• Initial run of copy
• Calls to copy at later stages return old cell,
  which is updated by adaptivity.

17
17
17
17
43
Adaptive Memoization

• Permit inaccurate memoization.
• Allows recovery of old cells.
• Cached result will be incorrect.
• Adapt incorrect result to restore correctness.
• Use change propagation to revise answer.
• Only sensible in conjunction with adaptivity!

44
Adaptive Memoization

• fun map (llist)
• and memo map c mcase c of nil ) return
  (nil) cons (h, t) ) let !hh in let ?tt
  in return(cons (h10, map t))

Do not record dependency!
Memo match only on nil/cons and head.
45
Adaptively Memoized Map

• On the initial input
• Map yields the output

46
Adaptively Memoized Map

• After a functional update, the input is

47
Adaptively Memoized Map

• Now maps yields the inaccurate result
• Memo matches on head value 2, yielding old
  result, with incorrect tail.

Tail is incorrect!
Result of map
48
Adaptively Memoized Map

• Restore accuracy by self-assignment

49
Adaptively Memoized Map

• Change to input propagates to output

50
Some Results

• Quicksort expected O(lg n) update after insert
  or delete at a random position.
• Mergesort expected O(lg n) update after insert
  or delete.
• Tree Contraction O(lg n) update after adding or
  deleting an edge.
• Kinetic Quickhull O(lg n) per event measured.

51
Ongoing Work

• When can an algorithm be dynamized?
• Consider edit distance between traces for a class
  of input changes.
• Small edit distance suggests we can build a
  dynamic version using SAC.
• What is a good semantic model?
• Current methods are rather ad hoc.
• Are there better models?

52
Conclusion

• Self-adjusting computation is a powerful method
  for building dynamic algorithms.
• Systematic methodology.
• Simple correctness criteria.
• Easy to implement.
• The interplay between linguistic and algorithmic
  methods is vital!
• ? is also a powerful algorithmic tool!

53
Questions?