Time-space tradeoff lower bounds for non-uniform computation

1
Time-space tradeoff lower bounds for
non-uniform computation

• Paul Beame
• University of Washington

4 July 2000
2
Why study time-space tradeoffs?

• To understand relationships between the two most
  critical measures of computation
• unified comparison of algorithms with varying
  time and space requirements.
• non-trivial tradeoffs arise frequently in
  practice
• avoid storing intermediate results by
  re-computing them

3
e.g. Sorting n integers from 1,n2

• Merge sort
• S O(n log n), T O(n log n)
• Radix sort
• S O(n log n), T O(n)
• Selection sort
• only need - smallest value output so far
  - index of current
  element
• S O(log n) , T O(n2)

4
Complexity theory

• Hard problems
• prove L P
• prove non-trivial time lower bounds for natural
  decision problems in P
• First step
• Prove a space lower bound, e.g. Sw (log n),
  given an upper bound on time T, e.g. TO(n) for a
  natural problem in P

5
An annoyance

• Time hierarchy theorems imply
• unnatural problems in P not solvable in time O(n)
• Makes first step vacuous for unnatural problems

6
Non-uniform computation

• Non-trivial time lower bounds still open for
  problems in P
• First step still very interesting even without
  the restriction to natural problems
• Can yield bounds with precise constants
• But proving lower bounds may be harder

7
Talk outline

• The right non-uniform model (for now)
• branching programs
• Early success
• multi-output functions, e.g. sorting
• Progress on problems in P
• Crawling
• restricted branching programs
• That breakthrough first step (and more)
• true time-space tradeoffs
• The path ahead

8
Branching programs
x1
1
x2
x3
0
x4
x5
x5
x3
x1
x2
x7
x7
x8
1
0
9
Branching programs
x1
1
x2
x3
0
x4
x5
x5
x(0,0,1,0,...)
x3
x1
To compute f0,1 n 0,1 on input
(x1,,xn) follow path from source to sink
x2
x7
x7
x8
1
0
10
Branching program properties

• Length length of longest path
• Size of nodes
• Simulate TMs
• node configuration with input bits erased
• time T Length
• space Slog2Size TM space log2n (head)
  space on an index TM
• polysize non-uniform L

11
TM space complexity
read-only input
x1 x2 x3 x4 xn
working storage
Space of bits of working storage
output
12
Branching program properties

• Simulate random-access machines (RAMs)
• not just sequential access
• Generalizations
• Multi-way version for xi in arbitrary domain D
• good for modeling RAM input registers
• Outputs on the edges
• good for modeling output tape for multi-output
  functions such as sorting
• BPs can be leveled w.l.o.g.
• like adding a clock to a TM

13
Talk outline

• The right non-uniform model (for now)
• branching programs
• Early success
• multi-output functions, e.g. sorting
• Progress on problems in P
• Crawling
• restricted branching programs
• That breakthrough first step (and more)
• true time-space tradeoffs
• The path ahead

14
Success for multi-output problems

• Sorting
• T S W (n2/log n) Borodin-Cook 82
• T S W (n2) Beame 89
• Matrix-vector product
• T S W (n3) Abrahamson 89
• Many others including
• Matrix multiplication
• Pattern matching

15
Proof ideas layers and trees

• m outputs on input x
• at least m/r outputs in some tree Tv
• Only 2S trees Tv
• Typical Claim
• if T/r en, each tree Tv outputs p correct
  answers on only a c-p fraction of inputs
• Correct for all x implies 2Sc-m/r is at least 1
• SW(m/r)W(mn/T)

v0
v1
T/r
v
T/r
vr-1
T
vr
0
1
16
Limitation of the technique

• Never more than T S W (nm) where m is number of
  outputs
• It is unfortunately crucial to our proof that
  sorting requires many output bits, and it remains
  an interesting open question whether a similar
  lower bound can be made to apply to a set
  recognition problem, such as recognizing whether
  all n input numbers are distinct.
  Cook Turing
  Award Lecture, 1983

17
Talk outline

• The right non-uniform model (for now)
• branching programs
• Early success
• multi-output functions, e.g. sorting
• Problems in P
• Crawling
• restricted branching programs
• That breakthrough first step (and more)
• true time-space tradeoffs
• The path ahead

18
Restricted branching programs

• Constant-width - only a constant number
  of nodes per level
• Chandra-Furst-Lipton 83
• Read-once - every variable read at most
  once per path
• Wegener 84, Simon-Szegedy 89, etc.
• Oblivious - same variable queried per level
• Babai-Pudlak-Rodl-Szemeredi 87,
  Alon-Maass 87, Babai-Nisan-Szegedy 89
• BDD Oblivious read-once

19
BDDs and best-partition communication complexity

• Given f0,18-gt0,1
• Two-player game
• Player A has x1,x3,x6,x7
• Player B has x2,x4,x5,x8
• Goal communicate fewest bits possible to compute
  f
• Possible protocol Player A sends the name of
  node.
• BDD space of bits sent for best partition
  into A and B

x7
x1
x6
A
x3
x2
x8
B
x4
x5
0
1
20
Communication complexity ideas

• Each conversation for f0,1Ax0,1B 0,1
  corresponds to a rectangle YAxYB of inputs
  YA 0,1A YB 0,1B
• BDD lower bounds
• size min(A,B) of rectangles in tiling of
  inputs by f-constant rectangles with partition
  (A,B)
• Read-once bounds
• same tiling as BDD bounds but each rectangle in
  tiling may have a different partition

21
Restricted branching programs

• Read-k - no variable queried gt k times on
• any path - syntactic read-k
• Borodin-Razborov-Smolensky 89, Okolnishnikova
  89, etc.
• any consistent path - semantic read-k
• many years of no results
• nothing for general branching programs either

22
Uniform tradeoffs

• SAT is not solvable using O(n1-e) space if time
  is n1o(1). Fortnow 97
• uses diagonalization
• works for co-nondeterministic TMs
• Extensions for SAT
• SlogO(1) n implies T W (n1.4142..-e )
  deterministic Lipton-Viglas 99
• with up to no(1) advice Tourlakis 00
• S O(n1-e) implies TW (n 1.618..-e ).
  Fortnow-van Melkebeek 00

23
Non-uniform computation

• Beame-Saks-Thathachar FOCS 98
• Syntactic read-k branching programs exponentially
  weaker than semantic read-twice.
• f(x) xTMx0 (mod q) x GF(q)n
• e nloglog n time W(n log1-en) space for qn
• f(x) xTMx0 (mod 3) x 0,1n
• 1.017n time implies W (n) space
• first Boolean result above time n for general
  branching programs

24
Non-uniform computation

• Ajtai STOC 99
• 0.5log n Hamming distance for x 1,n2n
• kn time implies W(n log n) space
• follows from Beame-Saks-Thathachar 98
• improved to W(nlog n) time by Pagter-00
• element distinctness for x 1,n2n
• kn time implies W(n) space
• requires significant extension of techniques

25
That breakthrough first step!

• Ajtai FOCS 99
• f(x,y) xTMyx (mod 2)
• kn time implies W(n) space
• First result for non-uniform Boolean computation
  showing
• time O(n) space w(log n)

x 0,1n
y 0,12n-1
26
Ajtais Boolean function
y1
0
y2
f(x,y) xTMyx (mod 2)
y3
y4
y2n-1
y8
y7
y6
yn
My
My is a modified Hankel matrix
27
Superlinear lower bounds

• Beame-Saks-Sun-Vee FOCS 00
• Extension to e-error randomized
  non-uniform algorithms
• Better time-space tradeoffs
• Apply to both element distinctness and f(x,y)
  xTMyx (mod 2)

28
(m,a)-rectangles

• An (m,a)-rectangle R DX is a subset defined
  by
  disjoint sets A,B X, s
  DAUB SA DA, SB
  DB such that
• R z zAUB s, zA SA, zB SB
• A,B m
• SA/DA, SB/DB a

29
An (m,a)-rectangle
SA
SB
DB
DA
SA and SB each have density at least a
In general A and B may be interleaved in 1,n
30
Key lemma BST 98

• Let program P use
• time T kn
• space S
• accept fraction d of its inputs in Dn
• then P accepts all inputs in some
  (m,a)-rectangle where
• m bn
• a is at least d 2-4(k1) m - (S1) r
  
• b-1 2k and r k2 2k

31
Improved key lemma Ajtai 99 s

• Let program P use
• time T kn
• space S
• accept fraction d of its inputs in Dn
• then P accepts all inputs in some
  (m,a)-rectangle where
• m bn
• a is at least
• b-1 and r are constants depending on k
  
  

32
Proving lower bounds using the key lemmas

• Show that the desired function f
• evaluates to 1 a large fraction of the time
• i.e., d is large
• evaluates to 0 on some input in any large
  (m,a)-rectangle
• where large is given by the lemma bounds
• or ... do the same for f
  

33
Our new key lemma

• Let program P use time T kn space S and accept
  fraction d of its inputs in Dn
• Almost all inputs P accepts are in
  (m,a)-rectangles accepted by P where
• m bn
• a is at least
• b-1 and r are
• no input is in more than O(k) rectangles
  
  

34
Proving randomized lower bounds from our key lemma

• Show that the desired function f
• evaluates to 1 a large fraction of the time
• i.e, d is large
• evaluates to 0 on a g fraction of inputs in any
  large-enough (m,a)-rectangle
  
• or ... do the same for f
• Gives space lower bound for O(gd/k)-error
  randomized algorithms running in time kn

35
Proof ideas layers and trees
v0
f
v1
kn/r
v2
of (v1,,vr-1) is 2S(r-1)
kn
r
kn/r
vr-1

i1
vr
can be computed in kn/r height
0
1
36
(r,e)-decision forest

• The conjunction of r decision trees (BPs that
  are trees) of height en
• Each is a computed by a
  (r,k/r)-decision forest
• Only 2S(r-1) of them
• The various accept disjoint
  sets of inputs

37
Decision forest

• Assume wlog all variables read on every input
• Fix an input x accepted by the forest
• Each tree reads only a small fraction of the
  variables on input x
• Fix two disjoint subsets of trees, F and G

kn/r
38
Core variables

• Can split the set of variables into
• core(x,F)variables read only in F (not read
  outside F)
• core(x,G)variables read only in G (not read
  outside G)
• remaining variables
• stem(x,F,G)assignment to remaining variables
• General idea use core(x,F), core(x,G), and
  stem(x,F,G) to define (m,a)-rectangles

kn/r
T1
T2
T3
T4
Tt
39
A partition of accepted inputs

• Fix F, G,x accepted by P
• Rx,F,G y core(y,F)core(x,F),
  
  core(y,G)core(x,G),
  stem(y,F,G)stem(x,F,G),
  and P accepts y
• For each F, G the Rx,F,G partition the accepted
  inputs into equivalence classes
• Claim the Rx,F,G are (m,a)-rectangles

40
Classes are rectangles

• Let Acore(x,F), Bcore(x,G), sstem(x,F,G)
• SAyA y in Rx,F,G , SBzB z in Rx,F,G
• Let w(s,yA,zB)
• w agrees with y in all trees outside G
• core(w,G)core(y,G)core(x,G)
• w agrees with z in all trees outside F
• core(w,F)core(z,F)core(x,F)
• stem(w,F,G)sstem(x,F,G)
• P accepts w since it accepts y and z
• So... w is in Rx,F,G

41
Few partitions suffice

• Only 4k pairs F,G suffice to cover almost all
  inputs accepted by P by large (m,a)-rectangles
  Rx,F,G
• Choose F,G uniformly at random of suitable size,
  depending on access pattern of input
• probability that F,G isnt good is tiny
• one such pair will work for almost all inputs
  with the given access pattern
• Only 4k sizes needed.

42
Special case oblivious BPs

• core(x,F), core(x,G) dont depend on x
• Choose Ti in F with prob q
  G with prob q
  neither with prob 1-2q

43
xTMyx on an (m,a)-rectangle

A
B
x
For every s on AUB, f(xAUB,s,y)
xAT MAB xB g(xA,y)
h(xB,y)
A
B
My
x
44
Rectangles, rank, rigidity

• largest rectangle on which xATMxB is constant has
  a 2-rank(M)
• Borodin-Razborov-Smolensky 89
• Lemma Ajtai 99 Can fix y s.t. every bnxbn minor
  MAB of My has
  rank(MAB) cbn/log2(1/b)
• improvement of bounds of
  Beame-Saks-Thathachar 98
  Borodin-Razborov-Smolensky 89 for
  Sylvester matrices

45
High rank implies balance

• For any rectangle SAxSB 0,1Ax0,1B with
  m(SAxSB) AB23-rank(M)
  Pr xATMxB 1 xA SA, xB SB 1/32
  Pr xATMxB 0 xA SA, xB SB 1/32
• derived from result for inner product in r
  dimensions
• So rigidity also implies balance for all large
  rectangles and so
• Also follows for element distinctness
• Babai-Frankl-Simon 86

46
Talk outline

• The right non-uniform model (for now)
• branching programs
• Early success
• multi-output functions, e.g. sorting
• Progress on problems in P
• Crawling
• restricted branching programs
• That breakthrough first step (and more)
• true time-space tradeoffs
• The path ahead

47
Improving the bounds

• What is the limit?
• TW(nlog(n/S)) ?
• TW(n2/S) ?
• Current bounds for general BPs are almost equal
  to best current bounds for oblivious BPs !
• TW(nlog(n/S)) using 2-party CC AM
• TW(nlog2(n/S)) using multi-party CC BNS

48
Improving the bounds

• (m,a)-rectangles a 2-party CC idea
• insight generalizing to non-oblivious BPs
• yields same bound as AM for oblivious BPs
• Generalize to multi-party CC ideas to get better
  bounds for general BPs?
• similar framework yields same bound as BNS for
  oblivious BPs
• Improve oblivious BP lower bounds?
• ideas other than communication complexity?

49
Extension to other problems

• Problem should be hard for (best-partition)
  2-party communication complexity (after most
  variables fixed).
• try oblivious BPs first
• Prime candidate (directed) st-connectivity
• Many non-uniform lower bounds in structured JAG
  models Cook-Rackoff, BBRRT, Edmonds,
  Barnes-Edmonds, Achlioptas-Edmonds-Poon
• Best-partition communication complexity bounds
  known

50
Limitations of current method

• Need ngtT/r decision tree height
• else all functions trivial
• so r gt T/n
• A decision forest works on a 2-Sr fraction of the
  accepted inputs
• only place space bound is used
• So need Srltn else d.f. need only work on one
  input
• implies ST/n lt n, i.e. T lt n2/S