Hardness of Reconstructing Multivariate Polynomials'

1
Hardness of Reconstructing Multivariate
Polynomials.

• Parikshit Gopalan U. Washington
• Subhash Khot NYU/Gatech
• Rishi Saket Gatech/NYU

2
Curve Fitting
Problem Given data points, find a low degree
polynomial that fits best. Easy if there is a
perfect fit. Well studied problem
3
Curve Fitting through the ages
4
Curve Fitting through the ages
5
Curve Fitting through the ages
6
!
7
Statistics Least Squares
8
Coding Theory
Computational Learning
Polynomial Reconstruction
PCPs
Cryptography
Pseudorandom-ness
9
The Reconstruction Problem

• Input Degree d.

Output A degree d polynomial that best fits the
data. In this talk Finite fields, Hamming
distance.
10
The Reconstruction Problem

• Input Degree d, set S, values f(x) for x 2 S.
• Output A degree d polynomial that best fits the
  data.
• Parameters that matter
• Degree d, Field F.
• Set S.
• How good is the best fit? (error-rate ?)

11
Algorithms for Reconstruction

• Univariate Case Sudan, Guruswami-Sudan
• Multivariate Case Goldreich-Levin,
  Goldreich-Rubinfeld-Sudan, Arora-Sudan,
  Sudan-Trevisan-Vadhan
• Can tolerate very high error rate ?.
• Are these algorithms optimal?

12
Hardness Results Univariate Case

• Degree d polynomials, n points in F.
• Guruswami-Vardy NP-hard to tell if some degree
  d poly. has d 2 agreements.
• Guruswami-Sudan Can tell if some degree d
  poly. has (nd)0.5 agreement.

13
Hardness Results Multivariate Case

• Linear polynomials over F2
• Hastad NP-hard to tell if
• Some linear poly. satisfies 1- ? fraction of
  points.
• Every linear poly. satisfies less than 0.5 ?
  fraction of points.
• Extends to any F and d 1.
• Implies something for d lt F.
• d 2 over F2 Nothing known.

14
Our Results

• Over F2 for any d, NP-hard to tell whether
• Some linear polynomial satisfies 1- ? fraction
  of points.
• Every degree d polynomial satisfies at most 1
  -2-d ? fraction of points.
• SZ Lemma For a degree d poly P ? 0 over F2,
• Prx P(x) ? 0 2-d.

15
Our Results

• Over Fq for any d, NP-hard to tell whether
• Some linear polynomial satisfies 1- ? fraction
  of points.
• Every degree d polynomial satisfies at most
  c(d,q) ? fraction of points.
• c(d,q) Schwartz-Zippel for polynomials of total
  degree d over Fq.

16
Overview of Reduction

• Reducing from Label-Cover.
• Dictatorship Testing.
• Consistency Testing.
• Putting it all together.

17
Label Cover
1
Graph G(V,E), V n. Labels k Edges pe ½
k k Goal Find a labeling satisfying all
edges.
2
n
3

• Thm PCP Raz It is NP-hard to tell if
• Some labeling satisfies all edges.
• Every labeling satisfies ? frac. of edges.

18
The Reduction
Henceforth d 2, field F2.
X11 X12 X1k
Xn1 Xn2 Xnk
X21 X22 X2k
X31 X32 X3k
Constraints Points in 0,1nk values. Yes
Case Some L satisfies most constraints. No Case
No Q satisfies many constraints.
19
The Reduction

• If l(v) is a good labelling, then L ?v Xvl(v)
  will satisfy most points.

20
The Reduction
X11 X12 X1k
Xn1 Xn2 Xnk
X21 X22 X2k
X31 X32 X3k

• If l(v) is a good labelling, then L ?v Xvl(v)
  will satisfy most points.
• Any Q that does ¾ ? gives a labelling
  satisfying ? fraction of edges.

21
Overview of Reduction
3, 71, 99

• Dictatorship
• Q1 Q(X11,,X1k,0,..,0).
• Q1 looks like a Dictator X1j.
• Will settle for small list.

?
17, 45
Constant independent of k.
Consistency Some pair of labels in the list
satisfy ?.
22
Overview of Reduction
3, 71, 99

• Dictatorship
• Q1 Q(X11,,X1k,0,..,0).
• Q1 looks like a Dictator X1j.
• Will settle for small list.
• Can enforce this for ? frac. of vertices.

?
17, 45
Consistency Some pair of labels in the list
satisfy ?. Can enforce this for all edges.
23
Overview of Reduction
3, 71, 99
?
17, 45

• If Q does ¾ ?
• Small list for ? frac. of vertices.
• Consistency for all edges.
• Assign random labels from list.
• Satisfies constant fraction of edges.

24
Overview of Reduction

• Dictatorship Testing.
• Consistency Testing.
• Putting it all together.

25
Overview of Reduction

• Dictatorship Testing.
• Consistency Testing.
• Putting it all together.

26
Dictatorship Testing for low-degree Polynomials.

• Input Q(X1,,Xk) of degree 2.
• Goal Design a test s.t
• Every dictatorship Xi passes w.p close to 1.
• If Q does better than ¾, it is close to a
  dictatorship.
• Test Pick a random point x 2 0,1k.
• Check if Q(x) y.
• Mini reconstruction problem!

Small List
27
Dictatorship Testing for low-degree Polynomials.
All polys.
Quadratic polys.
Dictatorships
28
Dictatorship Testing Hastad, Bourgain, MOO
Hard to do with just 2 queries.
All polys.
Dictatorships
29
Dictatorship Testing for low-degree Polynomials.

• Poly. is of low degree.
• Allowed one query (!)

Quadratic polys.
Dictatorships
30
Dictatorship Test

• Dictatorship Test
• Pick ? 2 0,1k from the ?-biased distribution.
• Check if Q(?) 0.

Each ?i 1 independently. w.p ?

• Uniform dist Quadratic polys. are 31 balanced.
• ?-biased Dictatorships are highly skewed.
• Is there a converse?

(1,,1)
(0,,0)
31
Dictatorship Test

• Dictatorship Test
• Pick ? 2 0,1k from the ?-biased distribution.
• Check if Q(?) 0.

• Xi passes w.p 1- ?.
• XiXj passes w.p 1- ?2.
• X1(X1 Xk) X2(X1 ) passes w.p 1 - 2?

32
Dictatorship Test

• Dictatorship Test
• Pick ? 2 0,1k from the ?-biased distribution.
• Check if Q(?) 0.

2
Define G(Q) to be the graph of Q. Q X1X2
X2X3, G(Q)
3
1
Thm If Q passes w.p ¾ ?, then G(Q) has no
large matchings.
33
Dictatorship Test

• Dictatorship Test
• Pick ? 2 0,1k from the ?-biased distribution.
• Check if Q(?) 0.
• Thm If Q passes w.p ¾ ?, then G(Q) has no
  large matchings.

1. Large matching Independent monomials.
2. Only small matchings Small vertex cover.
X1L1 X2L2
34
Dictatorship Test

• Dictatorship Test
• Pick ? 2 0,1k from the ?-biased distribution.
• Check if Q(?) 0.
• Thm If Q does better than ¾, then G(Q) has no
  large matchings.

Xi 0 w.p 1- 2?
Xi 2R 0,1
Q
Q
c ? 0

• If G(Q) has a large matching, then Q ? 0 w.h.p.
  
• If Q ? 0, then c 1 w.p ¼ (SZ lemma).
• If Q does well, G(Q) has no large matchings.

35
Dictatorship Test

• Dictatorship Test
• Pick ? 2 0,1k from the ?-biased distribution.
• Check if Q(?) 0.
• Thm If Q does better than ¾, then G(Q) has no
  large matchings.

If G(Q) has a large matching, then Q ? 0 w.h.p.

• Each edge survives w.p 4?2.
• Events for each matching edge are independent.

36
Dictatorship Test

• Dictatorship Test
• Pick ? 2 0,1k from the ?-biased distribution.
• Check if Q(?) 0.

2
Define G(Q) to be the graph of Q. Q X1X2
X2X3, G(Q)
3
1
Thm If Q passes w.p ¾ ?, then G(Q) has no
large matchings.
Small List Vertex set of a maximal matching.
37
Overview of Reduction
3, 71, 99

• Dictatorship
• Assign a small list to a vertex.

?
17, 45
Consistency Some pair of labels in the list
satisfy ?.
38
Overview of Reduction

• Dictatorship Testing.
• Consistency Testing.
• Putting it all together.

39
Consistency Testing
l(x) l(y)
40
Consistency Testing
l(x) l(y)
X1 X2 Xk
Y1 Y2 Yk
Given Q(X1,,Xk,Y1,,Yk) s.t Q(Xi) and Q(Yj) both
pass the dict. Test. Want Q(X1,..,Xk,0,,0)
Q(0,,0,Y1,,Yk).
Test Q(r,0) Q(0,r) for r 2R 0,1k.

• Two queries!

41
Consistency via Folding
l(x) l(y)
X1 X2 Xk
Y1 Y2 Yk

• Yes case Q Xi Yi for some i.
• All of them vanish over H (r,r).
• Constant on each coset of H.
• Enforce this on Q even in the No case.

42
Consistency via Folding
Def Q is folded over subspace H µ 0,1k if Q
is constant on every coset of H. Examples Linear
polys., juntas.
Thm Q is folded over H iff for some nice basis
(?1,,?t,?1,...,?k-t), Q
R(?1,,?t) is a t-junta for t k dim(H)
In the nice basis (?1,,?t,?1,...,?k-t) ?is
coset of H, ?js position in coset.
43
Template for Folding

• Want Q folded over a subspace H.
• Compute nice basis (?i, ?j).
• Ask for R(?1,,?t).
• To test if Q(x) y
• Let x (?, ?) test R(?) y.
• For analysis Rewrite R(?) as Q(x).
• Now Q is folded.

0,1n/H
44
Consistency via Folding
l(x) l(y)
Fold over H (r,r) for r 2 0,1k. Polys. folded
over H can be written as
Q(X1,,Xk,Y1,,Yk) R(X1 Y1, , Xk Yk)
Gives Q(X1,,Xk) Q(Y1,,Yk).
45
Overview of Reduction
3, 71, 99

• Dictatorship
• Assign a small list to a vertex.

?
17, 45
Consistency Some pair of labels in the list
satisfy ?.
46
Consistency via Folding
l(x) l(y)
Fold over H (r,r) for r 2 0,1k. Polys. folded
over H can be written as
Q(X1,,Xk,Y1,,Yk) R(X1 Y1, , Xk Yk)
Gives Q(X1,,Xk) Q(Y1,,Yk). List of Xis
Vertex set of maximal matching. Every two maximal
matchings intersect.
47
Consistency Test
G(Q(Y1,,Yk))
G(Q(X1,,Xk))

• Graphs of restrictions are the same.
• Graph has no large matchings.
• List Vertex set of maximal matching.

48
Consistency Test
G(Q(Y1,,Yk))
G(Q(X1,,Xk))

• Graphs of restrictions are the same.
• Graph has no large matchings.
• List Vertex set of maximal matching.

49
Summary of Reduction

• Each constraint ? gives H? ½ 0,1nk.
• Fold over the span of all H?.
• Run Dict. test on every vertex.
• No explicit consistency tests.
• If Q passes w.p ¾ ?,
• ? fraction of vertices do well on Dict. test.
• Consistency for all edges by folding.

50
Overview of Reduction

• Dictatorship Testing.
• Consistency Testing.
• Putting it all together.

51
Projections
X11 X12 X1k
Xn1 Xn2 Xnk
X21 X22 X2k
X31 X32 X3k

• Can handle equality, permutations.
• Need perfect completeness no UGC.
• Have to deal with _at_! projections.

52
Projections
?(lu) ?(lv)
1, 2, k
1, 2, k
? k !t
s k !t
1, ,t
53
Projections
?(lu) ?(lv)
1, 2, k
1, 2, k
? k !t
s k !t
1, ,t
54
Projections
Decoding is a vertex cover for G(Qi). Need to
show that every two vertex covers intersect.
55
Projections
Do every two vertex covers of G intersect?
No
56
Projections
Do every two vertex covers of G intersect?
No
but in any three VCs, some pair intersects.
57
Hypergraph Label Cover
1, 2, k
1, 2, k
s k !t
? k !t
1, ,t
t k !t
1, 2, k
Strongly satisfied all 3 projections
agree. Weakly satisfied some 2 agree. Thm
NP-hard to tell if all edges are strongly
satsified or at most ? are weakly satified.
58
Main Theorem

• Over F2 for any d, NP-hard to tell whether
• Some linear polynomial satisfies 1- ? fraction
  of points.
• Every degree d polynomial satisfies at most 1
  -2-d ? fraction of points.

59
Better Hardness?

• Problem Can we improve soundness to 0.5 ??
• Bottleneck Dictatorship test.
• Present analysis is optimal in general
• Q (X1 .. Xk)(Xk1 X2k) passes w.p ¾.
• Can assume that Q is balanced.

60
Thank You!
61
Curve Fitting in Deep Space
62
Curve Fitting in Deep Space
63
Curve Fitting in Deep Space
64
!
65
Consistency Testing
G
G graph on k vertices. Alice and Bob have
G. Want to pick a common vertex.
66
Consistency Testing
G
G graph on k vertices. Pick a maximal matching
output a random vertex.
67
Overall Reduction
X11 X12 X1k

• Fold over consistency constraints.
• Test dictatorship on every vertex.
• If ? fraction do better than ¾, many edges are
  satisfied.

X21 X22 X2k
X31 X32 X3k