Simple Extractors

1
Simple Extractors for all Min-Entropies

R.Shaltiel and C.Umans
2
Definitions

• Def (min-entropy) The min-entropy of a random
  variable X over 0, 1n is defined as
• Thus a random variable X has min-entropy at least
  k if PrXx2-k for all x. The maximum possible
  min-entropy for such a R.V. is n
• Def (statistical distance) Two distributions on
  a domain D are e-close if the probabilities they
  give to any A?D differ by at most e (namely,
  using norm 1)

3
Definitions

• Def (extractor) A (k,e)-extractor is a
  function E 0,1n ? 0,1t ? 0,1ms.t. for
  any R.V. X with min-entropy k E(X,Ut) is
  e-close to Um(where Um denotes the uniform
  distribution over 0,1m)

4
Parameters

• The relevant parameters are
• min entropy of the weak random source input k.
  Relevant values log(n)? k ? n (the seed length
  is t log(n), hence useless to consider lower
  min entropy).
• seed length t log(n) .
• Quality of the output e.
• Size of the output mf(k). The optimum is mk.

5
Extractors
High Min-Entropy distribution
Uniform-distribution seed
2t
2n
2m
E
Close to uniform output
6
Next Bit Predictors

• Claim to prove E is an extractor, it suffices to
  prove that for all 0ltiltm1 and all predictors
  f0,1i-1?0,1
• Proof Assume E is not an extractor then exists
  a distribution s.t. X s.t. E(X,Ut) is not e-close
  to Um, that is

7
Proof

• Now define the following hybrid distributions

8
Proof

• Summing the probabilities for the event
  corresponding to the set A for all distributions
  yields
• And because ?ai ?ai there exists an index
  0ltiltm1 for which

9
The Predictor

• We now define a function f0,1i-1 ? 0,1 that
  can predict the ith bit with probability at
  least ½e/m (a next bit predictor)
• The function f uniformly and independently draws
  the bits yi,,ym and outputs
• Note the above definition is not constructive,
  as A is not known!

10
Proof

• And f is indeed a next bit predictor
• Q.E.D.

11
Basic Example Safra, Ta-Shma, Zukerman

• Construction
• Let BCF?0,1s be a (inefficient) binary-code
• Given
• x, a weak random source, interpreted as a
  polynomial xF2?F and
• s, a seed, interpreted as a random point (a,b),
  and an index j to a binary code.
• Def

12
Basic Example Illustration of Construction

• x ? x, s ((a,b), 2)
• E(x,s)01001

(a,b)
(inefficient) binary code
13
Basic Example Proof Sketch

• Assume, by way of contradictionexists a next
  bit predicator function f.
• Next, show a reconstruction function R
• Conclude, a contradiction!(to the min-entropy
  assumption of X)

14
Basic Example Reconstruction Function
h n1/2 j lgn m desired entropy
Random line
advice Few red points amjO(h)
Repeat using the new points, until all Fd is
evaluated
List decoding by the predictor f
Resolve into one value on the line
15
Counting Argument

• For Y?? X, let ?(Y)?y?YPry (the weight of Y)
• Let R0,1a ? 0,1n, s.t. PrxX?z R(z)x ?
  1/2
• (for a uniform X, R(S) ? X/2 )
• For an arbitrary distribution X, ?(R(S)) ? ?(X)/2
  
• Let X min-entropy ? k,
• then ?(R(S)) ? 2a-k (there are at most 2a
  strings in R(S), and ?x?X Prx ? 2-k)
• and therefore k ? a - log2(1/2) (1 ?(X) ?
  ?(R(S)) ?2 ? 2a-k ?2-1 ? a-k hence k ? a1)

2nX
R(S)
R
2aS
16
Problems with Safra, Ta-Shma, Zukerman

• Curse of dimensionality - too many
  lines!Solution generator matrix.

17
Next-q-it List-Predictor

• f is allowed to output a small list of l possible
  next elements

18
q-ary Extractor

• Def Let F be a field with q elements.
• A (k, l) q-ary extractor is a function E 0,1n
  ? 0,1t ?Fms.t. for all R.V. X with min-entropy
  k
• and all 0ltiltm
• and all list-predictors fFi-1 ? Fl

19
Generator Matrix

• Def Define the generator matrix for the vector
  space Fd as a matrix A?dd, s.t. for any non-zero
  vector v?Fd
• (that is, any vector 0?v?Fd multiplied by all
  powers of A generates the entire vector space Fd
  except for 0)
• Lemma Such a generator matrix exists and can be
  found in time qO(d).

20
Construction

• Let F be a field with q elements,
• Let Fd be a vector space over F.
• Let h be the smallest integer s.t.
• For x? 0,1n, let x denote the unique d-variate
  polynomial of total degree h-1 whose coefficients
  are specified by x.

21
Construction

• The definition of the q-ary extractor E
  0,1n ? 0,1d log q ? Fm

seed, interpreted as a vector v? Fd
Generator matrix
22
Main Theorem

• Thm For any n,q,d and h as previously defined,
  E is a (k, l) q-ary extractor if
• Alternatively, E is a (k, l) q-ary extractor if

23
Whats Ahead

• Proving existence of a generator matrix
• How the counting argument works
• The reconstruction paradigm
• Basic example Safra, Ta-Shma, Zukerman
• Proof of the main theorem
• From extractors to PRGs

24
Extension Fields

• A field F2 is called an extension of another
  field F if F is contained in F2 as a subfield.
• Thm For every power pk (p prime, kgt0) there is
  a unique (up to isomorphism) finite field
  containing pk elements. These fields are denoted
  GF(pk).All finite fields cardinality have that
  form.
• Def A polynomial is called irreducible in GF(p)
  if it does not factor over GF(p)
• Thm Let f(x) be an irreducible polynomial of
  degree k over GF(p). The finite field GF(pk) can
  be constructed using the set of degree k-1
  polynomials over Zp, with addition and
  multiplication carried out modulo f(x)

25
Extension Fields - Example

• Construct GF(25) as follows
• Let the irreducible polynomial be
• Represent every k degree polynomial as a vector
  of k1 coefficient
• Addition over this field

26
Extension Fields - Example

• And multiplication
• And now modulo the irreducible polynomial

27
Generator Matrix Existence Proof

• Denote by GF(qd) the multiplicative group of the
  Galois Field GF(qd).
• This multiplicative group of the Galois Field is
  cyclic, and thus has a generator g
• Let j be the natural isomorphism between the
  Galois Field GF(qd) and the vector space Fd,
  which matches a polynomial with its vector of
  coefficients

28
Generator Matrix Existence Proof

• Now define the generator matrix A of Fd as the
  linear transformation that corresponds to
  multiplication by the generator in GF(qd)
• A is a linear transformation because of the
  distributive property of both the vector space
  and the field GF(qd), according to the
  isomorphism properties

29
Generator Matrix Existence Proof

• It remains to show that the generator matrix A of
  Fd can be found in time qO(d).
• And indeed
• The Galois Field GF(qd) can be constructed in
  time qO(d) using an irreducible polynomial of
  degree d over the field Zq (and such a polynomial
  can also be found in time qO(d) by exhaustive
  search).
• The generator of GF(qd) can be found in time
  qO(d) by exhaustive search
• Using the generator, for any basis of Fd, one can
  construct d independent equations so as to find
  the linear transformation A. This linear equation
  system is also solvable in time qO(d) .

30
Reconstruction Proof Paradigm

• Proof sketch
• For a certain R.V. X with min-entropy at least k,
  
• assume a function f that violates the properties
  of a q-ary extractor,
• construct another function, R 0,1a ? 0,1n,
  the reconstruction function.
• This function, using f as a procedure, has the
  property that
• Applying the counting argument, this is a
  contradiction to the assumption that X has
  min-entropy at least k

31
Proof Sketch

• Let X be a random variable with min-entropy at
  least k
• Assume, by way of contradictionexists a next
  bit predicator function f.
• Next, show a reconstruction function R
• Conclude, a contradiction!(to the min-entropy
  assumption of X)

32
Main Lemma

• Lemma Let n,q,d,h be as in the main theorem.
  There exists a probabilistic function
  R0,1a?0,1n with a O(mhd logq) such that
  for every x on which
• The following holds (the probability is over the
  random coins of R)

33
The Reconstruction Function (R)

• Task allow many strings x in the support of X to
  be reconstructed from very short advice strings.
• Outlines
• Use f in a sequence of prediction steps to
  evaluate z on all points of Fd,.
• Interpolate to recover coefficients of z,
• which gives x
• Next We Show there exists a sequence of
  prediction steps that works for many x in the
  support of X and requires few advice strings

34
Curves

• Let rQ(d),
• Pick random vectors and values
• 2r random points y1,,y2r?Fd, and
• 2r values t1,,t2r?F, and
• Define degree 2r-1 polynomials p1,p2
• p1F?Fd defined by p1(ti)yi, ?i1,..,2r.
• p2F?Fd defined by p2(ti)Ayi, ?i1,..,r, and
  p2(ti)yi, ?ir1,..,2r.
• Define vector sets P1p1(z)z?F and
  P2p2(z)z?F
• ?igt0 define P2i1AP2i-1 and P2i2AP2i(Pi,
  the sequence of prediction steps are low-degree
  curves in Fd, chosen using the coin tosses of R)

35
Curves
Fd
F
36
Simple Observations

• A is non-singular linear-transform, hence ?i
• Pi is 2r-wise independent collection of points
• Pi and Pi1 intersect at r random points
• zPi is a univariate polynomial of degree at most
  2hr.
• Given evaluation of z on Av,A2v,,Amv, we may use
  the predictor function f to predict z(Am1v) to
  within l values.
• We need advice string 2hr coefficients of zPi
  for i1,,m. (length at most mhr log q a)

37
Using N.B.P.
Cannot resolve into one value!
Fd
F
38
Using N.B.P.
Can resolve into one value using the second curve!
Fd
F
39
Using N.B.P.
Can resolve into one value using the second curve!
Fd
F
40
Main Lemma Proof Cont.

• Claim with probability at least 1-1/8qd over the
  coins tosses of R
• Proof We use the following tail bound
• Let tgt4 be an even integer, and X1,,Xn be
  t-wise independent R.V. with values in 0,1. Let
  X?Xi, ?EX, and Agt0. Then

41
Main Lemma Proof Cont.

• According to the next bit predictor, the
  probability for successful prediction is at least
  1/2vl.
• In the ith iteration we make q predictions (as
  many points as there are on the curve).
• Using the tail bounds provides the result.
• Q.E.D (of the claim).
• Main Lemma Proof (cont.) Therefore, w.h.p. there
  are at least q/4vl evaluations points of Pi that
  agree with the degree 2hr polynomial on the ith
  curve (out of a total of at most lq).

42
Main Lemma Proof Cont.

• A list decoding bound given n distinct pairs
  (xi,yi) in field F and Parameters k and d, with
  kgt(2dn)1/2, There are at most 2n/k degree d
  polynomials g such that g(xi)yi for at least k
  pairs.
• Furthermore, a list of all such polynomials can
  be computed in time poly(n,logF).
• Using this bound and the previous claim, at most
  8l3/2 degree 2rh polynomials agree on this number
  of points (q/4vl ).

43
Lemma Proof Cont.

• Now,
• Pi intersect Pi-1 at r random positions, and
• we know the evaluation of z at the points in Pi-1
• Two degree 2rh polynomials can agree on at most
  2rh/q fraction of their points,
• So the probability that an incorrect polynomial
  among our candidates agrees on all r random
  points in at most

44
Main Lemma Proof Cont.

• So, with probability at least we learn points
  Pi successfully.
• After 2qd prediction steps, we have learned z on
  Fd\0 (since A is a generator of Fd\0)
• by the union bound, the probability that every
  step of the reconstruction is successful is at
  least ½.
• Q.E.D (main lemma)

45
Proof of Main Theorem Cont.

• First,
• By averaging argument
• Therefore, there must be a fixing of the coins of
  R, such that

46
Using N.B.P. Take 2
Unse N.B.P over all points in F, so that we get
enough good evaluation
Fd
F
47
Proof of Main Theorem Cont.

• According to the counting argument, this implies
  that
• Recall that rQ(d).
• A contradiction to the parameter choice
• Q.E.D (main theorem)!

48
(No Transcript)
49
From q-ary extractors to (regular) extractors

• The simple technique - using error correcting
  codes
• Lemma Let F be a field with q elements. Let
  C0,1klog(q)?0,1n be a binary error
  correcting code with distance at least 0.5-O(?2)
  . If
• E 0,1n 0,1t -gt Fm is a (k,O(r)) q-ary
  extractor, then
• E 0,1n 0,1tlog(n) -gt Fm defined by

Is a (k,rm) binary extractor.
50
From q-ary extractors to (regular) extractors

• A more complex transformation from q-ary
  extractors to binary extractors achieves the
  following parameters
• Thm Let F be a field with qlt2m elements. There
  is a polynomial time computable function

Such that for any (k,r) q-ary extractor E,
E(x(y,j))B(E(xy),j) is a (k,r logm) binary
extractor.
51
From q-ary extractors to (regular) extractors

• The last theorem allows using theorem 1 for ?
  O(e/logm) , and implies a (k,e) extractor with
  seed length tO(log n) and output length mk/(log
  n)O(1)

52
Extractor ? PRG

• Identify
• string x?0,1log n with the
• function x0,1log n?0,1 by setting x(i)xi
• Denote by S(x) the size of the smallest circuit
  computing function x
• Def (PRG) an ?-PRG for size s is a function
  G0,1t?0,1m with the following property
  ?1?i?m and all function f0,1i-1?0,1i with
  size s circuits,
• Prf(G(Ut)1...i-1)G(Ut)i ? ½ ?/m
• This imply
• for all size s-O(1) circuits C
• PrC(G(Ut))1 PrC(Um)1? ?

53
q-ary PRG

• Def (q-ary PRG) Let F be the field with q
  elements. A ?-q-ary PRG for size s is a function
  G0,1t?Fm with the following property ?1?i?m
  and all function fFi-1?F(?-2) with size s
  circuits,
• Pr?j f(G(Ut)1...i-1)jG(Ut)i ? ?
• Fact O(?)-q-ary PRG for size s can be
  transformed into (regular) m?-PRG for size not
  much smaller than s

54
The Construction
Note Gx(j) corresponds to using our q-ary
extractor construction with the successor
function Amj
We show x is hard ? at least one Gx(j) is a
q-ary PRG

• Plan for building a PRG Gx0,1t ? 0,1m
• use a hard function x0,1log n ? 0,1
• let z be the low-degree extension of x
• obtain l candidate PRGs, where ld(log q / log
  m) as followsFor 0?jltl define Gx(j)0,1d log
  q ? Fm byGx(j)(v) z(A1?mjv) ? z(A2?mjv) ?...?
  z(AM?mjv)where A is a generator of Fd\0

55
Getting into Details
Note Fd is a subset of Fd
think of Fd as both a vector space and the
extension field of F

• perhaps we should just say immediate from the
  correspondence between the cyclic group GF(qd)
  and Fd\0 ??? otherwise in details we may say
• Proof
• There exists a natural correspondence between Fd
  and GF(qd), and between Fd and GF(hd),
• GF(qd) is cyclic of order qd-1, i.e. there exists
  a generator g
• gp generates the unique subgroup of order hd-1,
  the multiplicative group of GF(hd).
• A and A are the linear transforms corresponding
  to g and gp respectively.

• Let F be a subfield of F of size h
• Lemma there exist invertible d?d matrices A and
  A with entries from F which satisfy
• ? v?Fd s.t. v?0, AiviFd\0
• ? v?Fd s.t. v?0, AiviFd\0
• AAp for p(qd-1)/(hd-1)
• A and A can be found in time qO(d)

56

• since hdgtn, there are enough slots to embed all
  x in a d dimensional cube of size hd
• and since A generates Fd\0, indeed x is
  embedded in a d dimensional cube of size hd

• Note h denotes the degree in individual
  variables, and the total degree is at most hd

• The computation of z from x can be done in
  poly(n,qd)qO(d) time

• require hdgtn
• Define z as follows z(Ai1)x(i), where 1 is the
  all 1 vector (low degree extension).
• Recall For 0?jltl define Gx(j)0,1d log q ? Fm
  byGx(j)(v) z(A1?mjv) ? z(A2?mjv) ?...?
  z(AM?mjv
• Theorem (PRG main) for every n,d, and h
  satisfying hdgtn, at least one of Gx(j) is an
  ?-q-ary PRG for size ?(?-4 h d2 log2q).
  Furthermore, all the Gx(j)s are computable in
  time poly(qd,n) with oracle access to x.

57

• ??????

58
(No Transcript)
59
Extension Field

• Def if F is a subset of E, then we say that E is
  an extension field of F.
• Lemma let
• E be an extension field of F,
• f(x) be a polynomial over F (i.e. f(x)?FX),
• c?E,
• then f(x)?f(c) is an homomorphism of FX into E.

60
Construction of the Galois Field GF(qd)

• Thm let p(x) be irreducible in FX, then there
  exists E, an extension field of F, where there
  exists a root of p(x).
• Proof Sketch
• add a ?? (a new element) to F.? is to be a root
  of p(x).
• In F? (polynomials with variable ?)

61

• Example
• Freals
• p(x)x21