Extracting Randomness From Few Independent Sources

1
Extracting Randomness From Few Independent Sources

• Boaz Barak, IASRussell Impagliazzo, UCSDAvi
  Wigderson, IAS

2
Plan
1. Discuss problem and model
2. State our result
3. Introduce main tool Thm by BKT,K
4. Prove our main theorem.
3
Randomness Extraction
Randomness is central to CS (c.f., randomized
algorithms, cryptography, distributed
computing)
How do you execute randomized algorithms and
protocols?
Solution sample some random physical data
(coin tossing, thermal noise, hard disk
movement,)
Problem data from physical sources is not a
sequence of ideal coin tosses.
4
Randomness Extractors
Definition E0,1n?0,10.1k is an extractor
if 8 r.v. X with entropy k , E(X) is close to
U0.1k
Idea
X
E
randomized algorithm / protocol
extractor
uniform output
high entropy data
5
Randomness Extractors
Definition E0,1n?0,10.1k is an extractor
if 8 r.v. X with entropy k , E(X) is close to
U0.1k
Problem No extractor exists.
Thm 8 E0,1n?0,10.1k theres a r.v. X w/
entropy n-1 s.t. first bit of E(X) is constant.
Proof Sketch Assume wlog x E1(x)0
2n/2 let X be the uniform dist over this set.
6
Solution 1 Seeded Extractors
Def E0,1n0,1d?0,10.1k is a (seeded)
extractor if 8 r.v. X w/ min-entropy k
E(X,Ud) U0.1k 1 lt 1/100 .
Definition E0,1n?0,10.1k is an extractor
if 8 r.v. X with entropy k , E(X) is close to
U0.1k
Many exciting results, applications and
connections Z,NZ,Ta,Tr,RSW,STV,TSZ,SU,.
Thm LRVW For every n,k theres a seeded
extractor with dO(log n)
Corollary Any probabilistic algorithm can be
simulated w/ weak random source polynomial
overhead.
7
Solution 1 Seeded Extractors
Thm LRVW For every n,k theres a seeded
extractor with dO(log n)
Corollary Any probabilistic algorithm can be
simulated w/ weak random source polynomial
overhead.
Question What about other uses of randomness?
For example, can we use this for cryptography?
Answer No! For example, if we concatenate
encryptions according to all possible seeds this
wont be secure!
Need to use seedless extractors!
8
Seedless Extractors
Idea Bypass impossibility result by making
additional assumption on the high entropy input.
Long history and many results vN,P,B,SV,CW,TV,KZ,
..
In this work We assume that input comes from few
independent distributions (CG).
Def E0,1nc?0,10.1k is a c-sample extractor
if 8 ind. r.v. X1,,Xc w/ min-entropy k
E(X1,,Xc) U0.1k 1 lt 1/100
Motivation mathematically clean and plausible
model.
9
Def E0,1nc?0,10.1k is a c-sample extractor
if 8 ind. r.v. X1,,Xc w/ min-entropy k
E(X1,,Xc) U0.1k 1 lt 2-?(k)
Optimal (non-explicit) construction c2 ,
every k?(log n)
Previous best explicit construction
SV,V,CG,ER,DEOR c2 , every k(1?)n/2
Obtained by variants of following 1-bit output
extractor E(x,y) ltx,ygt
Problematic, since natural entropy sources often
have entropy less than n/2.
10
Def E0,1nc?0,10.1k is a c-sample extractor
if 8 ind. r.v. X1,,Xc w/ min-entropy k
E(X1,,Xc) U0.1k 1 lt 2-?(k)
Optimal (non-explicit) construction c2 ,
every k?(log n)
Previous best explicit construction
SV,V,CG,ER,DEOR c2 , every k(1?)n/2
Our Result For every ?gt0 cpoly(1/?) , k?n
11
Plan
Main Thm 8 ?gt0 9 cpoly(1/?) and poly-time
E0,1nc?0,1n s.t. if 8 ind. r.v. X1,,Xc w/
min-entropy ?n E(X1,,Xc) Un 1 lt 2-?(n)
1. Discuss problem and model
2. State our result
3. Introduce main tool Thm by BKT,K
Show BKT (almost) immediately implies dispersers.
4. Prove our main theorem.
12
Main Thm 8 ?gt0 9 cpoly(1/?) and poly-time
E0,1nc?0,1n s.t. if 8 ind. r.v. X1,,Xc w/
min-entropy ?n E(X1,,Xc) Un 1 lt 2-?(n)
Our main tool is the following result
Thm 1 BKT,K 9 absolute constant ?gt0 s.t. for
prime field F, and set AµF, max AA , A
A min A1? , F
1. Finite field analog of a theorem by ES.2.
Note Thm 1 would be false if F had non-trivial
subfields.3. Note if A is arithmetic (resp.
geometric) sequence, then AA (resp. A
A) is small.
AA ab a,b 2 A A A ab a,b 2
A
13
Thm 1 BKT,K 9 absolute constant ?gt0 s.t. for
prime field F, and set AµF, max AA , A
A A1?
How is this related to extractors?
Disperser Lemma BKT Let ?gt0 and F a prime
field, then 9cpoly(1/?) and poly-time EFc?F
s.t. if X1,,XcµF satisfy Xi F?, then
E(X1,,Xc) F
Corollary Identify 0,1n w/ prime field F of
size 2n. Then, we get poly-time E s.t. if r.v.s
X1,,Xc have entropy ?n, then
SuppE(X1,,Xc)0,1nThis is called a
disperser.
14
Thm 1 BKT,K 9 absolute constant ?gt0 s.t. for
prime field F, and set AµF, max AA , A
A A1?
Thm 1 BKT,K 9 absolute constant ?gt0 s.t. for
prime field F, and sets A,B,CµF, (with
ABC) ABC A1?
How is this related to extractors?
Disperser Lemma BKT Let ?gt0 and F a prime
field, then 9cpoly(1/?) and poly-time EFc?F
s.t. if X1,,XcµF satisfy Xi F?, then
E(X1,,Xc) F
Proof Use lemma of Rusza to get asymmetric
version of Thm 1.
Lemma R,N If A,B µG w/ ABM, and AB
M1?, then AA M1O(?)
We let E be recursive application of a,b,c?abc
with depth O(log(1/?)).
A A large ) A B large ) ABC
largeAA large ) AC large ) ABC large
15
Thm 1 BKT,K 9 absolute constant ?gt0 s.t. for
prime field F, and sets A,B,CµF, (with
ABC) ABC A1?

.
.

.
.
.
.
.
.
.
a1 , a2,
apoly(1/delta)
16
Plan
1. Discuss problem and model
2. State our result
3. Introduce main tool Thm by BKT,K
Show BKT (almost) immediately implies dispersers.
4. Prove our main theorem.
17
Distributional Version of BKT
Thm 1 BKT,K 9 absolute constant ?gt0 s.t. for
prime field F, and sets A,B,CµF, (with
ABC) ABC A1?
Our Main Lemma 9 absolute constant ?gt0 s.t. for
prime field F, and distributions A,B,CµF, (with
H(A)H(B)H(C)), the distribution ABC is
2-?H(A) close to having entropy (1?)H(A)
( The distribution ABC assigns to x the prob
that abcx with a2RA , b2RB , c2RC )
Main Lemma ) Main Theorem.
18
Our Main Lemma 9 absolute constant ?gt0 s.t. for
prime field F, and distributions A,B,CµF, (with
H(A)H(B)H(C)), the distribution ABC is
2-?H(A) close to having entropy (1?)H(A)
Main Lemma ) Main Theorem.
19
Plan
Our Main Lemma 9 absolute constant ?gt0 s.t. for
prime field F, and distributions A,B,CµF, (with
H(A)H(B)H(C)), the distribution ABC is
2-?H(A) close to having entropy (1?)H(A)
Prove Main Lemma by reducing to BKT.We use
magic lemmas of Gowers Ruszain the reduction.
20
Detailed Plan
Our Main Lemma 9 absolute constant ?gt0 s.t. for
prime field F, and distributions A,B,CµF, (with
H(A)H(B)H(C)), the distribution ABC is
2-?H(A) close to having entropy (1?)H(A)
1. Introduce collision probability a different
entropy measure.
2. Rephrase Main Lemma in terms of C.P.
3. Show naïve approach to proving, and show
counterexample
4. Use Gowers Ruszas lemmas to show
counterexample essentially captures all cases
21
Collision Probability
cp(X) Prx,x?X x x ?x px2
Fact 1 If H(X)k then cp(X)2-k
Fact 2 If cp(X)2-k(1?) then is 2-?k/2 close
to having min-entropy at least k(1?/2).
Notation If D is r.v., then the 2-entropy of D
is H2(D) log(1/cp(D))
Fact 1 Fact 2 ) H2(D) H(D)
Fact 3 If X is convex combination of X1,,Xm
then cp(X) max cp(X1), , cp(Xm)
22
Main Lemma 9 ?gt0 s.t. for prime field F,
dists A,B,CµF, (with H(A)H(B)H(C), the
distribution ABC is 2-?H(A) close to entropy
(1?)H(A)
Main Lemma (CP version) 9 ?gt0 s.t. for prime
field F, and sets A,B,CµF (with ABC ),
the distribution ABC is A-? close to
having 2-entropy (1?)log A
Thus, it is sufficient to prove CP version.
23
Detailed Plan
Main Lemma (CP version) 9 ?gt0 s.t. for prime
field F, and sets A,B,CµF (with ABC ),
the distribution ABC is A-? close to
having 2-entropy (1?)log A
1. Introduce collision probability a different
entropy measure.
2. Rephrase Main Lemma in terms of C.P.
3. Show naïve approach to proving, and show
counterexample
4. Use Gowers and Ruszas lemmas to show
counterexample essentially captures all cases
24
Naïve Approach
Prove direct analog to BKT
Conjecture 9 ?gt0 s.t. for prime F, and set
AµF max H2(AA) , H2(A A) (1?)logA
Counter Example AAG AA AG - geometric seq.
AA - (disjoint) arithmetic seq.
cp(AA),cp(AA)1/10A hence H2(AA),
H2(AA)logAO(1)
However, in this case H2(A AA) (1?)log A
25
Naïve Approach
Counter Example AAG AA AG - geometric
seq.AA - (disjoint) arithmetic seq.
Claim H2(AA A) (1?)log A
Sketch AAA is convex comb of AA AA and
AGAA.
cp(AA AA) cp(AAA) which is low since A is
an arithmetic seq
AGAA is convex comb of AGaA but cp(AGaA)
is low since AGa is a geometric seq
26
Detailed Plan
Main Lemma 9 absolute constant ?gt0 s.t. for
prime field F, and sets A,B,CµF (with
ABC ), the distribution ABC is A-?
close to having c.p. A-(1?)
1. Introduce collision probability a different
entropy measure.
2. Rephrase Main Lemma in terms of C.P.
3. Show naïve approach to proving, and show
counterexample
4. Use Gowers and Ruszas lemmas to show
counterexample essentially captures all cases
27
Proof of Main Lemma
Main Lemma (CP version) 9 absolute constant ?gt0
s.t. for prime field F, and sets A,B,CµF (with
ABC ), the distribution ABC is A-?
close to having 2-entropy (1?)log A
(Loose) Notations
Let MABC and fix some ?gt0 (e.g., BKTs ?
divided by 100)
A number M1? is called largeA number
M1-?(?) is called not-too-small A distribution
D has high 2-entropy if H2(D) (1?)log M
Our Goal Prove that ABC is close to having
high 2-entropy.(i.e., it is close to having c.p.
1/M1?)
28
Tools
Thm 1 BKT,K If AµF is not too small then
either AA or AA is large.
Lemma R,N If AA is large then AB is
large.
Magic Lemma G,BS Either H2(AB) is large or
9 not-too-small subsets AµA, BµB s.t. AB
is not large.
29
A First Distributional Analog
Cor BKTR If 9 not-too-small B s.t. AB is
not large then AC is large 8 not-too-small C.
Proof AB is not large ) AA is not large
R ) AA is large BKT ) AC is large R.
Natural Analog If 9 not-too-small B s.t. H2(AB)
is not large then H2(AC) is large 8
not-too-small C.
This is false e.g., ABCAG AA
However, the following is true
PF Lemma If 9 not-too-small B s.t. AB is not
large then H2(AC) is large 8 not-too-small C.
30
PF Lemma If 9 not-too-small B s.t. AB is not
large then H2(AC) is large 8 not-too-small C.
Proof If H2(AC) is not large then by Gowerss
Lemma 9 not-too-small AµA, CµC s.t. AC is
not large.
By Ruszas lemma AA is not large ) by BKT
AA is large.
Since AµA , AA is also large ) by Ruszas
lemma AB is large contradiction!
Def A not-too-small set AµF is plus friendly
if H2(AC) is large 8 not-too-small set C.
1. A plus-friendly, b2F ) Ab plus-friendly.2.
A , A plus-friendly, disjoint ) AA
plus-friendly.
31
Our Goal Prove ABC close to having low c.p..
Assume H2(ABC) not large.Well show AAA
s.t. A,Aare disjoint and
1) A is plus friendly (or A is empty)
2) H2(A B) is large (or A M1-?)
12 ) contradiction since ABC is M-? close to
convex comb of ABC and ABC, but
a) H2(ABC) is large since convex comb of
AbC and Ab is plus-friendly.
b) H2(ABC) is large since convex comb of
ABc which are permutations of AB.
32
Our Goal Prove ABC close to having low c.p..
Assume H2(ABC) not large.Well show AAA
s.t. A,A disjoint and
1) A is plus friendly (or A is empty)
2) H2(A B) is large (or A M1-?)
We build partition iteratively. Initially A ,
AA.
Assume A is not-too-small (o/w were done).
Assume H2(AB) is not large (o/w were done).
By Gowers lemma, 9 not-too-small subsets AµA,
BµB s.t. AB not large.
By PF Lemma A is plus-friendly, remove A from
A and add it to A.
33
This finishes the proof of the Main Lemma and
hence the Main Theorem.
Main Lemma 9 absolute constant ?gt0 s.t. for
prime field F, and distributions A,B,CµF, (with
H(A)H(B)H(C)lt0.8logF), the distribution ABC
is 2-?H(A) close to having entropy (1?)H(A)
Main Thm 8 ?gt0 9 cpoly(1/?) and poly-time
E0,1nc?0,1n s.t. if 8 ind. r.v. X1,,Xc w/
min-entropy ?n E(X1,,Xc) Un 1 lt 2-?(n)
2-10n
34
Another Result

• A disperser for the case that all samplescome
  from same distribution, which only requires ?(log
  n) entropy (using EH).

35
Open Problems

• Extractors/Dispersers with lower entropy
  requirement (kn?(1) or even k?(log n) )
• Improvement for the case of two samples (related
  to constructing Ramsey graphs).
• More applications of results/techniques.