Randomness Extractors: Motivation, Applications and Constructions

1
Randomness Extractors Motivation, Applications
and Constructions

• Ronen Shaltiel
• University of Haifa

2
Outline of talk

1. Extractors as graphs with expansion properties
2. Extractors as functions which extract randomness
3. Applications
4. Explicit Constructions

3
Extractor graphs Definition NZ

• An extractor is an (unbalanced) bipartite graph
  MltltN. (e.g. MNd, Mexp( (log N)d ).
• Every vertex x on the left has D neighbors.
• the extractor is better when D is small. (e.g.
  Dpolylog N)
• Convention N2n, M2m, D2d
• 1,,N 0,1n

N0,1n
M0,1m
D edges
x
4
Extractor graphs expansion properties
Identify X with the uniform distribution on X

• (K,e)-Extractor
• ?set X of size K the dist.
• E(X,U) e-close to uniform.
• gtexpansion property
• ?set X of size K,
• G)x) (1-e)M.
• Distribution versus Set size

N0,1n
M0,1m
x
X
G(X)
K
(1-e)M
A distribution P is e-close to uniform if
P-U1 2e gt P supports 1-e elements.
5
Extractors and Expander graphs
N0,1n
N0,1n
N0,1n
M0,1m
D2d edges
X
G(X)
K
K
(1d)K
(1-e)M
Extractor
(1d)-Expander
6
Extractors and Expander graphs
N0,1n
N0,1n
N0,1n
Allows constant degree
Expands sets smaller than threshold K
Balanced graph
Absolute expansion K -gt (1d)K
M0,1m
X
G(X)
K
K
(1d)K
(1-e)M
Unbalanced graph
Relative expansion K -gt (1-e)M K/N -gt (1-e)
Requires degree log N
Expands sets larger than threshold K
Extractor
(1d)-Expander
7
Outline of talk

1. Extractors as graphs with expansion properties
2. Extractors as functions which extract randomness
3. Applications
4. Explicit Constructions

8
The initial motivation running probabilistic
algorithms with real-life sources
Successful Paradigm in CS Probabilistic
Algorithms. Probabilistic Algorithms/Protocols
Use an additional input stream of independent
coin tosses. Helpful in solving computational
problems. Where can we get random bits?

• We have access to distributions in nature
• Electric noise
• Key strokes of user
• Timing of past events
• These distributions are somewhat random but not
  truly random.
• Paradigm SV,V,VV,CG,V,CW,Z. Randomness
  Extractors
• Assumption for this talk Somewhat random
  uniform over subset of size K.

random coins
Somewhat random
Probabilistic algorithm
input
output
9
Extractors as functions that use few bits to
extract randomness

• We allow an extractor to also receive an
  additional input of (very few) random bits.
• Extractors use few random bits to extract many
  random bits from arbitrary distributions which
  contain sufficient randomness.

• Parameters (function view)
• Source length n ( log N)
• Seed length d O(log n)
• Entropy threshold k n/100
• Output length m k
• Required error e 1/100

source distribution X
Randomness
Definition A (K,e)-extractor is a function
E(x,y) s.t. For every set. X of size K, E(X,U) is
e-close to uniform. Lower bounds NZ,RT seed
length (in bits) log n Probabilistic method
S,RT Exists optimal extractor which matches
lower bound and extracts all the klog K random
bits in the source distribution. Explicit
constructions E(x,y) can be computed in
poly-time.
10
Simulating probabilistic algorithms using weak
random sources

• Goal Run prob algorithm using a somewhat random
  distribution.
• Where can we get a seed?
• Idea Go over all seeds.
• Given a source element x.
• ?y compute zy E(x,y)
• Compute Alg(input,zy)
• Answer majority vote.
• SeedO(logn) gt poly-time
• Explicit constructions.

random coins
Somewhat random
Probabilistic algorithm
input
output
11
Outline of talk

1. Extractors as graphs with expansion properties
2. Extractors as functions which extract randomness
3. Applications
4. Explicit Constructions

12
Applications

• Simulating probabilistic algorithms using weak
  sources of randomness vN,SV,V,VV,CG,V,CW,Z.
• Constructing Graphs (Expanders,
  Super-concentrators) WZ.
• Oblivious sampling S,Z.
• Constructions of various pseudorandom generators
  NZ,RR,STV,GW,MV.
• Distributed algorithms WZ,Z,RZ.
• Cryptography CDHK,L,V,DS,MST.
• Hardness of approximations Z,U,MU.
• Error correcting codes TZ.

13
Expanders that beat the eigenvalue bound WZ

• Goal Construct low deg expanders with huge
  expansion.
• Line up two low degree extractors.
• ?set X of size K ,
• G)x) (1-e)M gt M/2.
• ?sets X,X of size K
• X and X have common neighbour.
• Contract middle layer.
• Low degree (ND2/K) bipartite graph in which every
  set of size K sees N-K vertices.
• Better constructions for large K CRVW.

N0,1n
N0,1n
X
X
14
Randomness efficient (oblivious) sampling using
expanders
Random walk on constant degree expander

• Random walk variables v1..vD behave like i.i.d
• ?A of size ½M
• Hitting property
  Pr?i vi?A d 2-O(D).
• Chernoff style property
  Pri vi?A far from exp. 2-O(D).
• of random bits used for walk
  mO(D)mO(log(1/d))
• of random bits for i.i.d.
• mDm O(log(1/d))

M0,1m
v2
v3
v1
vD
15
Randomness efficient (oblivious) sampling using
extractors S

• Given parameters m,d
• Use E with KM2m, NM/d and small D.
• Choose random x mlog(1/d) random
  bits.
• Set viE(x,i)
• Ext property ? Hitting property
• ?A of size ½M
• Call x bad if E(x) inside A.
• of bad xs lt K
• Prx is bad lt K/N d

N0,1n
M0,1m
D edges
x
(1-e)M
A
bad xs
16
Every (oblivious) sampling scheme yields an
extractor

• An (oblivious) sampling scheme uses a random n
  bit string x to generated D random variables with
  Chrnoff style property.
• Thm Z The derived graph is an extractor.
• Extractors ? oblvs Sampling

N0,1n
M0,1m
D2d edges
x
17
Outline of talk

1. Extractors as graphs with expansion properties
2. Extractors as functions which extract randomness
3. Applications
4. Explicit Constructions

18
Constructions
19
Extractors from error correcting codes

• Can construct extractors from
  error-correcting code ILL,SZ,T.
• Short seed.
• Extract one additional bit
• Extractors that extract one additional bit ?
• List-decodable error-correcting codes
• Extractors that extract many bits ? codes with
  strong list-recovering properties TZ.

20
List-decodable error-correcting codes S
20 errors
List decoding
49 errors

• EC(x) is 20-decodable if for every w there is a
  unique x s.t. EC(x) differs from w in 20 of
  positions.
• EC(x) is (49,t)-list-decodable if for every w
  there are at most t xs s.t. EC(x) differs from w
  in 49 of positions.
• There are explicit constructions of such codes.

21
Extractors from list-decodable error-correcting
codes ILL,T

• Thm If EC(x) is (½-e,eK)-list-decodable then
  E(x,y)(y,EC(x)y) is a (K,2e)-extractor.
• Note E outputs its seed y. Such an extractor is
  called strong.
• E outputs only one additional output bit EC(x)y
• There are constructions of list-decodable error
  correcting codes with yO(log n).
• Strong extractors with one additional bit ?
  List-decodable error correcting codes.
• Strong extractors with many additional bits
  translate into very strong error correcting codes
  TZ.

22
Extractors from list-decodable error-correcting
codes proof

• Thm If EC(x) is (½-e,eK)-list-decodable then
  E(x,y)(y,EC(x)y) is a (K,2e)-extractor.
• Proof by contradiction.
• Let X be a distribution/set of size K s.t.
  E(X,Y)(Y,EC(X)Y) is far from uniform.
• Observation Y and EC(X)Y are both uniform.
• They are correlated.
• Exists P s.t. P(Y)EC(X)Y with prob gt ½2e.

23
Extractors from list-decodable error-correcting
codes proof II

• Thm If EC(x) is (½-e,eK)-list-decodable then
  E(x,y)(y,EC(x)y) is a (K,2e)-extractor.
• Exists P s.t.
• PrX,YP(Y)EC(X)Y gt ½2e.
• By a Markov argument For eK xs in X
• PrYP(Y)EC(x)Y gt ½e.
• Think of P as a string PyP(y).
• We have that P and EC(x) differ in ½-e
  coordinates.
• Story so far If E is bad then there is a string
  P s.t. for eK xs P and EC(x) differ in few
  coordinates.

24
Extractors from list-decodable error-correcting
codes proof III

• Thm If EC(x) is (½-e,eK)-list-decodable then
  E(x,y)(y,EC(x)y) is a (K,2e)-extractor.
• Story so far If E is bad then there is a string
  P s.t. for eK xs P and EC(x) differ in ½-e
  coordinates.

List decoding
49 errors

• By list-decoding properties of the code
• of such xs lt eK.
• Contradiction!

25
Roadmap

• Can construct extractors from
  error-correcting code.
• Short seed.
• Output Seed 1.
• Next How to extract more bits.
• General paradigm Once you construct one
  extractor you can try to boost its quality.

26
Extracting more bits WZ

• Starting point An extractor E that extracts only
  few bits.
• Idea (XE(X,Y)) contains randomness.
• We can apply E to extract randomness from
  (XE(X,Y)).
• Need a fresh seed.
• E(X(Y,Y))E(X,Y),E(X,Y)
• Extract more randomness.
• Use larger seed.

X
Z
Z
Z
Extractor
Y
Y
X
New Extractor
Y
Y


27
Trevisans extractor reducing the
seed length

• Idea Use few random bits to generate
  (correlated) seeds Y1,Y2,Y3
• Walk on expander?
• Extractor?
• Works but gives small savings.
• Trevisan use Nisan-Wigderson pseudorandom
  generator (based on combinatorial designs).
• TZS,SU Use Y,Y1,Y2,...
• (based on the STV algorithm for list-decoding
  Reed-Muller code).

X
Extractor
Y1
Y2
Y


28
The extractor designer tool kit

• Many ways to compose extractors with themselves
  and related objects.
• Arguments use entropy manipulations depend on
  function view of extractors.
• Impact on other graph construction problems
• Expander graphs (zig-zag product) RVW,CRVW.
• Ramsey graphs that beat the Frankl-Wilson
  construction BKSSW,BRSW.

29
Entropy manipulations composing two extractors
Z,NZ
Two independent sources

X2
Z
X1
Small Extractor
Y
Large Extractor


Observation Can compose a small ext. and a large
ext. and obtain ext. which inherits small seed
and large output. Paradigm If given only one
source try to convert it into two sources that
are sufficiently independent.
30
Summary Extractors are
Functions
Graphs
M0,1m
source distribution X
X
G(X)
Randomness
K2k
(1-e)M
31
Conclusion

• Unifying role of extractors
• Expanders, Oblivious samplers, Error correcting
  codes, Pseudorandom generators, hash functions
• Open problems
• More applications/connections.
• The quest for explicitly constructing the optimal
  extractor. (Current record LRVW).
• Direct and simple constructions.
• Things I didnt talk about
• Seedless extractors for special families of
  sources.

32
Thats it
33
(No Transcript)
34
Extractor graphs
N0,1n
M0,1m
D2d edges
D2d edges
x
x
35
Extractor graphs expansion
N0,1n
M0,1m
X
G(X)
K2k
(1-e)M
36
Issues in a formal definition 2. One
extractor for all sources

• Goal Design one extractor function E(x) that
  works on all sufficiently high entropy
  distributions.
• Problem Impossible to extract even 1 bit from
  distributions with n-1 bits of entropy.
• Have to settle for less!

source distribution X
Randomness
0,1n
xE(x)1
xE(x)0
Distribution X with entropy n-1 on which E(X) is
fixed
37
Definition of extractors NZ

• We allow an extractor to also receive an
  additional seed of (very few) random bits.
• Extractors use few random bits to extract many
  random bits from arbitrary distributions with
  sufficiently high entropy.

• Parameters
• Source length n
• Seed length d O(log n)
• Entropy threshold k n/100
• Output length m k
• Required error e 1/100

source distribution X
Randomness
Definition A (k,e)-extractor is a function
E(x,y) s.t. For every distribution X with
min-entropy k, E(X,Y) is e-close to
uniform. Lower bounds NZ,RT seed length log
n 2log(1/e) Probabilistic method S,RT Exists
optimal extractor which matches lower bound and
extracts kd-2log(1/e) bits.
A distribution P is e-close to uniform if
P-U1 2e gt P supports 1-e elements.
38
Extractor graphs Definition NZ

• An extractor is an (unbalanced) bipartite graph
  MltltN. (e.g. MNd, Mexp( (log N)d ).
• Every vertex x on the left has D neighbors.
• E(x)(E(x)1,..,E(x)D)
• the extractor is better when D is small. (e.g.
  Dpolylog N)
• Convention E(x,y) E(x)y

N0,1n
M0,1m
D edges
x
39
Issues in a formal definition 1. Notion of
entropy

• The source distribution X must contain
  randomness
• Necessary condition for extracting k bits
• ?x PrXx2-k
• Dfn X has min-entropy k if
• ?x PrXx2-k
• Example flat distributions X is uniformly
  distributed on a subset of size 2k.
• Every X with min-entropy k is a convex
  combination of flat distributions.

source distribution X
Randomness
0,1n
2kS
40
Noisy channels and error corrections
Goal Transmit messages using a noisy channel
errors
Guarantee x differs from x in at most (say) 20
positions. Coding Theory Encode x prior to
transmission.