New Coins from old:

1
New Coins from old Computing with unknown bias
Elchanan Mossel, U.C. Berkeley mossel_at_stat.berkele
y.edu, http//www.cs.berkeley.edu/mossel/ Joint
work with Yuval Peres, U.C. Berkeley peres_at_stat.be
rkeley.edu, http//www.stat.berkeley.edu/peres/
Supported by Microsoft Research and the Miller
institute
2
von Neumann extractor (1951)

• Given a sequence of i.i.d. coins (p,1-p) coins
  want to toss a fair coin.
• 0 lt p lt 1 is unknown.
• Want
• Efficient in randomness (preserves entropy)
• Computationally simple (finite automaton)
• Efficient in time (small expected running time)

3
von Neumann extractor (1951)

• P01 p(1-p) P10.
• Map 01 ? 0, 10 ? 1 and delete 00,11.
• Properties
• Linear time.
• Rate p(1-p) (compared to H(p)).
• Easy to generalize to Markov chains.
• Can implement via finite automata.

4
Since von Neumann

• In information theory extracting (1 e) of
  entropy
• Elias (72) block construction.
• Peres (92) iterative construction.
• In computer science, named extractors.
• General distributions with bound on (min)
  entropy.
• Extra randomness needed.
• Nearly optimal constructions in recent years.

5
Our model

• Input is i.i.d. (p,1-p) coins, where 0 lt p lt 1 is
  unknown.
• Want to Output (f(p), 1 f(p)) coin.
• Simulation power?
• Unbounded (infinite memory),
• Turing machines,
• Pushdown automata,
• Finite automata.
• Asked by S. Asmussen and J. Propp.

6
Our model

• Keane OBrien If there are no computational
  restriction can simulate any continuous function
  f (0,1) ? (0,1).
• Ergodic theory techniques.
• Need min(f(x),1-f(x)) gt min(x,1-x)n, for some n.

7
Coins via finite and pushdown automata

• Which functions can be simulated via finite
  automata? Which functions can be simulated via
  push-down automata?
• Examples
• f(p) p2 ?
• f(p) p/2 ?
• f(p) 2p for 0 lt p lt 1/4 ?
• f(p) ?p ?
• f(p) p2 / (p2 (1 p)2) ?
• f(p) p/6 ?

8
Exact simulation, computability, etc.

• Theory of exact simulation simulating
  complicated distributions from simples ones.
• Here both are simple. But, some examples
• Simulating percolation configuration on the
  triangular lattice from a configuration on the
  square lattice with same distance 2 connectivity
  functions (p unknown!).
• Given a sequence yi AND zi where yi, zi are
  i.i.d. with unkown mean p, find xi i.i.d. with
  mean p.
• Theory of computability the computation of real
  functions.
• Trivial for every computable constant 0 lt q lt
  1, the function f(p) q can be simulated via a
  Turing machine.

9
Coins via finite automata
10
Coins via finite automata

• f(p) p2 V
• f(p) p/2 V
• f(p) 2p for 0 lt p lt 1/4 X
• f(p) ?p X
• f(p) p2 / (p2 (1 p)2) V
• f(p) p/6 X

11
Coins via pushdown automata

• f(p) 2p for 0 lt p lt 1/4 ??
• f(p) ?p V
• f(p) p/6 X

• Open problem Does a converse hold?
• If f (0,1) ? (0,1) is algebraic over Q, does
  there exist a push down automata simulating f?

12
Pushdown automaton simulating ?p

• Take a random walk on the ladder with Pup/down
  (1 p)/2, Pleft/right p.
• Let t be the probability starting at (0,1) that
  the 1st hitting of level 0 is (0,0).
• t (1 p)/2 p (1 t) (1
  p)(t2 (1-t)2)/2
• Can simulate the random walk with a push down
  automaton and t (1 -
  ?p)/(1-p)
• With another coin toss can get ?p

½ - p/2
½ - p/2
p
½ - p/2
½ - p/2
13
Finite automaton implies rationality
14
Finite automaton implies rationality

• Proof 1
• Let F(p s) be the probability to stop at 1 given
  that the current state is s.
• F(p s) p F(p d(s,1))
• (1-p) F(p d(s,0)),
• and f(p) F(p s0).
• Equations determine F by maximum principle for
  harmonic functions on directed graphs.
• By Cramers rule F(p) g(p) / h(p).

15
Finite automaton implies rationality

• Proof 2 (Chomsky-Schützenberger)
• If L is a regular language, then

• is a rational function in the non-commutative
  variables x0 and x1.
• Let L be the language where the automaton stops
  at 1.
• Looking at the homomorphism 0 ? 1-p, 1 ? p, we
  see that

is a rational function.
16
Algebraic properties of Pushdown automata

• Our results for push-down automata do not follow
  from Chomsky-Schützenberger theorem.
• Instead, we prove that f(p) is determined by a
  set of polynomial equations.
• The proof uses the fact that bounded harmonic
  functions on recurrent infinite graphs are
  determined by boundary values.
• Then we invoke the following result due to Hillar
  (2002).

17
Hillar 2002
18
Rationality implies simulation by finite automata

• Definition block simulation of f, is given by
• A0, A1 disjoint subsets of 0,1k, A' 0,1k \
  (A0 ? A1), and the following procedure.
• Read a k bit string w.
• For i1,2, if w ? Ai, output i.
• Otherwise, discard w and reads a new k bit
  string.
• Block simulation ? automata simulation, and

19
Rationality implies simulation by finite automata
Already seen I ? II ? III
20
Rationality implies simulation by finite automata
Need to show If f(p) g(p)/h(p), where g,h ?
Zp, and 0 lt f(p) lt 1, then f(p) is block
simulated. Easier to show
Remark claim doesnt cover (p2 2p(1-p)
2(1-p)2)/2
21
Rationality implies simulation by finite automata
Proof
1100y0yr
if 0 ? y0yr ? di
1
i
k-i
1100y0yr
if di ? y0yr ? ei
0
i
k-i
1100y0yr
if ei ? y0yr
X
i
k-i
22
The general case
23
The general case

• Remark The lemma reduces the general case to
  the easy case.
• Proof of Lemma
• f(p) D(p)/E(p), where D, E ? Zp have positive
  values.
• Write D and E as homogenous polynomials in p, 1
  p
• D(p) d(p,1-p) and E(p) e(p,1-p).
• d(p,1-p), e(p,1-p) and (e d)(p,1-p) are
  positive for all 0 lt p lt 1,
• ? for large enough n all the coefficients of
• d(p,q) (pq)n d(p,q), e(p,q) (pq)n
  e(p,q), and
• (e d)(p,q) (pq)n (e d)(p,q) are
  positive.
• - Write f(p) d(p,1-p)/e(p,1-p).

24
Open problems

• Can every algebraic function f (0,1) ? (0,1) be
  simulated via a pushdown automaton?
• For rational functions f, what is the minimal
  size of automaton simulating f?
• The best bound known in Polyas Theorem for f is
• (Powers and Reznick 2002).
• For a rational function f, does there exist a
  finite automaton which extracts almost all
  entropy?