Complexity

1
Complexity
22-1
Probabilistic Algorithms
Complexity Andrei Bulatov
2
Complexity
22-2
Non-Deterministic vs. Probabilistic
All algorithms we nave seen so far are either
deterministic or impractical (non-deterministic)
To make non-deterministic algorithms more
practical we introduce probabilistic algorithms
A probabilistic algorithm (Turing Machine) is a
non-deterministic algorithm that makes
non-deterministic choices randomly, e.g. by
flipping a coin
This is still not practical, because sometimes
the algorithm should be extremely lucky to solve
problems
3
Complexity
22-3
Interactive Proofs
Prover
Verifier
Can perform polynomial time computations
Has unlimited computational power
Wants to convince Verifier in something
Accepts or rejects after performing some
computation
They can exchange massages
4
Complexity
22-4
Proofs for Problems in NP
SAT
Prover and Verifier get an instance of SAT
Prover solves the instance using his unlimited
computational power and send a satisfying
assignment to Verifier
Verifier checks (in polynomial time) if what
obtained is a satisfying assignment, and
accepts if it is or rejects otherwise
5
Complexity
22-5
Problems from coNP
Graph Non-Isomorphism
Instance Graphs G and H. Question Are G
and H isomorphic?
This problem belongs to coNP, but is not
believed to be coNP-complete
Apparently, there is no way to prove
interactively that two graphs are not isomorphic
6
Complexity
22-6
Randomized Verifier
Now suppose that Verifier has a fair coin
Given graphs G and H

• Verifier choose one of G and H by
  flipping a coin

• Verifier then rename somehow the vertices of
  the chosen
• graph and send it to Prover

• Prover decides which graph it received

• Prover send the answer to Verifier

• Repeat the procedure

7
Complexity
22-7
Analysis
If the graphs are not isomorphic then Prover
always gives the right answer
If the graphs are isomorphic then Prover gives a
correct answer with probability 1/2
Therefore if Prover is wrong we conclude that the
graphs are isomorphic
If after n repetitions of the protocol, Prover
gives only right answers, then Verifier can
conclude with probability that graphs are
not isomorphic
8
Complexity
22-8
Probabilistic Turing Machines
Definition A Probabilistic
Turing Machine is a nondeterministic polynomial
time Turing Machine PT such that
from each configuration of PT,
there are at most two
possible next configurations
PT chooses which of the two possible next
configurations to take by
flipping a fair coin
Thus, with each computational path, we can
associate the probability of taking this path.
This probability is equal to where k is
the number of coin flips made along this path
Denote this probability by Prp
9
Complexity
22-9
Define the probability that PT accepts w to be
Clearly
10
Complexity
22-10
Class BPP
Definition A Probabilistic
Turing Machine PT recognizes language L with
error probability ? if
w ? L implies PrPT accepts w ? 1 ?
w ? L
implies PrPT rejects w ? 1 ?
We say that PT operates with error probability
if the above inequalities hold for
where n is the length of w
Definition BPP is the class of
languages that are recognizable by
probabilistic Turing Machines with error
probability of 1/3
11
Complexity
22-11
Amplification
The error probability 1/3 may seem random
Actually, we can choose any value 0 ? ? ? 1
Amplification Lemma
Let 0 ? ? ? 1. Then for any polynomial p(n)
and a probabilistic TM that
operates with error probability ?,
there is a probabilistic TM that
operates with an error probability
The main idea is to run many times and
then output the majority of votes
12
Complexity
22-12
Math Prerequisites
Let be a series of
independent experiments (for example, coin
flips) such that the probability of success in
each of them is p
Theorem (Chernoff Bound)
If for some ? ? ?,
then the probability that the number of
successes in a series of n experiments is less
than is at most
13
Complexity
22-13
Proof of Amplification Lemma
Machine works as follows
On input w

• for i 1 to t(w) do

- simulate on w

• if most runs of accept, then
  accept otherwise reject

14
Complexity
22-14
Analysis
The number t(w) must be such that
15
Complexity
22-15
Primes
Complexity Andrei Bulatov
16
Complexity
22-16
The Problem
The complement of Primes, the Composite
problem, belongs to NP. Therefore Primes is
in coNP
Recently M.Agarwal et al. Proved that Primes
can be solved in polynomial time (see
http//www.cse.iitk.ac.in/news/primality.html)
However, the probabilistic algorithm we are going
describe is far more efficient
17
Complexity
22-17
Residues
For a positive integer n, we denote

• the set 0,1,2,,n 1

• the set 1,2,,n 1

• addition, multiplication and
  exponentiation modulo n

together with these operations is called
the set of residues modulo n
Every integer m, positive or negative, has a
corresponding residue m mod n
For example,
17 mod 5 2
20 mod 5 0
-1 mod 5 4
18
Complexity
22-18
Complexity of Arithmetic
Given two integers, a and b, we can compute

• a b in O(max(log a, log b))

• a ? b in O(log a ? log b)

cannot be computed in polynomial time,
because the size of this number is blog a
It is possible modulo n
Let be the binary
representation of b (k log b)
Then
that implies
First, we consecutively compute
in
Then we compute the product again in
19
Complexity
22-19
Prime and Coprime
Integers a and b are called coprime if
their greatest common divisor is 1
For example, 16 and 27 are coprime, and 15
and 18 are not
Theorem (Chinese Remainder Theorem)
If p and q are coprime
then, for any a and b, there is x
such that
For example, if p 5, q 3, and a 2, b
1, then x can be chosen to be 7
20
Complexity
22-20
Fermats Theorem
Theorem (Fermats Little Theorem)
If p is prime then, for any
we have
If the converse were true, we could use it for a
probabilistic primality test

• Choose k residues modulo n

• Compute their n 1 powers

• Accept if all results are 1 (mod n), reject
  otherwise

21
Complexity
22-21
Carmichael Numbers
Unfortunately, the converse is true just almost
One can straightforwardly check that, for any
, coprime with 561,
561 is a Carmichael number
n is said to be a Carmichael number if, for any
prime divisor p of n, p 1 n 1
Pseudo-prime Prime Carmichael
22
Complexity
22-22
Roots of 1
A square root of 1 modulo n is a number a
such that
Clearly, 1 and -1 (that is n 1) are
always roots of 1, but if n is composite,
then it may have more than two roots of 1
For example,
8 has four roots of 1 1, -1, 3, and 5
561 has eight 1, -1, 188, 373 (find the
remaining four)
Lemma Any
Carmichael number has at least 8 roots of 1
23
Complexity
22-23
Algorithm
On input n

• if n is even, then if n 2 accept,
  otherwise reject

• select randomly

• for i 1 to k do

- if then
reject
- let n 1 st where s is odd and
is a power of 2
- compute the sequence
modulo n
- if then
let j be the maximal with this property
if then reject

• accept

24
Complexity
22-24
Analysis
First we show that the algorithm does not give
false negatives, that is it accepts all prime
numbers
If n 2 then n is accepted. Let n be an
odd prime number
Then n passes Fermat test
n cannot be rejected in the last line, because
n has only two roots of 1
25
Complexity
22-25
Next we show that if n is composite, then
Prn accepted
A number such that a does not
pass either Fermat test or the square root test,
is called a witness
It is enough to prove that Pra is a witness ?
1/2, or, in other words, that at least half of
the elements of are witnesses
For every nonwitness d we find a witness d
such that if then
For a nonwitness a the sequence
either contains 1s only, or
it contains -1 followed by 1s
Nonwitnesses of both types are present 1 is a
nonwitness of the first type, and -1 is a
nonwitness of the second type
26
Complexity
22-26
Let d be a nonwitness of the second type such
that the 1 appears in the largest position in
the sequence
Let and
Since n is composite, n qr for some coprime
q and r
Note that
and
By the Chinese Reminder Theorem, there is t
such that
therefore
Hence t is a witness, because
but
27
Complexity
22-26
Now, for every nonwitness a we set a a t

• a is a witness, because
  and

but

• if then

Assume the contrary
Then, since we
have
Finally, we have