Randomized

1
Randomized
Algorithms
Fundamentals of Algorithmics.
Gilles Brassard and Paul
Bratley.
Chapter 10.
Randomiized Algorithms
SC440/CSC401/CPE401
1
2
Introduction (Traditional algorithms)
Traditional algorithms have the following
properties

They are always correct They are deterministic
--- there may be more than one correct output,
but the same instance of a problem always
produces the same output They are always precise
--- the answer is not given as a range


Each of them operates at the same
efficiency
for
the
same instance
of
a problem
? 2
24 ?
Prime factor
Randomiized Algorithms
SC440/CSC401/CPE401
2
3
Introduction (Randomized algorithms)
Randomized (probabilistic) algorithms can be

Nondeterministic --- they can make random
but correct decisions the same algorithm may
behave differently when it is applied twice to
the same
instance Not very is given,
of a problem. precise sometimes --- usually the
more the better precision can be obtained

time

operating at different efficiencies
for the runs.
same
instance
of a problem in different
2 or 3
?
24 ?
Prime factor
Randomiized Algorithms
SC440/CSC401/CPE401
3
4

Incorrect sometimes hope we have a high,
known probability of being correct. Nonterminating
sometimes do not produce an answer at all
(hope fore a low probability for that)

because

A randonmized algorithm is one
that
makes
random
choices
during
the
execution
Randomiized Algorithms
SC440/CSC401/CPE401
4
5
Why do we need randomized (probabilistic
algorithms)

When an algorithm is confronted by a choice, it
is sometimes preferable to choose a course of
action at random, rather than spending time to
work out which alternative is the best. Sometimes
we do not have a better method than making random
choices

Randomiized Algorithms
SC440/CSC401/CPE401
5
6
Classes of randomized (probabilistic) algorithms
1.
Numerical probabilistic algorithms --- give
an approximation to the correct answer. Monte
Carlo Algorithms --- always give an answer, but
there is a probability of being completely wrong.
2.
3.
Las Vegas Algorithms --- sometimes fail to give
an
answer, but if an answer is given, it is
correct.
These
are
what
we
cover
in
this
chapter.
Randomiized Algorithms
SC440/CSC401/CPE401
6
7
Expected versus average time

The average time of a deterministic algorithm
refers to the average time taken by the algorithm
when each
possible instance of a given size
is
considered
equally
likely.
E.g
sorting
3
integers
T123 T132 T213 T231 T312 T321
1, 1, 2, 2, 3, 3,
2, 3, 1, 3, 1, 2,
3 2 3 1 2 1
A
sorting
algorithm

6
Randomiized Algorithms
SC440/CSC401/CPE401
7
8

The expected time of a probabilistic algorithm is
defined on each individual instance.
It is the mean
time over
that and
it would take over again.
to solve the
same
instance
T1 T2 T3 T4 T5 T6
1,
2,
3
A
sorting
algorithm
Compute
the
mean
Randomiized Algorithms
SC440/CSC401/CPE401
8
9

The worst case expected time of a
probabilistic algorithm refers to the expected
time taken by the worst possible instance of a
given size, not the time incurred if the worst
possible probabilistic choices are unfortunately
taken.
Randomiized Algorithms
SC440/CSC401/CPE401
9
10
Numerical probabilistic algorithms

For certain real life problems, computation of
an exact solution is not possible, maybe

because of uncertainties in the experimental
data to be used Because a digital computer cannot
represent an irrational number exactly Because a
precise answer will take too long to compute



A numerical probabilistic algorithm will give
an approximate answer (a confidence interval can
be provided). The precision of the answer
increases when more time is given to the
algorithm to work on

Randomiized Algorithms
SC440/CSC401/CPE401
10
11
Buffons needle
18th
In
the
century, Georges de Buffon, proved that if
you throw a needle at random (in a random
position and at a random angle, with uniform
distribution) on a floor made of planks of
constant width, if the needle is exactly half as
long as the planks in the floor are wide and if
the width of the cracks between the planks
is zero,
the probability
that
the
needle
will
fall
across
1/p.
a
crack
is
Randomiized Algorithms
SC440/CSC401/CPE401
11
12
Monte Carlo Algorithms

There are problems for which no efficient
algorithm is known, and approximate solutions do
not make sense. A Monte Carlo algorithm always
gives an answer but occasionally makes a
mistake. But it finds a correct solution with
high probability whatever the instance is
processed, i.e. there is no instance on which the
probability of error is high. However, no warning
is usually given when the algorithm gives a wrong
solution.



Example
to find the mode (most common element)
in
a
bag of values.
Pick an element at random the
probability that it is the mode is high, but
there is
a
Randomiized Algorithms
SC440/CSC401/CPE401
17
13
non-zero probability that it is completely wrong.
Making several random choices and combining the
results improves the probability of correctness.
Randomiized Algorithms
SC440/CSC401/CPE401
18
14
Primality testing The problem is to decide
whether a given odd integer is prime or
composite. The problem is important for
public-key cryptography.


A
Naive method
Given an input number N, we check whether it
is divisible by any integer greater than 1 and
less than or equal to the square root of N. If
the answer is NO, then N is a prime, otherwise
not. This relies on the not so trivial fact that
if there are any factors other than 1 or N, they
cannot all be greater than this square
root. Composite
Divisible by
N ?
prime
Randomiized Algorithms
SC440/CSC401/CPE401
19
15
Probabilistic tests Most popular primality tests
are probabilistic tests. In a certain sense,
those tests are not really primality

tests -- they do not determine with certainty
whether number is prime or not. The basic idea is
as follows
a

1. 2.
Randomly pick a number x called a witness. Check
some formula involving x and the given number N.
If the formula is no fit, then N is a composite
number and the test stops. Repeat step 1 unless
the required certainty is achieved.
3.
After several iterations, if N is not composite
number, then it can be prime.
found to be a declared probably
Randomiized Algorithms
SC440/CSC401/CPE401
20
16

The simplest probabilistic primality test is the
Fermat primality test. It is sometimes used if a
rapid screening of numbers

is needed, for instance in the key generation
phase
of
the RSA public key cryptographical
algorithm.
Fermats little Theorem
If n is a prime, then
an-1
mod n 1
For any
integer
a such that
1 a n
- 1.
E.g.
n n n

7, a 8, a
1 or 2 or 3 or 6
27
2, we have
mod 8 0 ? 1
29
mod 10 2 ? 1
10, a 2, we have
Randomiized Algorithms
SC440/CSC401/CPE401
21
17
The contrapositive If there is a value a such
that 1 a n 1 and an-1 mod n ? 1, then n
is not a prime.
Fermat (n) a uniform(1, n-1)
(an-1
if
mod n ? 1) then return n is composite else
return n is prime
an-1

However, there are composite numbers such that
414
mod n 1.
For example, n 15,
mod 15 1.

The Miller-Rabin test is a more sophisticated
variant of the Fermat test but with a lower error
rate it is often the method of choice.
Randomiized Algorithms
SC440/CSC401/CPE401
22
18

Suppose n gt 4 is an odd integer which we want to
test for primality. Choose a random integer a
with 2 a n - 2. Find s and t such that t is
odd and n - 1 2st .

2rt
If (at mod n 1) or (a
mod n n - 1) for
at
least
one
r
0, ..., s-1, then n is declared "prime" If
not, then n is definitely composite.

MillerRabin(n, k) for (i 1 to k do) a
uniform(2, n 2) if (Btest(a, n) false) then
return return prime
composite
Randomiized Algorithms
SC440/CSC401/CPE401
23
19
Btest(a, n)
s 0
t n 1
Repeat s s 1
t t/2
Until t is odd
at
x
mod n
If (x 1 or x n 1) then return For (i 1
to s 1)
true
x2
x
mod n
if (x n 1) then return true Return false The
Miller-Rabin algorithm always answer when n gt 4
is a prime.

returns
the correct

When n gt 4 is an odd composite, each call
on Btest
has probability at most ¼ of making a mistake.
Randomiized Algorithms
SC440/CSC401/CPE401
24
20
Las Vegas Algorithms

Las Vegas algorithms make probabilistic choices
to help guide them more quickly to a correct
solution.

Unlike Monte Carlo algorithms, they never return
wrong answer. Two main categories of Las Vegas
algorithms.
a

First category

The first kind is often used when a known
deterministic algorithm to solve a problem runs
much faster on the average than in the worst
case, e.g. quicksort. See previous lecture
Randomiized Algorithms
SC440/CSC401/CPE401
26
21
Second category

The second kind of Las Vegas algorithms now and
then make choices that bring the problem solving
process to a dead end. For example,
Eight-queen problem
1st

When we use backtracking, we got the
solution
after building 114 nodes of the 2057 nodes in the
state-space tree.

A greedy Las Vegas algorithm places the queens
randomly on successive rows, taking care not to
put a queen in an illegal position. Being a
greedy algorithm, we do not backtrack. We either
succeed if we manage to place all the queens, or
we run into a dead end --- failure.

Randomiized Algorithms
SC440/CSC401/CPE401
33
22
Q
Q
Q
Q
Q
Q
Q
Using a computer, it is calculated that the
probability of success, p, is approximately
0.1293.
Randomiized Algorithms
SC440/CSC401/CPE401
34
23
After failing, try, try again

If we fail to place all the queens in one
attempt, we restart from the beginning.

The expected number of attempts 8
until success is
?i P X i i 1
EX


8 ?i
(1-
p)i -1 p
i 1
1
p 1 - p
8
p ?i (1-
p)i



2
1 - p
p
1 - p
p
i 1
So for the randomized 8-queen, attempts to reach
success.
we expect
8
Randomiized Algorithms
SC440/CSC401/CPE401
35
24
Finding Hamiltonian paths

Given a graph, is there a path, any path, that
passes through all the vertices of the graph
exactly once. Such a path is called Hamiltonian
path. There are n vertices in graph G. Try to
build the path vertex by vertex.

Start from an arbitrary vertex, v0 in G and
select a neighbour of v0 to get a path v0v1
randomly .

Continue to randomly choose the next vertex stop
when all vertices are chosen
and
Randomiized Algorithms
SC440/CSC401/CPE401
36
25

choose the next vertex randomly from the
neighbours of the current vertex which are not
already on the path
Randomiized Algorithms
SC440/CSC401/CPE401
37
26

If at a certain vi, all its neighbours are
already on the
path, we pick one neighbour, vj,
change the path from
v0 v1 ,,vjvj1,,
vi-1vi
to v0 v1 ,,vjvivi-1,,vj1 and
continue
to
choose
the
next
vertex for the path
v5
v0
The path built v0 v1 v2v3v4 v5
v3
v
gt v v
v v v v
(v
v )
4
0 1 2 5 4 3
j
2
v1
v2
What is the new path if vj v0?
Randomiized Algorithms
SC440/CSC401/CPE401
38
27
Randomized_hamiltonian_path(G) v0 Random vertex
in G i 0

do
N if N
N(vi) v0, , is not empty i i 1
vi-1
vi Random vertex
in N
j lt i - 1)
else
if (there is vj is in N(vi) for 0

(v0, , vi) (v0, , return no_solution
vjvivi-1,
,
vj1)
else
while (i lt V return (v0, , Randomiized Algori
thms
- 1) vi)

SC440/CSC401/CPE401
39
28

If randomized_hamiltonian_path returns true,
a Hamiltonian path for graph G has indeed been
found. It is possible that the algorithm runs
into a dead end and fails to find a path,
although such a path exists. The algorithm is
good at finding a Hamiltonian path in large
graphs. The algorithm is almost always successful
for



graphs
of
sufficiently large minimum
degree
Randomiized Algorithms
SC440/CSC401/CPE401
40