Communication Complexity

1
Communication Complexity

• ??????
• 317735678 ????? ????
• 01762926/2 ??? ????
• ??? ????? ?.?. 02801532/9

2
Communication Complexity

• The Model
• 2 Computers A,B
• All calculations for A are free
• All calculations for B are free
• Algorithm costs are measured by cost of
  communications.
• Cost is measured per bits

3
Communication Complexity

• The Model

14, 29,53,28,284,348
39, 67,98,22,35,253
B
A
Communication Complexity
4
Example

• Input
• A has array of n numbers
• B has array of n numbers
• Output
• median of all 2n numbers
• Heuristic
• numbers are O(log n) bits long

5
Naïve Algorithm

• A sends all of his numbers to B
• B calculates median of all 2n numbers
• Cost
• Each number is O(log n) bits
• N numbers are sent
• Total cost is O(nlog n) bits

6
Communication Complexity

• Naive algorithm

14, 29,53,28,284,348
39, 67,98,22,35,253
B
A
Communication Complexity
Total cost is O(nlog n) bits
7
Communication Complexity

• Naive algorithm

14, 29,53,28,284,348
39, 67,98,22,35,253
14, 29,53,28,284,348
B calculate the median
B
A
Communication Complexity
8
Better algorithm

• A sorts his array and sends his median ( )
  to B
• B sorts his array and sends his median ( )
  to A.
• Exercise
• define r real median
• b MAX
• s MIN
  
• prove

9
Communication Complexity

• Better algorithm

B sort his array and find his median
A sort his array and find his median
14, 28,29,53,284,348,500
22, 35,39,67,98,253,300
Communication Complexity
10
Communication Complexity

• Better algorithm

B send his median to A
A send his median to B
14, 28,29,53,284,348,500
22, 35,39,67,98,253,300
Communication Complexity
11

• 4. If then return
  ( )
• 5. If gt then A throws top
  (n/2) elements
• B throws
  low (n/2) elements
• 6. And vice versa
• We reduces the size of the problem by half
• 7. Back to step 1, until size of arrays 1

12
Communication Complexity
67gt53 Then B throws the big half of his array
53lt67 Then A throws the small half of his array

• Better algorithm

,53,284,348,500
22, 35,39,67
Communication Complexity
13
Communication Complexity

• Better algorithm

Since equal amount of members were thrown from
two sides of the median So, new median not
changed.
,53,284,348,500
22, 35,39,67
Communication Complexity
14
Communication Complexity

• Better algorithm

We will repeat this algorithm until the size of
the array will be 1, while every loop the array
is cut in half, and log n bits transferred
,53,284,348,500
22, 35,39,67
Total cost is O (Log2(n)) bits
Communication Complexity
15
Calculation of cost

• Each number is O(log n) bits
• Each step 2 numbers are sent
• log(n) steps
• Total cost is O (Log2(n)) bits

16
another improvement
- A calculates his median - B calculates his
median
17 22 27 31 45
13 18 20 25 32
13 18 20 25 32
Communication Complexity
17
Two assumptions

1. The requested median is between the As median
   and Bs median.

13 18 20 25 32
17 22 27 31 45
The requested median between As and Bs median
As median
Bs median
13 17 18 20 22 25 27 31 32 45
(we have an even amount of numbers, so we
choose the the 6th place of the 10 numbers)
18
Two assumptions
2) In binary representation we divide the
medians to common segment and different
segment. Note Sometimes there is no common
segment, but if there is common for As and Bs
medians then its common to the request median.
Decimal binary representation 27 - 1 1 0 1 1
25 - 1 1 0 0 1 20 - 1 0 1 0 0
Common segment
Different segment
19
Communication Complexity
After those assumptions we continue. Every side
send each bit of his median (start from MSB bit)
27 1 1 0 1 1 b
20 10100 b
1 b
1 b
17 22 27 31 45
13 18 20 25 32
13 18 20 25 32
Communication Complexity
20
Communication Complexity
Every side compares the bit he gets with the bit
of his median. If equal, those bits are in the
common segment (see slide 17) and this common
to the request median then every side continue
send the next bit of his median.
Communication Complexity
21
Communication Complexity
If not equal then A sees that his median is
bigger, so he throws the bigger half of his
array, B sees that his median is smaller then
As median, so he throws the smaller half of his
array
Communication Complexity
22
Communication Complexity

• After throwing half of his array, every side
  calculates his new median.
• Every side send bits of the new median, starting
  from the bit that there was different in the
  previous iteration ( and not from the MSB bit).
• Every side continues the algorithm.

Communication Complexity
23
Communication Complexity

• The algorithm stops when one of two events occur.
• 1) the arrays of A and B contain one element
  (each).
• 2) all bits of the medians were sent.

Communication Complexity
24
Communication Complexity

• If arrays contain one element so this element is
  the median.
• If all bits were sent so the medians are equal
  and this is the request median.

Communication Complexity
25
Complexity of algorithm
Every iteration one of this events occur Bit is
send to the other sides.(log n bits can be sent
from every side) The array become shorter by
half. (the array can reduce by half , log n
times) So, sum of bits can be sent limited by
O(log n).
26
Communication complexity
The previous subject talked about problem of
finding median of array that was divided to two
parts. Now we consider a new problem Each side
has a number and we want to know if the numbers
are equal.
? XY
Y
X
27
Communication complexity
Naïve algorithm A send X to B. B compares X to Y
and return yes/no X is logX bits long so cost is
logX
Y
X
28
Communication complexity
New random algorithm - GLOBAL CC
In this algorithm we have a random number R,
that can be changed and known for both sides.
? XY
Y
X
29
Communication complexity
GLOBAL CC
Lets define inner product of A and B
A?0..n-1, B?0..n-1
Y
X
30
Communication complexity
GLOBAL CC
A calculates (XR)mod 2
B calculates (YR)mod 2
Y
X
31
Communication complexity
GLOBAL CC
If XY then always bX bY If x ? Y then
Prob(bX bY)1/2
bY(YR)mod 2
bX(XR)mod 2
Y
X
32
Communication complexity
GLOBAL CC
Every side sends his result (b) and compares the
results. If the results are not equal so the
numbers are not equal.
bX(XR)mod 2
bY(YR)mod 2
Y
X
33
Communication complexity
GLOBAL CC
bY(YRnew)mod 2
bX(XRnew)mod 2
We choose a new R and repeat the algorithm. We do
so C times.
Y
X
34
Communication complexity
GLOBAL CC
If XY then all C times bX bY. If X?Y the
probability that all C times bX bY is 2-c.
Y
X
35
Communication complexity
Complexity of global CC
Each time every side transfers bX/Y by length of
1 bit. There are C times so every side transfers
C bits. Complexity O ( C )
Y
X
36
Communication complexity
Another random algorithm
In this algorithm every side constructs a polynom
from his number according to these steps Every
number consist of n bits.
X?an-1, an-2,..,a1,a0
Y?an-1, an-2,..,a1,a0
Y
X
37
Communication complexity
Step 1
Polynom of side A will be PolyA(T) an-1Tn-1
an-2Tn-2.. a0T0
X?an-1, an-2,..,a1,a0
Y?bn-1, bn-2,..,b1,b0
Y
X
38
Communication complexity
Step 1
Like side A polynom B will be Polyb(T)
bn-1Tn-1 bn-2Tn-2.. b0T0
X?an-1, an-2,..,a1,a0
Y?bn-1, bn-2,..,b1,b0
Y
X
39
Communication complexity
Step 2
A chooses random prime p n2 ltp lt n3 A sends p
to B
X?an-1, an-2,..,a1,a0
Y?bn-1, bn-2,..,b1,b0
Y
X
40
Communication complexity
Step 3
B chooses 1R p-1 and calculates (polyb(R) mod
p) Remind Polyb(R) bn-1Rn-1 bn-2Rn-2..
b0R0 B sends (R, (polyb(R) mod p) ) to A.
Y
X
41
Communication complexity
Step 4
A calculates polya(R) mod p A compares (polya(R)
mod p) with (polyb(R) mod p)
Y
X
42
Communication complexity
Analysis of algorithm If polya(R) ? polyb(R) Then
X ? Y
Y
X
43
Communication complexity
Analysis of algorithm But if polya(R) polyb(R)
there is some probability that X ? Y. We will
calculate F Fprob(X ? Y and polya(R) polyb(R))
Y
X
44
Communication complexity
Analysis of algorithm polya(R) polyb)R)
polya(R) - polyb)R)0 FprobX ? Y and
polya(R) polyb)R( probX ? Y and
polya(R)-polyb)R(0. polya(R)-polyb)R(0 is a
polynom of degree n it has n-1
roots.
Y
X
45
Communication complexity
Analysis of algorithm Because there are n-1 Rs
s.t. polya(R)-polyb)R(0. And 1ltRltp-1. So the
probability to choose one of those Rs is
(n-1)/p.
Y
X
46
Communication complexity
Analysis of algorithm We saw that n2 ltp lt
n3 So n-1/pgt(n-1)/ n2?1/n Fprob(X ? Y and
polya(R) polyb(R))lt1/n The probability of
mistake is1/n
Y
X
47
Communication complexity
Analysis of algorithm Communication
complexity The sides transferred p and R only
n2 ltp lt n3 pO(log n) And Rltp So the total
complexity is O(log n)
Y
X
48
Communication complexity
CC and Global CC What is the complexity relation
between CC and GCC Note CC is the last
alogorithm we had seen.
Y
X
49
Communication complexity
solution A and B have a deterministic Turing
machine which generates (deterministically) K
random rs. (while K )
Y
X
50
Communication complexity
solution Define R to be the chain of Ks random
rs
Y
R
X
51
Communication complexity
Now, instead of A sending r to B, A only has to
send index in R. The index can be sent with
O(log( ) bits O(log n)
Y
X
52
Communication complexity
Claim For each subset of R there is in the
subset that is never wrong for each (x,y).
Y
X
53
Communication complexity
Proof
Y
X
54
Communication complexity
Proof
Y
X
55
Communication complexity
So
Y
X
56
Communication complexity
Conclusion
Y
X