Fast Multiplication

1
Fast Multiplication

• Polynomial multiplication
• ToomCook method
• 3. Primitive Roots of Unity
• 4. The Discrete Fourier Transform
• 5. The FFT Algorithm
• 6. Schönhage-Strassens method
• 7. Conclusion

2
1. Polynomial multiplication

• Polynomial ???

(-5, 1, 2)
evaluation
interpolation
(-1,-4),(0,-5),(1,-2)
??
(p(-1), p(0), p(1))
by Lagrange Interpolating Polynomial
3
1. Polynomial multiplication
Lagrange Interpolating Polynomial
4
1. Polynomial multiplication
Polynomial Multiplication

• Given coefficients (a0,a1,a2,,an-1) and
  (b0,b1,b2,,bn-1) defining two polynomials, p()
  and q(), and number x, compute p(x)q(x).

5
1. Polynomial multiplication
Polynomial Multiplication
P(x)a0a1xan-1xn-1 q(x)b0b1xbn-1xn-1

• (a0,a1,a2,,an-1)
• (b0,b1,b2,,bn-1)

?2n1??
(p(w0), p(w1),, p(w2n) ) (q(w0), q(w1),, q(w2n)
)
evaluation
multiplication
Polynomial Multiplication
(p(w0)q(w0), p(w1)q(w1), , p(w2n)q(w2n) )
(c0,c1,c2,,cn,,c2n-1)
interpolation
6
1. Polynomial multiplication
Polynomial Multiplication
P(x)5x q(x)12x

• (5,1)
• (1,2)

?3??
(p(0)5, p(1)6, p(2)7 ) (q(0)1, q(1)3, q(2)5
)
evaluation
multiplication
Polynomial Multiplication
C(0)5, C(1)18, C(2)3
(5,11,2)
interpolation
7
1. Polynomial multiplication
????????????
Evaluation O(n)
Total O(n2)
8
1. Polynomial multiplication
Polynomial Evaluation

• Horners Rule
• Given coefficients (a0,a1,a2,,an-1), defining
  polynomial
• Given x, we can evaluate p(x) in O(n) time using
  the equation
• Eval(A,x) Where A(a0,a1,a2,,an-1)
• If n1, then return a0
• Else,
• Let A(a1,a2,,an-1) assume this can be
  done in constant time
• return a0xEval(A,x)

9
2.ToomCook method
Toom-k a simplification of a description of
ToomCook polynomial multiplication described by
Marco Bodrato in 2007.
?????M, N,???MN
five main steps
Splitting (????????) Evaluation
(?????????) Pointwise multiplication
(????) Interpolation (???????) Recomposition
(????????)
Marco Bodrato. Towards Optimal ToomCook
Multiplication for Univariate and Multivariate
Polynomials in Characteristic 2 and 0. In
WAIFI'07 proceedings, volume 4547 of LNCS, pages
116-133. June 21-22, 2007. author website
10
2.ToomCook methodToom-3
Toom-3 ????
M 12 3456 7890 1234 5678 9012 N 9 8765
4321 9876 5432 1098
(1) Splitting (????????)
??3??,?base B108
m0 56789012 m1 78901234 m2 123456 n0
54321098 n1 43219876 n2 98765
p(x) m0 m1x m2x2 q(x) n0 n1x n2x2
11
2.ToomCook methodToom-3
Toom-3 ????
(2) Evaluation (?????????)
p(x) m0 m1x m2x2 q(x) n0 n1x n2x2
p(0) m0 56789012 56789012 p(1) m0 m1
m2 56789012 78901234 123456 135813702
p(-1) m0 - m1 m2 56789012 - 78901234
123456 -21988766 p(-2) m0 - 2m1 4m2
56789012 - 278901234 4123456 -100519632
p(8) m2 123456 123456 q(0) n0
54321098 54321098 q(1) n0 n1 n2
54321098 43219876 98765 97639739 q(-1)
n0 - n1 n2 54321098 - 43219876 98765
11199987 q(-2) n0 - 2n1 4n2 54321098 -
243219876 498765 -31723594 q(8) n2
98765 98765
12
2.ToomCook methodToom-3
Toom-3 ????
(3) Pointwise multiplication (????)
r(x)p(x)q(x)
r(0) p(0)q(0) 56789012 54321098
3084841486175176 r(1) p(1)q(1) 135813702
97639739 13260814415903778 r(-1) p(-1)q(-1)
-21988766 11199987 -246273893346042 r(-2)
p(-2)q(-2) -100519632 -31723594
3188843994597408 r(8) p(8)q(8) 123456
98765 12193131840
13
2.ToomCook methodToom-3
Toom-3 ????
(4) Interpolation (???????)
14
2.ToomCook methodToom-3
Toom-3 ????
(4) Interpolation (???????)
15
2.ToomCook methodToom-3
Toom-3 ????
(4) Interpolation (???????)
one sequence given by Bodrato
16
2.ToomCook methodToom-3
Toom-3 ????
(4) Interpolation (???????)
17
2.ToomCook methodToom-3
Toom-3 ????
(5) Recomposition (????????)
18
2.ToomCook method
Complexity
Toom-3 reduces 9 multiplications to 5, and runs
in T(nlog(5)/log(3)), about T(n1.465). In
general, Toom-k runs in T(c(k)ne), where e
log(2k-1) / log(k), ne is the time spent on
sub-multiplications, and c is the time spent on
additions and multiplication by small constants
(in Knuths book)
19
Polynomial multiplication
????????????-Primitive Roots of Unity ??DFT
?? ??O(nlogn)
Evaluation O(n)
Total O(n2)
20
3. Primitive Roots of Unity

• A number w is a primitive n-th root of unity, for
  ngt1, if
• wn 1
• The numbers 1, w, w2, , wn-1 are all distinct
• Example 1
• Z11
• 224, 623, 725, 829 are 5-th roots of unity in
  Z11
• 2-16, 3-14, 4-13, 5-19, 6-12, 7-18, 8-17,
  9-15
• Example 2 The complex number e2pi/n is a
  primitive n-th root of unity, where

21
3. Primitive Roots of Unity

• Inverse Property If w is a primitive root of
  unity, then w -1wn-1
• Proof wwn-1wn1
• Cancellation Property For non-zero -nltkltn,
• Proof
• Reduction Property If w is a primitve (2n)-th
  root of unity, then w2 is a primitive n-th root
  of unity.
• Proof If 1,w,w2,,w2n-1 are all distinct, so are
  1,w2,(w2)2,,(w2)n-1
• Reflective Property If n is even, then wn/2
  -1.
• Proof By the cancellation property, for kn/2
• Corollary wkn/2 -wk.

22
4.The Discrete Fourier Transform

• Given coefficients (a0,a1,a2,,an-1) for an
  (n-1)-degree polynomial p(x)
• The Discrete Fourier Transform is to evaluate p
  at the values
• 1,w,w2,,wn-1
• We produce (y0,y1,y2,,yn-1), where yjp(wj)

23
4.The Discrete Fourier Transform

• Matrix form yFa, where Fi,jwij.
• The Inverse Discrete Fourier Transform recovers
  the coefficients of an (n-1)-degree polynomial
  given its values at 1,w,w2,,wn-1
• Matrix form aF -1y, where F -1i,jw-ij/n.

? the matrix F ?column????orthogonal??
FFnI,??F?F?conjugate
24
4.The Discrete Fourier Transform

• The DFT and inverse DFT really are inverse
  operations
• Proof Let AF -1F. We want to show that AI,
  where
• If ij, then
• If i and j are different, then

25
Convolution
4.The Discrete Fourier Transform

• The DFT and the inverse DFT can be used to
  multiply two polynomials
• So we can get the coefficients of the product
  polynomial quickly if we can compute the DFT (and
  its inverse) quickly

26
4.The Discrete Fourier Transform

• Convolutions

Convolutions
127?356
(3,11,37,47,42)
1 2 7

3 5 6
6 12 42
5 10 35
3 6 21
3 11 37 47 42
27
4.The Discrete Fourier Transform

• Convolutions

127?356
1 2 7
1 2 7


3 5 6
3 5 6
6 12 42
6 12 42
5 10 35
5 10 35
3 6 21
3 6 21
3 11 37 47 42
3 11 37 47 42
3 11
- 3 11
37 50 53
37 44 31
acyclic convolutions
(37,50,53)
(37,44,31)
negacyclic convolutions
28
4.The Discrete Fourier Transform

• Convolution theorem

CyclicConvolution(X, Y) IDFT(DFT(X) DFT(Y))
29
5.The Fast Fourier Transform

• The FFT is an efficient algorithm for computing
  the DFT
• The FFT is based on the divide-and-conquer
  paradigm
• If n is even, we can divide a polynomial
• into two polynomials
• and we can write

30
5.The Fast Fourier Transform

• Given coefficients (a0,a1,a2,,a7) and w
  primitive 8-th root of unity

31
5.The Fast Fourier Transform
32
5.The Fast Fourier Transform
33
5.The Fast Fourier Transform
?
34
5.The Fast Fourier Transform

• a0 a1 a2 a3 a4 a5 a6 a7

35
5.The Fast Fourier Transform
T(n)2T(n/2)O(n)

• The running time is O(n log n). inverse FFT is
  similar

36
Polynomial multiplication
Polynomial Multiplication
p(x)a0a1xan-1xn-1 q(x)b0b1xbn-1xn-1
?w primitive m-th root of unity
(m?2n)

• (a0,a1,a2,,an-1)
• (b0,b1,b2,,bn-1)

(p(w0), p(w1),, p(wm-1) ) (q(w0), q(w1),,
q(wm-1) )
Evaluation FFT
multiplication
Polynomial Multiplication
(p(w0)q(w0), p(w1)q(w1), , p(wm-1)q(wm-1) )
(c0,c1,c2,,cn,,cm-1)
Interpolation IFFT
The running time is O(n log n).
37
6.Schönhage-Strassens method
?????M,N???MN
(1) Splitting (????????)
?B2k,????degree ??n-1??????
p(x)a0a1Ban-1Bn-1 q(x)b0b1Bbn-1Bn-1
P(a0,a1,a2,,an-1) Q(b0,b1,b2,,bn-1)
(2) Evaluation (?????????)
?w primitive m-th root of unity
(m?2n)
38
6. Schönhage-Strassens method
(3) Pointwise multiplication (????)
(4) Interpolation (???????)
??????the inverse Fourier transform
39
6. Schönhage-Strassens method
(5)Recomposition (????????)
?xB
40
6. Schönhage-Strassens method
Complexity ??
???
The running time is O(n log n)
The bound on the numerical errors a on the ci
after the FFT process can be proved to be
where e ? 1.e-16
a 6n2B2log(n)e
the number of digits of these basic numbers
should be of the order of log(B)log(n)
41
6. Schönhage-Strassens method
Complexity ??
The running time is O(n log n(log(B) log(n))2)
?????????????recursive??,?
??
42
7. Conclusion
Method complexity integer
classical
Karatsuba 1962 10,000??
Toom3 1963
ToomCook (knuth) 1966 40,000??
Schönhage-Strassen 1971
Fürer 2007