Evolutionary Computing

1
CS4413 Numeric Algorithms (These materials
are used in the classroom only)
2
Goals

• Evaluate polynomials.
• More efficient matrix multiplication.
• Solve linear equations.
• Describe growth rates and order.

3
Why numeric algorithms?

• Mathematical calculation forms the basis for a
  wide range of programs.
• Computer Graphics and Computer Vision both
  require a large number of calculations involving
  polynomials and matrices.
• Because these are typically done for each
  location in an image, small improvements can have
  a great impact.

4
Why numeric algorithms?

• For example a typical image can be created with
  1024 pixels per row and 1024 pixels per column.
• Improving the calculation for each of these
  locations by even one multiplication would reduce
  the creation of this image by 1,048,576
  multiplications overall.
• Some software will repeatedly evaluate complex
  polynomial equations.
• Another application is trigonometric functions
  such as sine and cosine having power series
  expansions that take the form of polynomial
  equations.

5
Calculate polynomials

• The standard evaluation algorithm is very
  straightforward.
• Evaluate (x)
• x the value to use for evaluation of the
  polynomial
• result a0 a1 x
• xPower x
• for i 2 to n do
• xPower xPower x
• result result ai xPower
• end for
• return result

6
Horners method

• Horners method gives a better way to do
  evaluation without making the process very
  complex.
• p(x) ((anx an-1) x an-2 x a2
  x a1) x a0
• HornersMethod(x)
• x the value to use for evaluation of the
  polynomial
• result an
• for i n 1 down to 0 do
• result result x
• result result ai
• end for
• return result

7
Complexity

• Standard 2N 1 Multiplications and N
  Additions.
• Horners N Multiplications and N Additions.

8
Preprocessed Coefficients

• For preprocessed coefficients to work, we need
  our polynomial to be monic (an 1) and to have
  its largest degree equal to 1 less than a power
  of 2 (n 2k -1 for some k 1).

9
Matrix multiplication

• A matrix is a mathematical structure of numbers
  arranged in rows and columns that is equivalent
  to a two-dimensional array.
• Two matrices can be added or subtracted element
  by element if they are the same size.
• Two matrices can be multiplied if the number of
  columns in the first is equal to the number of
  rows in the second.

10
Matrix multiplication

• If we multiply a 3 x 4 matrix by a 4 x 7 matrix,
  we will get a 3 x 7 matrix as our answer.
• Matrix multiplication is not commutative.
• The standard matrix multiplication algorithm will
  do a b c multiplications and a (b -1) c
  additions for two matrices of size a x b and b x
  c.

11
Winograd multiplication

• If you look at each element of the result of a
  matrix multiplication, you will see that it is
  nothing more than the dot product of the
  corresponding row and column of the original
  matrices.
• Consider two of these vectors V(v1,v2,v3,v4)
  and W(w1,w2,w3,w4). Their dot product is given
  by VW.

12
Winograd Multiplication

• V(v1,v2,v3,v4)
• W(w1,w2,w3,w4)
• VWv1w1v2w2v3w3v4w4
• VW(v1w2)(v2w1) (v3w4)(v4w3)
  -v1v2-v3v4-w1w2-w3w4

13
Generalization
Compute Once, Use many times
14
Winograd algorithm

• Multiplying G a x b and H b x c to get result R a
  x c.
• d b / 2
• //calculate rowFactors for G
• for i 1 to a do
• rowFactori Gi,1 Gi,2
• for j 2 to d do
• rowFactori rowFactori Gi,2j-1 Gi,2j
• end for j
• end for i
• //calculate columnFactors for H
• for i 1 to c do
• columnFactori H1,i H2,i

15
Winograd algorithm

• for j 2 to d do
• columnFactori columnFactori H2j-1,i
  H2j,i
• end for j
• end for i
• //calculate R
• for i 1 to a do
• for j 1 to c do
• Ri,j -rowFactori columnFactorj
• for k 1 to d do
• Ri,j Ri,j (Gi,2k-1 H2k,j) (Gi,2k
  H2k-1,j)
• end for k
• end for j
• end for i

16
Winograd algorithm

• //add in terms for odd shared dimension
• if (2 (b/2) ? b) then
• for i 1 to a do
• for j 1 to c do
• Ri,j Ri,j Gi,b Hb,j
• end for j
• end for i
• end if

17
Analysis

• Ordinary Matrix Multiply, n3
• Winograds Matrix Multiply, (n3/2)n2
• Additions, n3-n2, v.s. (3/2)n32n2-2n
• Lower Bound, Best known is n2
• Best known Upper Bound and Best Known Lower Bound
  are not the same

18
Two By Two Multiplication

• c1,1a1,1b1,1a1,2b2,1
• c1,2a1,1b1,2a1,2b2,2
• c2,1a2,1b1,1a2,2b2,1
• c2,2a2,1b1,2a2,2b2,2

19
2x2 Works for Matrices

• C1,1A1,1B1,1A1,2B2,1
• C1,2A1,1B1,2A1,2B2,2
• C2,1A2,1B1,1A2,2B2,1
• C2,2A2,1B1,2A2,2B2,2

A1,1
A1,2
B1,1
B1,2
C1,1
C1,2


A2,1
A2,2
B2,1
B2,2
C2,1
C2,2
20
Divide and Conquer?

• Assume matrices of size 2n2n
• Multiplications M(n) 8M(n/2), M(1)8
• Additions A(n) 8A(n/2)n2
• M(n) 8lg n nlg 8 n3
• Additions are also Q(n3), but point is moot
• Can we reduce the 8 multiplications in the base
  equations

21
Strassens Equations

• x1(a1,1a2,2)(b1,1b2,2)
• x2(a2,1a2,2) b1,1
• x3a1,1(b1,2-b2,2)
• x4a2,2(b2,1-b1,1)
• x5(a1,1a1,2) b2,2
• x6(a2,1-a1,1)(b1,1b1,2)
• x7(a1,2-a2,2)(b2,1b2,2)

22
Using The Equations

• c1,1 x1 x4 - x5 x7
• c1,2 x3 x5
• c2,1 x2 x4
• c2,2 x1 x3 - x2 x6

23
Analysis of Strassen

• Only 7 multiplications are used
• Will work with matrices, because associative
  property is not used
• Multiplications M(n) 7M(n/2), M(1) 7
• Additions A(n) 7A(n/2)18(n2/4)
• M(n) 7lg n nlg 7 n2.81
• Additions are also Q(n2.81)
• First algorithm to break the n3 barrier

24
Gauss-Jordan Method

• Any system of linear equations must satisfy one
  of the following
• (1) has no solution.
• (2) has an unique solution.
• (3) has an infinite number of solutions.

25
Elementary Row Operations

• (1)Type?operationA is obtained by multiplying
  any row of A by a nonzero scalar c.
• e.g.

26
Elementary Row Operations

• (2)Type ? operationFor some j?i, let row i of
  Ac(row i of A )row j of A. And let the other
  rows of A be the same as the row of A.
• e.g.

27
Elementary Row Operations

• (3)Type ? operationInterchange any tow
• rows of A.
• e.g.
• -2(1)(2)
• (2)'

28
Continued

• (1)-(2)
• is the unique solution of such problem.
• Note the above 4 systems of linear equation are
  equivalent.

29
Example

• Ab want

30
Example

• 
  

31
Example

• 
  

32
Example
33
Usage of type ? operations

• ex
• Ab

34
Special CasesNo Solution

• ex
• one row no
  solution

35
Special Cases Infinite Solutions

• ex
• Homework
• one row infinite solutions