Complexity

1
Complexity

• Basic Idea
• We want to compare the performance of different
  algorithms for a given task in terms of some
  measure of efficiency.
• Could use execution time (of the algorithm coded
  as a program) as a measure of performance.

2
Complexity

• But execution time depends on
• Program code,
• Programming language,
• Compiler (and compiler flags),
• Processor,
• Operating system, etc.
• We seek a measure that is intrinsic to an
  algorithm (independent of the above
  considerations).

3
Complexity Measures

• Two common measures
• The Time Complexity of an algorithm consists of a
  count of the number of time-critical operations
  as the size of the input varies.
• The Space Complexity of an algorithm consists of
  a count of the number of space cells as the size
  of the input varies.

4
Matrix Multiplication
5
Matrix Multiplication

• Time complexity
• Space complexity

6
Matrix Multiplication

• Time complexity
• Space complexity

7
Matrix-vector Multiplication
8
Matrix-vector Multiplication

• Time complexity
• Space complexity

9
Vector (Inner) Product
10
Vector (Inner) Product

• Time complexity
• Space complexity

11
Matrix Multiplication

• Suppose that the matrices A and B have special
  structure.
• A is upper triangular
• Ai,j 0, i gt j.
• Time complexity

12
Matrix Multiplication

• Both A and B are upper triangular
• Ai,j Bi,j 0, i gt j.
• Then the product matrix C is also upper
  triangular.
• Time complexity

13
Determinants

• Calculate the determinant of the matrix
• The classic algorithm is based on the recursive
  definition of the algorithm.
• Let the time to calculate the determinant of an
  n by n matrix be t(n), then

14
Determinants

• Time complexity of classic algorithm
• This is very inefficient. Suppose the algorithm
  takes 1 sec to calculate the determinant of a 5
  by 5 matrix, it will take 8.4 hours for a 10 by
  10 matrix and 345.5 years for a 15 by 15 matrix.

15
Determinants

• An alternative algorithm is the Gauss-Jordan
  algorithm, which is based on reducing A to
  diagonal form.
• The time complexity of Gauss-Jordan is

16
Searching

• Given an ordered sequence of items (e.g.
  integers), decide whether a given item is present
  in the list.

17
Binary Search

• If sequence is empty return false.
• If sequence contains just 1 item return the
  result of the equality test with that item.
• Otherwise, compare with the middle item in the
  sequence
• If equal return true,
• If given item is less than (greater than) middle
  item, recursively call binary search on
  subsequence to the left (right) of the middle
  item.

18
Binary Search

• Complexity of binary search.
• Algorithm is recursive.
• The worst case is when each comparison with the
  middle item fails. Suppose the original sequence
  is of length 2M, then the failure of the first
  test means that the next sequence is of length
  2M-1, and so on. We find that after M recursions
  (comparisons) the sequence is of length 1, and
  the final comparison can be made.

19
Binary Search

• Complexity of binary search.
• For an ordered sequence of length N the worst
  case complexity (number of comparisons) is
  approximately log2 N.

20
Implications of Different Complexities