Algorithm Analysis

1
Algorithm Analysis

• Dr. Bernard Chen Ph.D.
• University of Central Arkansas

2
Outline

• Big O notation
• Two examples
• Search program
• Max. Contiguous Subsequence

3
Program

• Algorithms Data Structure Programs
• Algorithms
• Must be definite and unambiguous
• Simple enough to carry out by computer
• Need to be terminated after a finite number of
  operations

4
Why Algorithm analysis

• Generally, we use a computer because we need to
  process a large amount of data.
• When we run a program on large amounts of input,
  besides to make sure the program is correct, we
  must be certain that the program terminates
  within a reasonable amount of time.

5
What is Algorithm Analysis?

• Algorithm A clearly specified finite set of
  instructions a computer follows to solve a
  problem.
• Algorithm analysis a process of determining the
  amount of time, resource, etc. required when
  executing an algorithm.

6
Big O Notation

• Big O notation is used to capture the most
  dominant term in a function, and to represent the
  growth rate.
• Also called asymptotic upper bound.
• Ex 100n3 30000n gtO(n3)
• 100n3 2n5 30000n gtO(n5)

7
Examples of Algorithm Running Times

• Min element in an array O(n)
• Closest points in the plane (an X-Y coordinate),
  ie. Smallest distance pairs
• n(n-1)/2 gt O(n2)
• Colinear points in the plane, ie. 3 points on a
  straight line n(n-1)(n-2)/6 gt O(n3)

8
Examples of Algorithm Running Times

• In the function
• 10n3 n2 40n 80, for n1000, the value of
  the function is 10,001,040,080
• Of which 10,000,000,000 is due to the 10n3

9
Various growth rates
10
Functions in order of increasing growth rate
11
Static Searching problem

• Static Searching Problem
• Given an integer X and an array A, return the
  position of X in A or an indication that it is
  not present. If X occurs more than once, return
  any occurrence. The array A is never altered.

12
Sequential Search

• A sequential search steps through the data
  sequentially until an match is found.
• A sequential search is useful when the array is
  not sorted.
• A sequential search is linear O(n) (i.e.
  proportional to the size of input)
• Unsuccessful search --- n times
• Successful search (worst) --- n times
• Successful search (average) --- n/2 times

13
Binary Search

• If the array has been sorted, we can use binary
  search, which is performed from the middle of the
  array rather than the end.
• We keep track of low_end and high_end, which
  delimit the portion of the array in which an
  item, if present, must reside.
• If low_end is larger than high_end, we know the
  item is not present.

14
Cont.

• Sequential search gtO(n)
• Binary search (sorted data) gt O(logn)

15
Binary Search 3-ways comparisons

• int binarySearch(vectorltintgt a, int x)
• int low 0
• int high a.size() 1
• int mid
• while(low lt high)
• mid (low high) / 2
• if(amid lt x)
• low mid 1
• else if( amid gt x)
• high mid - 1
• else
• return mid
• return NOT_FOUND // NOT_FOUND -1
• //binary search using three-ways comparisons

16
The Max. Contiguous Subsequence

• Given (possibly negative) integers A1, A2, ..,
  An, find (and identify the sequence corresponding
  to) the max. value of sum of Ak where k i -gt j.
  The max. contiguous sequence sum is zero if all
  the integer are negative.
• -2, 11, -4, 13, -5, 2 gt20
• 1, -3, 4, -2, -1, 6 gt 7

17
Real life Example from http//solvealgo.blogspot.c
om/2009/03/dynamic-programming-1-maximum-value.htm
l

• Real life example The club X never closes. Its
  public entrance, a revolving door, just keeps on
  spinning. With each rotation some punters enter
  and others leave. The clubs owners would like to
  track this traffic. Specifically, theyd like to
  know the maximum increase in people entering the
  club over a given period.
•  Entry log 0 1 2 -3 3 -1 0 -4 0 -1 -4 2 4 1 3 1
•  Positive people entering into the club
• Negative leaving.

18
Brute Force Algorithm O(n3)

• int maxSubSum(int a)
• int n a.size()
• int maxSum 0
• for(int i 0 i lt n i)
  // for each possible start point
• for(int j i j lt n j) // for
  each possible end point
• int thisSum 0
• for(int k i k lt j k)
• thisSum ak//dominant term
• if( thisSum gt maxSum)
• maxSum thisSum
• seqStart i
• seqEnd j
• return maxSum
• //A cubic maximum contiguous subsequence sum
  algorithm

19
O(n3) Algorithm Analysis

• We do not need precise calculations for a Big-Oh
  estimate. In many cases, we can use the simple
  rule of multiplying the size of all the nested
  loops

20
O(N2) algorithm

• An improved algorithm makes use of the fact that
  
• If we have already calculated the sum for the
  subsequence i, , j-1. Then we need only one more
  addition to get the sum for the subsequence i, ,
  j. However, the cubic algorithm throws away this
  information.
• If we use this observation, we obtain an improved
  algorithm with the running time O(N2).

21
O(N2) Algorithm cont.

• int maxSubsequenceSum(int a)
• int n a.size()
• int maxSum 0
• for( int i 0 i lt n i)
• int thisSum 0
• for( int j i j lt n j)
• thisSum aj
• if( thisSum gt maxSum)
• maxSum thisSum
• seqStart i
• seqEnd j
• return maxSum
• //figure 6.5

22
O(N) Algorithm

• If we remove another loop, we have a linear
  algorithm
• The algorithm is tricky. It uses a clever
  observation to step quickly over large numbers of
  subsequences that cannot be the best

23
O(N) Algorithm

• template ltclass Comparablegt
• int maxSubsequenceSum(int a)
• int n a.size()
• int thisSum 0, maxSum 0
• int i0
• for( int j 0 j lt n j)
• thisSum aj
• if( thisSum gt maxSum)
• maxSum thisSum
• seqStart i
• seqEnd j
• else if( thisSum lt 0)
• i j 1
• thisSum 0
• return maxSum
• //figure 6.8

24
Checking an Algorithm Analysis

• If it is possible, write codes to test your
  algorithm for various large n.

25
Limitations of Big-Oh Analysis

• Big-Oh is an estimate tool for algorithm
  analysis. It ignores the costs of memory access,
  data movements, memory allocation, etc. gt hard
  to have a precise analysis.
• Ex 2nlogn vs. 1000n. Which is faster? gt it
  depends on n

26
Common errors

• For nested loops, the total time is effected by
  the product of the loop size, for consecutive
  loops, it is not.
• Do not write expressions such as O(2N2) or
  O(N22). Only the dominant term, with the leading
  constant removed is needed.

27
Some practice

• Determine the Big O( ) notation for the
  following