Algorithm Analysis

1
Algorithm Analysis
2
Math Review 1.2

• Exponents
• XAXB XAB
• XA/XBXA-B
• (XA)B XAB
• XNXN 2XN ? X2N
• 2N2N 2N1
• Logarithms
• XAB iff logXBA
• logAB logCB/logCA A,B,C gt 0, A ? 1
• logAB logA logB A, B gt 0
• Series
• N
• ? 2i 2N1-1
• i0
• N
• ? Ai AN1-1/A-1
• i0
• N
• ? i N(N1)/2

3
Running Time

• Why do we need to analyze the running time of a
  program?
• Option 1 Run the program and time it
• Why is this option bad?
• What can we do about it?

4
Pseudo-Code

• Used to specify algorithms
• Part English, part code
• Algorithm (arrayMax(A, n))
• curMax A0
• for i1 iltn i
• if curMax lt Ai
• curMax Ai
• return curMax

5
Counting Operations

• Algorithm (arrayMax(A, n))
• curMax A0 //2
• for i1 iltn i //1n
• if curMax lt Ai //4(n-1) 6(n-1)
• curMax Ai
• return curMax //1
• Best case 5n
• Worst case 7n-2
• Average case hard to analyze

6
Asymptotic Notation

• 7n-2
• n5 -gt 33
• n100 -gt 698
• n1,000,000 -gt 6,999,998
• Running time grows proportionally to n
• What happens as n gets large?

7
Big-Oh

• T(N) is O(f(N)) if there are positive constants c
  and n0such that T(N) lt cf(N) when Ngtn0
• 7n2-2 is O(n2) n0gt1 c8
• Upper bound

8
Omega

• T(N) is O(g(N)) if there are positive constants c
  and n0such that T(N) gt cg(N) when Ngtn0
• 7n2-2 is O(n2) n0gt1 c1
• Lower bound

9
Theta

• T(N) is T(h(N)) if and only if T(N) Oh(N) and
  T(N) Oh(N)
• 7n2-2 is T(n2)
• Tightest result possible

10
Little-Oh

• T(N) is o(p(N)) if T(N) O(p(N)) and T(N) ?
  T(p(N))
• 7n2-2 is o(n3)
• O when n0gt8 c1
• Growth rate of T(N) is smaller than growth rate
  of p(N)

11
Rules

• Rule 1
• If T1(N) O(f(N) and T2(N) O(g(N)), then
• T1(N) T2(N) max(O(f(N)), O(g(N)))
• T1(N) T2(N) O(f(N)g(N))
• Rule 2
• If T(N) is a polynomial of degree k, then
  T(N) T(Nk)
• Rule 3
• logkN O(N) for any constant k. This tells us
  that logs grow very slowly.

12
Examples

• 87n4 7n
• 3nlogn 12logn
• 4n4 7n3logn

13
Terminology

• Logarithmic O(logn)
• Linear O(n)
• Linearithmic O(nlogn)
• Quadratic O(n2)
• Polynomial O(nk) kgt1
• Exponential O(an) agt1

14
Rule 1 for loops

• The running time of a for loop is at most the
  running time of the statements inside the for
  loop (including tests) times the number of
  iterations.
• for(i0 ilt5 i)
• cout ltlt hello\n

15
Rule 2 nested loops

• Analyze these inside out. The total running time
  of a statement inside a group of nested loops is
  the running time of the statement multiplied by
  the product of the sizes if all of the loops.
• for(i0 iltn i)
• for(j0 jltn j)
• cout ltlt hello\n

16
Rule 3 consecutive statements

• These just add (which means that the max is the
  one that counts)
• for(i0 iltn i)
• ai 0
• for(i0 iltn i)
• for(j0 jltn j)
• aiajij

17
Rule 4 if/else

• The running time of an if/else statement is never
  more than the running time of the test plus the
  larger of the running times of the statements,
• if(condition)
• S1
• else
• S2

18
Example
n-1
0
6
4

2
0

• Find maximum number in nxn matrix
• Algorithm

12
3

9




5
8

1
n-1
19
Example

• What is the big-oh running time of this
  algorithm?
• Algorithm
• Input A, n
• curMax A00
• for i0 iltn i
• for j0 jltn j
• if curMax lt Aij
• curMax Aij
• return curMax

20
Another Example
n-1
0
2
4

6
n-1
0
6
8

3

• Determine how many elements of array 1 match
  elements of array 2
• Algorithm?

21
Another Example
n-1
0
2
4

6
n-1
0
6
8

3

• Algorithm
• Input A, B, n
• for i0 iltn i
• for j0 jltn j
• if Ai Aj
• matches
• break
• What is the running time of the algorithm?

22
Logs in the Running Time

• An algorithm is O(logN) if it takes constant time
  to cut the problem size by a fraction.
• Binary search
• Given an integer X and integers A0, A1, , AN-1,
  which are presorted and already in memory, find i
  such that AiX, or return i -1 if X is not in
  the input.

23
Binary Search

• Algorithm?
• Running time is at most ceil(log(N-1))2 which is
  O(logN)

24
Example

• 1 (N-1) - 1
• 2 (N-1)/2 - 2
• 3 (N-1)/4 - 4
• 4 (N-1)/8 - 8
• i (N-1)/2i-1 - 2i-1 N-1
• difference between high and low is 1
• difference between high and low is 0

25
The Evil King

• King has N bottles of wine
• Exactly 1 bottle is poisoned
• How can the king figure out which bottle is
  poisoned and only kill at most logN of his
  testers?