Algorithm Analysis

1
Algorithm Analysis
2
AlgorithmDef
An algorithm is a step-by-step procedure
3
AlgorithmExample - Compute the sum
1011121330
Take the first number and continuing add next
number until the last number
public int sum (int nStart, int nEnd) int sum
0 for (int i nStart i ? nEnd i) sum
i return sum
sum 0 FOR i nStart TO nEnd sum sum
i ENDFOR RETURN sum
4
Algorithm AnalysisRequirements
sum 0 FOR i nStart TO nEnd sum sum
i ENDFOR RETURN sum
Simplicity Readability Verifiability Validity Robu
stness Modifiability Reuseability Efficiency
public int sum (int nStart, int nEnd) int sum
0 for (int i nStart i ? nEnd i) sum
i return sum
5
AlgorithmExample - Finding the largest value
6
AlgorithmExample - selectionSort
7
AlgorithmExample - Character permutations
8
Asymtotic NotationNumber of statements -
Mathematical relations
Proof
9
Asymtotic NotationNumber of statements -
Mathematical relationsProof by Induction
10
AlgorithmBasic Timing Analysis - Consecutive
statements
count algorithm step 1 a 4 1 b
8 1 c a
Run time
11
AlgorithmBasic Timing Analysis - For loop
count algorithm step 1 for (i
1 n1 i lt n n i) n x x 1
for (i 1 i lt n i) x x 1
Run time
12
AlgorithmBasic Timing Analysis - For loop,
if statement
count algorithm step 1 max
array0 1 for ( i 1 n i lt
n n-1 i) n-1 if (arrayi gt
max) p max arrayi 0
? p ? n-1 1 return max
Run time
13
AlgorithmBasic Timing Analysis
count algorithm step 1 sum
0 1 (n1) n for ( i 1 i lt n
i) 1n (n1)n nn for ( j 1 j lt n
j) nn sum sum
1 1 return sum
Run time
14
AlgorithmBasic Timing Analysis - Simple For
loop
count algorithm step 1(n1)n for ( i
1 i lt n i) kn Sk
Run time
Complexity Asymptotic Notation
15
AlgorithmBasic Timing Analysis - Simple For
loop
count algorithm step for ( i 1 i lt
n i) kn Sk
Run time
Complexity Asymptotic Notation
16
AlgorithmBasic Timing Analysis - Double For
loop
for (i 1 i lt n i) for (j 1 j lt n
j) Sk
17
AlgorithmBasic Timing Analysis - Single For
loop Double For loop
for (i 1 i lt n i) Sk for (i 1 i lt n
i) for (j 1 j lt n j) Sk
18
AlgorithmBasic Timing Analysis - Double For
loop
for (i 1 i lt n i) for (j i j lt n
j) Sk
19
AlgorithmBasic Timing Analysis - Triple For
loop
for (i 1 i lt n i) for (j i j lt n
j) for (ki k ltj k) Sm
20
Different functions and their graphs
Expression Name -------------------------------
------------- 1 constant log
n logarithmic log2n log
squared n linear n log n n log
n n2 quadratic n3 cubic 2n
exponential -------------------------------------
-------
21
AlgorithmGrowth of Common Functions
22
AlgorithmGeneral Rules
Consecutive statements These just add.
if-then-else The running time of an if-then-else
statement is never more than the running time of
the test plus the larger of the running times of
each part.
Loops The running time of a loop is at most the
running time of the statements inside the loop
(including tests) times the number of iterations.
Nested Loops Analyze these inside out. The total
running time of at statement inside a group of
nested loops is the running time of the statement
multiplied by the products of the sizes of all
the loops.
23
Basic AxiomsA 01
A 01 The time required to fetch an operand from
memory is a constant The time required to
store an operand in memory is a constant
Memory
24
Basic AxiomsC 01
C 01 The time required to perform a simple
assignment statement is the time required to
fetch an operand plus the time required to store
an operand
x

y
25
Basic AxiomsA 02
A 02 The time required to perform elementary
operations, such as addition, subtraction,
multiplication, division, and comparison, are all
contants.
26
Basic AxiomsA 01 - A 02 - Example
Time required to perform the following statement
27
Basic AxiomsA 03
A 03 The time required to call a method is a
contant, and the time required to return from a
method is a constant.
28
Basic AxiomsA 04
A 04 The time required to pass an argument to a
method is the same as the time required to store
a value in memory
29
Basic AxiomsA 04 - Example
Time required to perform the following statement
fetch x
call f
passing x
running time of f
assignment to y
30
Basic AxiomsA 01 - A 02 - A 03 - Example
Running time Time required to compute the
following simple arithmetic series summation
1 public class Example 2 public static int
sum (int n) 3 int result 0 4 for (int
i 0 i lt n i) 5 result
i 6 return result 7 8
31
Basic AxiomsA 05
A 05 The time required for the address
calculation implied by an array subscripting
operation, ai, is a constant. This time does
not include the time to compute the subscript
expression, nor does it include the time to
access (fetch or store) the array element.
32
Basic AxiomsA 05 - Example
Time required to perform the following statement
fetch a
fetch i
address subscripting calcutaion
fetch ai
assignment to y
33
Basic AxiomsA 01 - A 02 - A 03 - A 04 -
Example - Horners Rule
1 public class Example 2 public static int
horner (int a, int n, int x) 3 int result
an 4 for (int i n-1 i gt 0
i--) 5 result result x
ai 6 return result 7 8
34
Basic AxiomsA 01 - A 02 - A 03 - A 04 -
Recursive Methods - Factorial 1/2
1 public class Example 2 public static int
factorial (int n) 3 if (n 0) 4 return
1 5 else 6 return n factorial(n-1) 7
8
35
Basic AxiomsA 01 - A 02 - A 03 - A 04 -
Recursive Methods - Factorial 2/2
36
Basic AxiomsA 01 A 05 - Finding the
Largest Element of an Array 1/2
1 public class Example 2 public static int
max (int a) 3 int result a0 4 for
(int i 1 i lt a.length i) 5 if (ai gt
result) 6 result ai 7 return
result 8 9
37
Basic AxiomsA 01 A 05 - Finding the
Largest Element of an Array 2/2
pi probability that line 6 is executed
38
Asymtotic NotationDefinition 01 - An
Asymtotic Upper Bound - Big Oh
D 01 Consider a function f(n). We say that f(n)
is big oh g(n), which we write f(n) ? O(g(n)), if
there exists an integer n0 and a constant c gt 0
such that for all integers n ?? n0, 0 ? f(n) ?
cg(n)
cg(n)
f(n)
n0
39
Asymtotic NotationDefinition 01 - An
Asymtotic Upper Bound - Big OhAlternative
definition
D 01 Consider a function f(n) that is
non-negative for all integers n gt 0. We say that
f(n) is big oh g(n), which we write f(n)
O(g(n)), if there exists an integer n0 and a
constant c gt 0 such that for all integers n ?n0,
f(n) ? cg(n)
cg(n)
f(n)
n0
40
Asymtotic NotationDefinition 01 - An
Asymtotic Upper Bound - Big Oh
cg(n)
f(n)
n0
41
Asymtotic NotationAn Asymtotic Upper Bound -
Big Oh
f(n) 5n 50 We wish to show that f(n)
? O(n)
42
Asymtotic NotationAn Asymtotic Upper Bound -
Big Oh
f(n) 5n 50 We wish to show that f(n)
O(n2)
43
Asymtotic NotationT 01
T 01 If f1(n) O(g1(n)) and f2(n)
O(g2(n)), then f1(n) f2(n)
O(max(g1(n),g2(n))).
Proof
44
Asymtotic NotationExamples
45
Asymtotic NotationExample - 123n
Outer Body Level Loop
1 1 2 1 1
0 or 1 0 or 1 0 or 1
46
Asymtotic NotationExample - Selection sort
Outer Body Body Level Outer Inner Loop Lo
op
11 11 1 11 11 1 0 or
1 1 0 or 1 0 or 1 0 or 1
47
Asymtotic NotationNumber of statements
Inner loop for a certain i
Outer loop
Total number
48
Asymtotic NotationNumber of statements
49
Asymtotic NotationT 01
T 02 If f(n) f1(n) f2(n) where f1(n) and
f2(n) both are non-negative for all intergers n
gt 0 such that limn-gt? f2(n)/f1(n) L for some
limit L gt 0, then f(n) O(f1(n)).
Proof
50
Asymtotic NotationT 01 - Example
f(n) f1(n) f2(n) n3 n2
51
Asymtotic NotationT 03
T 03 If f1(n) O(g1(n)) and f2(n) O(g2(n))
then f1(n) x f2(n) O(g1(n) x g2(n)).
Proof
52
Asymtotic NotationT 03 - Example
f1(n) n3 n2 n 1 O(n3) f2(n) n2 n
1 O(n2) f1(n) x f2(n) O(n3 x
n2) O(n5)
53
Asymtotic NotationT 04
T 04 If f1(n) O(g1(n)) and g2(n) is a
function whose value is non-negative for integers
n gt 0, then f1(n) x g2(n) O(g1(n) x g2(n)).
Proof
54
Asymtotic NotationT 05 - Transitive Property
T 05 If f(n) O(g(n)) and g(n) O(h(n)) then
f(n) O(h(n)).
Proof
55
Asymtotic NotationT 05 - Example
f1(n) 7n3 O(n3) f2(n) 2n2
O(n2) f1(n) f2(n) O(f1(n)) because lim
f2(n)/ f1(n) 0 f1(n) f2(n) O(f1(n)) O(n3)
56
Asymtotic NotationT 06 - Polynomials
T 06
Proof
57
Asymtotic NotationLogarithms
logn lt n gt logn O(n)
58
Asymtotic NotationT 07 - Logarithms
T 07 For every integer k gt 1, logkn O(n).
Proof
59
Names of Common Big Oh Expressions
Expression Name -------------------------------
------------- O(1) constant O(log
n) logarithmic O(log2n) log
squared O(n) linear O(n log n) n log
n O(n2) quadratic O(n3) cubic O(2n)
exponential ----------------------------------
----------
60
Asymtotic Lower Bound - Omega
D 01 Consider a function f(n) that is
non-negative for all integers n gt 0. We say that
f(n) is omega g(n), which we write f(n)
?(g(n)), if there exists an integer n0 and a
constant c gt 0 such that for all integers n gt
n0, f(n) gt cg(n)
f(n)
cg(n)
n0
61
Asymtotic Upper and Lower Bound - Theta
D 01 Consider a function f(n) that is
non-negative for all integers n gt 0. We say that
f(n) is theta g(n), which we write f(n)
?(g(n)), if and only if f(n) O(g(n)) and f(n)
?(g(n)).
c2g(n)
f(n)
c1g(n)
n0
62
Asymtotic Lower Bound - Little Oh
D 01 Consider a function f(n) that is
non-negative for all integers n gt 0. We say that
f(n) is little oh g(n), which we write f(n)
o(g(n)), if and only if f(n) O(g(n)) and f(n)
? ?(g(n)).
cg(n)
f(n)
n0
63
Asymtotic NotationBig Oh - Omega - Theta
- Little Oh
64
Asymtotic NotationExample An Asymtotic Upper
and Lower Bound - Theta
f(n) n 3
65
Asymtotic NotationExample An Asymtotic Upper
and Lower Bound - Little Oh
f(n) n 3
66
Asymtotic NotationT 06 - Polynomials
T 08
Proof
67
Asymtotic NotationExample An Asymtotic Upper
and Lower Bound - Theta
f(n) amnm am-1nm-1 a1n a0
68
Asymtotic NotationRules for Big Oh Analysis
Sequence
S1 S2 Sm
O(max ( T1(n), T2(n), , Tm(n))
Iteration
for (S1 S2 S3) S4
O(max ( T1(n), T2(n) x (I(n)1), T3(n)
x I(n), T4(n) x I(n) )
Conditional
if (S1) S2 else S3
O(max ( T1(n), T2(n), T3(n) )
69
Actual Lower BoundsAssuming a 100 MHz Clock, c
1 Cycle, n0 0
70
RecursionOne recursive call with half size -
Binary Search
71
RecursionOne recursive call with half size -
Binary Search - Proof I
Proof
72
RecursionOne recursive call with half size -
Binary Search - Proof 2
Proof
73
RecursionTwo recursive calls with half size -
Sum
74
RecursionTwo recursive calls with half size -
Sum - Proof I
Proof
75
RecursionTwo recursive calls with size (about)
n-1 - Fibonacci number - Recursive
76
RecursionTwo recursive calls with size (about)
n-1 - Fibonacci number - Proof 1
Proof
77
RecursionFibonacci number - Iterative
78
RecursionMore than two recursive calls in every
level - Permutation
79
RecursionMore than two recursive calls in every
level - Permutation - Proof I
Proof
80
Recursive calls
81
Recursive callsExamples
82
Recursive callsProof
83
Complexity
Algorithm Running time --------------------
---------------------------------- Linear
search O(n) Binary search O(logn) Bubble
sort O(n2) Quick sort O(nlogn) -----
-------------------------------------------------
84
END