Algorithm Cost Algorithm Complexity - PowerPoint PPT Presentation

1 / 42
About This Presentation
Title:

Algorithm Cost Algorithm Complexity

Description:

Algorithm Cost. Algorithm Complexity. Lecture 28. Algorithm Cost. Back to Bunnies. Recall that we calculated Fibonacci Numbers using two different techniques ... – PowerPoint PPT presentation

Number of Views:124
Avg rating:3.0/5.0
Slides: 43
Provided by: ccGa
Category:

less

Transcript and Presenter's Notes

Title: Algorithm Cost Algorithm Complexity


1
Algorithm Cost Algorithm Complexity
Lecture 28
2
Algorithm Cost
3
Back to Bunnies
LB
  • Recall that we calculated Fibonacci Numbers using
    two different techniques
  • Recursion
  • Iteration

4
Back to Bunnies
LB
  • Recursive calculation of Fibonacci Numbers
  • Fib(1) 1
  • Fib(2) 1
  • Fib(N) Fib(N-1) Fib(N-2)
  • So
  • Fib(3) Fib(2) Fib(1)
  • 1 1
  • 2

5
Tree Recursion?
LB
f(n)
f(n-1)
f(n-2)
f(n-2)
f(n-3)
f(n-4)
f(n-3)
f(n-3)
f(n-4)
f(n-4)
f(n-5)
f(n-5)
f(n-6)
f(n-4)
f(n-5)
6
Tree Recursion Example
LB
f(6)
f(5)
f(4)
f(2)
f(4)
f(3)
f(3)
f(2)
f(3)
f(2)
f(1)
f(2)
f(1)
f(2)
f(1)
7
Recursively
LB
  • public static int fibR(int n)
  • if(n 1 n 2)
  • return 1
  • else
  • return fibR(n-1) fibR(n-2)

8
Iteratively
LB
  • public static int fibI(int n)
  • int oldest 1
  • int old 1
  • int fib 1
  • while(n-- gt 2)
  • fib old oldest
  • oldest old
  • old fib
  • return fib

9
Slight Modifications
LB
public static int fibI(int n) int
oldest 1 int old 1 int fib 1
while(n-- gt 2) fib old oldest
oldest old old fib
return fib
Add Counters
public static int fibR(int n) if(n 1 n
2) return 1 else return fibR(n-1)
fibR(n-2)
10
Demo
LB
11
Conclusion
LB
  • Algorithm choice or design can make a big
    difference!

12
Correctness is Not Enough
  • It isnt sufficient that our algorithms perform
    the required tasks.
  • We want them to do so efficiently, making the
    best use of
  • Space
  • Time

13
Time and Space
  • Time
  • Instructions take time.
  • How fast does the algorithm perform?
  • What affects its runtime?
  • Space
  • Data structures take space.
  • What kind of data structures can be used?
  • How does the choice of data structure affect the
    runtime?

14
Time vs. Space
  • Very often, we can trade space for time
  • For example maintain a collection of students
    with SSN information.
  • Use an array of a billion elements and have
    immediate access (better time)
  • Use an array of 35 elements and have to search
    (better space)

15
The Right Balance
  • The best solution uses a reasonable mix of space
    and time.
  • Select effective data structures to represent
    your data model.
  • Utilize efficient methods on these data
    structures.

16
Questions?
17
Algorithm Complexity
18
Scenarios
  • Ive got two algorithms that accomplish the same
    task
  • Which is better?
  • Given an algorithm, can I determine how long it
    will take to run?
  • Input is unknown
  • Dont want to trace all possible paths of
    execution
  • For different input, can I determine how an
    algorithms runtime changes?

19
Measuring the Growth of Work
  • While it is possible to measure the work done
    by an algorithm for a given set of input, we need
    a way to
  • Measure the rate of growth of an algorithm based
    upon the size of the input
  • Compare algorithms to determine which is better
    for the situation

20
Introducing Big O
LB
  • Will allow us to evaluate algorithms.
  • Has precise mathematical definition
  • We will use simplified version in CS 1311
  • Caution for the real world Only tells part of
    the story!
  • Used in a sense to put algorithms into families

21
Why Use Big-O Notation
  • Used when we only know the asymptotic upper
    bound.
  • If you are not guaranteed certain input, then it
    is a valid upper bound that even the worst-case
    input will be below.
  • May often be determined by inspection of an
    algorithm.
  • Thus we dont have to do a proof!

22
Size of Input
  • In analyzing rate of growth based upon size of
    input, well use a variable
  • For each factor in the size, use a new variable
  • N is most common
  • Examples
  • A linked list of N elements
  • A 2D array of N x M elements
  • A Binary Search Tree of P elements

23
Formal Definition
  • For a given function g(n), O(g(n)) is defined to
    be the set of functions
  • O(g(n)) f(n) there exist positive
    constants c and n0 such that 0 ? f(n) ?
    cg(n) for all n ? n0

24
Visual O() Meaning
cg(n)
Upper Bound
f(n)
f(n) O(g(n))
Work done
Our Algorithm
n0
Size of input
25
Simplifying O() Answers(Throw-Away Math!)
  • We say 3n2 2 O(n2) ? drop constants!
  • because we can show that there is a n0 and a c
    such that
  • 0 ? 3n2 2 ? cn2 for n ? n0
  • i.e. c 4 and n0 2 yields
  • 0 ? 3n2 2 ? 4n2 for n ? 2

26
Correct but Meaningless
  • You could say
  • 3n2 2 O(n6) or 3n2 2 O(n7)
  • But this is like answering
  • Whats the world record for the mile?
  • Less than 3 days.
  • How long does it take to drive to Chicago?
  • Less than 11 years.

27
Comparing Algorithms
  • Now that we know the formal definition of O()
    notation (and what it means)
  • If we can determine the O() of algorithms
  • This establishes the worst they perform.
  • Thus now we can compare them and see which has
    the better performance.

28
Comparing Factors
N2
N
Work done
log N
1
Size of input
29
Correctly Interpreting O()
  • O(1) or Order One
  • Does not mean that it takes only one operation
  • Does mean that the work doesnt change as N
    changes
  • Is notation for constant work
  • O(N) or Order N
  • Does not mean that it takes N operations
  • Does mean that the work changes in a way that is
    proportional to N
  • Is a notation for work grows at a linear rate

30
Complex/Combined Factors
  • Algorithms typically consist of a sequence of
    logical steps/sections
  • We need a way to analyze these more complex
    algorithms
  • Its easy analyze the sections and then combine
    them!

31
Example Insert in a Sorted Linked List
  • Insert an element into an ordered list
  • Find the right location
  • Do the steps to create the node and add it to the
    list

//
head
17
38
142
Step 1 find the location O(N)
Inserting 75
32
Example Insert in a Sorted Linked List
  • Insert an element into an ordered list
  • Find the right location
  • Do the steps to create the node and add it to the
    list

//
head
17
38
142
75
Step 2 Do the node insertion O(1)
33
Combine the Analysis
  • Find the right location O(N)
  • Insert Node O(1)
  • Sequential, so add
  • O(N) O(1) O(N 1)

34
Example Search a 2D Array
  • Search an unsorted 2D array (row, then column)
  • Traverse all rows
  • For each row, examine all the cells (changing
    columns)

Row
1 2 3 4 5
O(N)










1 2 3 4 5 6 7 8 9 10
Column
35
Example Search a 2D Array
  • Search an unsorted 2D array (row, then column)
  • Traverse all rows
  • For each row, examine all the cells (changing
    columns)

Row
1 2 3 4 5



















1 2 3 4 5 6 7 8 9 10
Column
O(M)
36
Combine the Analysis
  • Traverse rows O(N)
  • Examine all cells in row O(M)
  • Embedded, so multiply
  • O(N) x O(M) O(NM)

37
Sequential Steps
  • If steps appear sequentially (one after another),
    then add their respective O().
  • loop
  • . . .
  • endloop
  • loop
  • . . .
  • endloop

N
O(N M)
M
38
Embedded Steps
  • If steps appear embedded (one inside another),
    then multiply their respective O().
  • loop
  • loop
  • . . .
  • endloop
  • endloop

O(NM)
M
N
39
Correctly Determining O()
  • Can have multiple factors
  • O(NM)
  • O(logP N2)
  • But keep only the dominant factors
  • O(N NlogN) ?
  • O(NM P)?
  • O(V2 VlogV) ?
  • Drop constants
  • O(2N 3N2) ?

O(NlogN)
remains the same
O(V2)
? O(N2)
O(N N2)
40
Summary
  • We use O() notation to discuss the rate at which
    the work of an algorithm grows with respect to
    the size of the input.
  • O() is an upper bound, so only keep dominant
    terms and drop constants

41
Questions?
42
(No Transcript)
Write a Comment
User Comments (0)
About PowerShow.com