Today

1
Todays Objectives
Week 5

• Announcements
• Hand in Homework 1
• Homework 2 was posted on the Schedule last
  weekend, due on 28-Feb
• Analysis tools Chapter 4 (Section 4.3 will not
  be on the exam)?
• Measuring running time
• Asymptotic analysis
• Pseudocode and counting primitive operations
• Simplified steps for analyzing algorithms
• Best, worst, and average cases
• Big-Oh, Big-Omega, Big Theta
• Stacks and Queues Chapter 5.1 and 5.2
• Stack a LIFO data structure
• Queue a FIFO data structure
• Implementing a queue that holds data in a
  circular array

2
Analysis Tools

3
Data Structures Algorithms
Analysis Tools (Goodrich, 108)?

• Good data structures and algorithms are necessary
  for efficiency and speed in our applications
• What makes a data structure or algorithm good?
  How do we measure them?
• Easy to write? Short? Fast? Efficient?

4
Data Structures Algorithms
Analysis Tools (Goodrich, 108)?

• Good data structures and algorithms are necessary
  for efficiency and speed in our applications
• What makes a data structure or algorithm good?
  How do we measure them?
• Easy to write? Short? Fast? Efficient?
• The important measures
• Fast running time
• Efficient space usage

5
Food for Thought
Analysis Tools (Handout Eisner, email note)?

• Scenario description

6
Running Time
Analysis Tools (Goodrich, 109110)?

• The most important measure
• Absolute running time is dependent on the
  computer, the language, the compiler, the
  operating system, etc.
• Increases when the input size (n) increases
• Evaluating running time
• One possible approach is to measure the running
  time experimentally
• Use the same platform when testing different
  algorithms
• Use inputs of different sizes

7
Measuring Running Time Experimentally
Analysis Tools (Goodrich, 109110)?

• Test with inputs of different sizes
• Plot the running time vs. input size (n)?

Running Time f(n)?
0
Input Size (n)?
8
Experimental Results 1
Analysis Tools
Pentium III, 866 MHz
Pentium 4, 2.4 GHz
Testing push() with ArrayStack
9
Lessons Learned 1
Analysis Tools

• We cannot provide an exact running time for the
  algorithm because the results are different on
  different computers
• The running time increases as the input size
  increases

10
Experimental Results 2
Analysis Tools
Pentium III, 866 MHz
Pentium 4, 2.4 GHz
Testing push() with NodeStack and ArrayStack
11
Lessons Learned 2
Analysis Tools

• The data structure that we choose for our
  algorithm can have a big influence on the
  algorithms speed
• A stack implemented with an array is much faster
  than a stack implemented with a linked list of
  Nodes

12
Drawbacks ofExperimental Measurement
Analysis Tools (Goodrich, 110)?

• Results may be different for input sets that were
  not tested
• Tests may not have used the same hardware or
  software, so we may not be able to make good
  comparisons
• We have to code the algorithm before we can test
  it

13
General Analysis Method
Analysis Tools (Goodrich, 110)?

• Advantages
• All possible inputs are considered
• Characterizes running time as a function of the
  input size n
• Independent of hardware or software environment
• Can be used to evaluate an algorithm without
  having to code it
• Find a function f that describes the algorithms
  running time as a function of the input size n

14
Asymptotic Analysis
Analysis Tools (Goodrich, 166)?

• Developed by computer scientists for analyzing
  the running time of algorithms
• Describes the running time of an algorithm in
  terms of the size of the input in other words,
  how well does it scale as the input gets larger
• The most common metric used is
• Upper bound of an algorithm
• Called the Big-Oh of an algorithm, O(n)?

15
Algorithm Analysis
Analysis Tools (Goodrich web page)?

• Use asymptotic analysis
• Assume that we are using an idealized computer
  called the Random Access Machine (RAM)?
• CPU
• Unlimited amount of memory
• Accessing a memory cell takes one unit of time
• Primitive operations take one unit of time
• Perform the analysis on the pseudocode

16
Pseudocode
Analysis Tools (Goodrich, 48, 166)?

• High-level description of an algorithm
• Structured
• Not as detailed as a program
• Easier to understand (written for a human)?

Algorithm arrayMax(A,n) Input An array A
storing n gt 1 integers. Output The maximum
element in A. currentMax ? A0 for i ? 1 to n
1 do if currentMax lt Ai then currentMax ?
Ai return currentMax
17
Pseudocode Details
Analysis Tools (Goodrich, 48)?

• Declaration
• Algorithm functionName(arg,)?
• Input
• Output
• Control flow
• ifthenelse
• whiledo
• repeatuntil
• fordo
• Indentation instead of braces
• Return value
• return expression
• Expressions
• Assignment (like ? in Java)?
• Equality testing (like ?? in Java)?

18
Pseudocode Details
Analysis Tools (Goodrich, 48)?

• Declaration
• Algorithm functionName(arg,)?
• Input
• Output
• Control flow
• ifthenelse
• whiledo
• repeatuntil
• fordo
• Indentation instead of braces
• Return value
• return expression
• Expressions
• Assignment (like ? in Java)?
• Equality testing (like ?? in Java)?

Tips for writing an algorithm Use the correct
data structure Use the correct ADT operations Use
object-oriented syntax Indent clearly Use a
return statement at the end
19
Primitive Operations
Analysis Tools (Goodrich, 164165)?

• To do asymptotic analysis, we can start by
  counting the primitive operations in an algorithm
  and adding them up
• Primitive operations that we assume will take a
  constant amount of time
• Assigning a value to a variable
• Calling a function
• Performing an arithmetic operation
• Comparing two numbers
• Indexing into an array
• Returning from a function
• Following a pointer reference

20
Counting Primitive Operations
Analysis Tools (Goodrich, 166)?

• Inspect the pseudocode to count the primitive
  operations as a function of the input size (n)?

• Algorithm arrayMax(A,n)
• currentMax ? A0
• for i ? 1 to n 1 do
• if currentMax lt Ai then
• currentMax ? Ai
• return currentMax

• Count
• Array indexing Assignment 2
• Initializing i 1
• Verifying iltn n
• Array indexing Comparing 2(n-1)?
• Array indexing Assignment 2(n-1) worst
• Incrementing the counter 2(n-1)?
• Returning 1

Best case 21n4(n1)1 5n
Worst case 21n6(n1)1 7n-2
21
Best, Worst, or Average Case
Analysis Tools (Goodrich, 165)?

• Fig. 4.4, Goodrich, p. 165
• A program may run faster on some input data than
  on others
• Best case, worst case, and average case are terms
  that characterize the input data
• Best case the data is distributed so that the
  algorithm runs fastest
• Worst case the data distribution causes the
  slowest running time
• Average case very difficult to calculate
• We will concentrate on analyzing algorithms by
  identifying the running time for the worst case
  data

22
Estimating theRunning Time of arrayMax
Analysis Tools (Goodrich, 166)?

• Worst case running time of arrayMax
• f(n) (7n 2) primitive operations
• Actual running time depends on the speed of the
  primitive operationssome of them are faster than
  others
• Let t speed of the slowest primitive operation
• Let f(n) the worst-case running time of
  arrayMax
• f(n) t (7n 2)?

23
Growth Rate of arrayMax
Analysis Tools (Goodrich, 166)?

• Growth rate of arrayMax is linear
• Changing the hardware alters the value of t, so
  that arrayMax will run faster on a faster
  computer
• Growth rate is still linear

Slow PC 10(7n-2)?
Fast PC 5(7n-2)?
Fastest PC 1(7n-2)?
24
Tests of arrayMax
Analysis Tools

• Does it really work that way?
• Used arrayMax algorithm from p.166
• Tested on two PCs
• Yes, results both show a linear growth rate

Pentium III, 866 MHz
Pentium 4, 2.4 GHz
25
Big-Oh
Algorithm Analysis (Goodrich, 167)?

• Developed by computer scientists for analyzing
  the running time of algorithms
• Describes the running time in terms of the size
  of the input n
• In other words, how well does it scale as the
  input gets very large
• Upper bound of the growth of an algorithms
  running time

26
Growth Rates ofCommon Classes of Functions
Analysis Tools (Goodrich, 161)?
Exponential O(2n)?
Quadratic O(n2)?
Linear O(n)?
Logarithmic O(log n)?
Constant O(1)?
27
(No Transcript)
28
(No Transcript)
29
Big-Oh Notation
Analysis Tools (Goodrich, 167)?

• Definition
• Let f(n) and g(n) be functions mapping
  nonnegative integers to real numbers.
• f(n) is O(g(n)) if there is a real constant
  cgt0and an integer constant n0?1, such that f(n)
  ??cg(n) for every integer n?n0

30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
More About Asymptotic Analysis
Analysis Tools (Goodrich, 168169)?

• Simplify the count to get the Big-Oh
• Drop all lower-order terms
• 7n 2 7n
• Eliminate constants
• 7n n
• Remaining term is the Big-Oh
• 7n 2 is O(n)?

39
Another Example
Analysis Tools (Goodrich, 168169)?

• Example f(n) 5n3 2n2 1
• Drop all lower order terms
• 5n3 2n2 1 5n3
• Eliminate the constants
• 5n3 n3
• The remaining term is the Big-Oh
• f(n) is O(n3)?

40
Using the Big-Oh Notation
Analysis Tools (Goodrich, 168169)?

• The correct way to use Big-Oh notation is to say
  f(n) is O(g(n))?
• NOT f(n) ? O(g(n))?
• NOT f(n) O(g(n))?
• Since Big-Oh describes the upper bound, we can
  say that 7n 2 is O(n2)?
• It is also true if we say that 7n 2 is O(7n
  2)?
• However, both of these statements are not
  considered accurate
• It is customary to pick the smallest order when
  describing Big-Oh

41
Rules for Simplifying
Analysis Tools

• The most important rules
• We can drop the constants
• In a polynomial, the term with the highest degree
  establishes the Big-Oh
• The sum rule for a sequential loops add their
  Big-Oh values
• For nested loops, multiply their Big-Oh values

42
Finding the Big-Oh
Analysis Tools

• We do not need to use the Big-Oh definition to
  find the Big-Oh
• We do not need to count all the primitive
  operations
• We need to count only the operations that have
  the greatest effect on growth rate of the running
  time that usually means the loops.
• In Big-Oh analysis, we focus on the big picture
  
• for( int i0 iltn i )
• myArrayi 0

43
Steps for Analyzing an Algorithm
Algorithm Analysis

• Locate the loops
• Determine the Big-Oh by counting the number of
  iterations in terms of n
• Sequential loops Add their Big-Oh values
• Nested loops Start analyzing with the inner
  loop, multiply their Big-Oh values
• Are there branch statements (e.g., ifelse)?
• Use the runtime of the biggest branch
• If the result is a polynomial, the term with the
  highest degree establishes the Big-Oh

44
Finding the Big-Oh
Algorithm Analysis

• for( int i0 iltn i )
• myArrayi 0
• for( int i0 iltn i )
• for(int j0 jltn j)
• myArrayij 0

45
Finding the Big-Oh
Analysis Tools

• sum 0
• for( int i1 iltn i )
• for(int j1 jlti j)
• sum

46
PrefixAverages Example
Algorithm Analysis (Goodrich, 174)?

• The problem
• Given array X that stores numbers
• Compute the numbers in Array A
• Each number Ai is the average of the numbers in
  X from X0 to Xi

X
1
3
5
7
9
X0 X1 X2 9
9 / 3 3
1
2
3
4
5
A
47
Quadratic Time Solution
Algorithm Analysis (Goodrich, 174)?

• Algorithm prefixAverages1
• for i ? 0 to n-1 do
• a ? 0
• for j ? 0 to i do
• a ? a Xj
• Ai ? a/(i1)?
• Two nested loops

Inner loop loops through X, adding the numbers
from element 0 through element i
Outer loop loops through A, calculating the
averages and putting the result into Ai
48
Linear Time Solution
Algorithm Analysis (Goodrich, 175)?

• Algorithm prefixAverages2
• s ? 0
• for i ? 0 to n-1 do
• s ? s Xi
• Ai ? s/(i1)?
• Only one loop

Sum keeps track of the sum of the numbers in
X so that we dont have to loop through X every
time
Loop loops through A, adding to the sum,
calculating the averages, and putting the
result into Ai
49
Lessons Learned
Algorithm Analysis

• Both algorithms correctly solved the problem
• Lesson There may be more than one way to write
  your program.
• One of the algorithms was significantly faster
• Lesson The algorithm that we choose can have a
  big influence on the programs speed.
• Evaluate the solution that you pick, and ask
  whether it is the most efficient way to do it.

50
Best, Worst, and Average Cases
Algorithm Analysis (Goodrich, 122)?

• Fig. 4.4, Goodrich, p. 165
• A program may run faster on some inputs than on
  others
• Best case the data allows the algorithm to work
  as fast as it can
• Worst case the data causes the algorithm to
  perform poorly
• Not the same as upper bound and lower bound
• Algorithms can have different complexities,
  depending on the data that they work with
• Examples
• The running time of this algorithm is O(nlogn)
  in the best case
• The running time of this algorithm is O(n2) in
  the worst case

51
Relatives of Big-Oh
Algorithm Analysis (Goodrich, 170)?

• Big-Oh
• f(n) is O(g(n)) if f(n) ? cg(n) for every integer
  n?n0
• Upper Bound f(n) is less than or equal to
  cg(n)?
• Big-Omega
• f(n) is ?(g(n)) if f(n) ? cg(n) for every integer
  n?n0
• Lower Bound f(n) is greater than or equal to
  cg(n)?
• Big-Theta
• f(n) is ?(g(n))
• if c'g(n) ? f(n) ? c''g(n) for every integer n?n0
• Two functions are asymptotically equal.
• Exact specification of the growth of runtime

52
(No Transcript)
53
Big-Omega
Analysis Tools (Goodrich, 170, Sedgewick, 62)?

• Big-Omega
• f(n) is ?(g(n)) if f(n) ? cg(n) for every integer
  n?n0
• Lower Bound f(n) is greater than or equal to
  cg(n)?
• We are usually only interested in the lower bound
  for a given problem, not a specific algorithm
• If we can prove that there is a bound where the
  performance cannot improve, then we can stop
  seeking a better algorithm
• Example
• Problem How fast is it possible to sort?
• Answer The lower bound for sorting algorithms
  that rely on comparing key values is ?(nlogn)?
• Since the Bubble Sort is O(n2), we know we can do
  better
• Insertion Sort is O(nlogn) in the best case, but
  still O(n2) in worst case
• Heap Sort is O(nlogn) in the worst case, so we
  know we cannot get better performance from a sort
  that compares key values

54
More Relatives of Big-Oh
Algorithm Analysis

• little-oh
• f(n) is o(g(n)) if f(n) is strictly less than
  g(n)?
• In other words, f(n) is not ?(g(n))?
• little-omega
• f(n) is ?(g(n)) if f(n) is strictly greater than
  g(n)?
• In other words, f(n) is not ?(g(n))?
• Not used very much

55
Stack

• LIFO data structure

56
Stack
Stacks (Goodrich, 188)?

• Container that stores data elements that are
  inserted or removed in LIFO order (last-in
  first-out)?
• How are the data related?
• Sequential (sequence is determined by the
  insertion order)?
• Operations
• push, pop, top, size, isEmpty

57
Some Uses of Stacks
Stacks (Goodrich, 188)?

• Backtracking
• Examples
• Storing addresses in a Web browsers history
• Implementing an undo feature by storing changes
• The run-time stack
• Reversing data
• Example
• Solving the palindrome problem
• Parsing
• Examples
• Check for matching brackets in a mathematical
  expression, e.g. ((3n1)4)
• Check whether the elements in an XML document are
  properly nested

58
(No Transcript)
59
The Run-Time Stack
Stacks

• An important use for a stack
• Every Java program has its own stack called the
  run-time stack
• The run-time stack is used to keep track of the
  details when a function is invoked
• Details are stored in a stack frame that is
  pushed onto the run-time stack
• When the function finishes, the stack frame is
  popped off the run-time stack and execution
  backtracks

60
(No Transcript)
61
Queue

• FIFO data structure

62
Queue
Queues (Goodrich, 204)?

• Container that stores data elements that are
  inserted or removed in FIFO order (first-in
  first-out)?
• (Like a line of people waiting for a theater
  ticket.)?
• How are the data related?
• Sequential (sequence is determined by the
  insertion order)?
• Operations
• enqueue inserts an element at the rear of a
  queue
• dequeue removes an element from the front of a
  queue
• front returns a reference to the element at
  the front
• size
• isEmpty

63
(No Transcript)
64
Some Uses of Queues
Queues (Goodrich, 205)?

• How do operating systems use queues?
• Holding jobs in order until they can be serviced
• Examples
• Print job spooler
• Process scheduling
• How do customer service applications use queues?
• Holding customer transactions in the order that
  they were submitted until they can be processed
• Example
• Holding transactions in an online airline
  reservation system in the order that they were
  submitted
• How do simulations use queues?
• Can be used in a computerized model to simulate
  the arrival of customers and the customer
  processing time
• Example
• Modeling a grocery business to decide whether to
  expand the number of checkout lanes
• Queues can be used to categorize data
• Rearranging data without destroying the sequence
• Example
• Separating runners into age groups at the finish
  line, without destroying their order of arrival

65
(No Transcript)
66
(No Transcript)
67
(No Transcript)
68
(No Transcript)
69
(No Transcript)
70
(No Transcript)
71
(No Transcript)
72
(No Transcript)
73
(No Transcript)
74
(No Transcript)
75
(No Transcript)
76
(No Transcript)
77
(No Transcript)
78
(No Transcript)
79
(No Transcript)
80
References

• Eisner, Jason, Johns Hopkins University,
  Baltimore, MD.
• Goodrich, M. T. and R. Tamassia, Data Structures
  and Algorithms in Java. Hoboken, NJ John Wiley
  Sons, Inc., 2006.
• Sedgewick, R., Algorithms in C, Third Edition.
  Boston Addison-Wesley, 1998.