CS 312 Algorithm Efficiency

1
CS 312 Algorithm Efficiency

• Lecture 1 Algorithms Everywhere
• Problem
• Input Set
• Instance
• Problem Data
• Output Set
• Solution
• Algorithm
• Finite Domain
• Finite Range
• Must Terminate
• Correctness
• Elementary Operations
• Complexity

• Lecture 2 Algorithm Efficiency
• Why Rigorous Analysis?
• Efficiency
• Complexity
• Measuring Size of an Instance
• Orders of Growth
• Kinds of Efficiency
• Asymptotic Notation
• Big Q
• Big O
• Big W

2
Why Rigorous Analysis?

• A person well trained in computer science knows
  how to deal with algorithms how to construct
  them, manipulate them, understand them, analyze
  them. This knowledge is preparation for much
  more than writing good computer programs it is a
  general-purpose mental tool that will be a
  definite aid to the understanding of other
  subjects, whether they be chemistry, linguistics,
  or music, etc. The reason for this may be
  understood in the following way It has often
  been said that a person does not really
  understand something until after teaching it to
  someone else. Actually, a person does not really
  understand something until after teaching it to a
  computer, i.e. expressing it as an algorithmAn
  attempt to formalize things as algorithms leads
  to a much deeper understanding than if we simply
  try to comprehend things in the traditional way.
• Donald Knuth
• from Selected Papers on Computer Science,1996

3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
What Happened?
10
Analyze Rigorously
Choose any two digit number, add together both
digits And then subtract the total from your
original number.
11
Analyze Rigorously
Choose any two digit number, add together both
digits And then subtract the total from your
original number.
12
(No Transcript)
13
Not so Impressive!!
14
Efficiency

• Complexity is the cost of computing a solution

• (Work) Space

Measured in bits

• Time

How should this be measured?

• Want to measure algorithm, not implementation
• (computer speed, programming language,
  compiler quality, etc.)

• Define Elementary Operation
• Definition depends on the problem
• Examples Element Comparison for sorting
  algorithms
• Scalar Multiplication for matrix multiplication

• Model execution time as the total number of
  Elementary
• Operations, C, times the time it takes that
  particular
• implementation to execute an Elementary
  Operation, cop

15
Efficiency

• Complexity is the cost of computing a solution

• (Work) Space

Measured in bits

• Time

16
Efficiency

• Complexity is the cost of computing a solution

• (Work) Space

Measured in bits

• Time

• Efficiency is how complexity grows with the
• difficulty of the instance

• Difficulty means size of the instance, i.e.
  the
• number of parameters needed to completely
• characterize the instance (e.g. lists,
  matrices, etc.)

• Example we expect the complexity of sorting
• 100 numbers to be worse than sorting 10
  numbers

17
Efficiency

• Complexity is the cost of computing a solution

• (Work) Space

Measured in bits

• Time

• Efficiency is how complexity grows with the
• difficulty of the instance

• Difficulty means size of the instance, i.e.
  the
• number of parameters needed to completely
• characterize the instance (e.g. lists,
  matrices, etc.)

• Example we expect the complexity of sorting
• 100 numbers to be worse than sorting 10
  numbers

18
Orders of Growth

• Suppose

• How much longer will the algorithm run if we
  double the input size?

19
Orders of Growth

• Implementation constant cop is irrelevant
• Also note the coefficient on C(n) cancelled

• Order of growth describes the functional
• form of C(n) up to a constant multiple as
• n goes to infinity

20
Orders of Growth
21
Kinds of Efficiency

• Have machinery to measure how complexity grows
  with input size
• Need to decide which instance of a given size to
  use as the representative for that class

Algorithm Domain
Instances

• Average Case

1 2 3 4 5
Instance Size
22
Asymptotic Notation

• Definition given asymptotically nonnegative
  g(n),

23
Asymptotic Notation

• Definition given asymptotically nonnegative
  g(n),

24
Asymptotic Notation
Assignment Read Chap. 1-3 in Brassard