Evaluating

1
Evaluating Improving Algorithms
2
Analyzing US Change

• While Mgt0
• c lt- value of largest coin with value lt M
• give c coin to customer
• M lt- M-c
• Three statements per loop
• Loop executes k times, where k is
• Therefore, running time is 3k

3
Generalized US Change

• for i lt- 0 to d-1
• Calculate and give the max number of this
  coin (ci)
• How many times does this loop run? (k)
• How many operations are in calculate give max?
  (x)
• Resulting time is kx

4
General Rules for Calculating Time

• Steps in sequence are added
• A function is replaced by its steps (or count
  each call as 1 step)
• Total of steps in a loop gets multiplied by the
  of times the loop executes

5
Summary so far

• Original US change
• O(N) where N is coins given
• Generalized change algorithm
• O(N) where N is different coins in system
• But this doesnt work for all cases!

6
Brute Force Change

• N lt- 1
• While (not done)
• construct a combination of N coins
• if it adds up to M
• return it (donetrue)
• else
• N lt- N1

Brute Force try all combinations
7
How many loops?

• One loop for every combination with too few coins
  1! 2!
• One loop for every combination with the right
  number of coins before the correct one
• Best case for this part is 0
• Worst case for this part is k! where k is the
  number of coins in the solution

8
Summary and Analysis

• Original US change
• O(N) where N is coins given (linear)
• Generalized change algorithm
• O(N) where N is different coins
• O(1) for a fixed coin system (constant)
• Brute force change algorithm
• O (N N!) where N is coins given
• (Worse than 2N)!

9
What Weve Learned

• Brute force algorithms (try all possibilities)
  are guaranteed to work, but
• Brute force algorithms (usually) take too long
• We need ways to speed up the work

10
Detour Recursion

• Specify a solution to a problem in terms of
  solutions to one or more smaller problems
• Mathematically elegant

11
Recursive Fibonacci

• Mathematical definition
• F(0) 0
• F(1) 1
• F(N) F(N-1) F(N-2)
• Implemented directly
• F(4) F(3)F(2) F(2)F(1)F(2)
  F(1)F(0)F(1)F(2) 111F(2) 3F(1)F(0)
  311 5

12
A Tree of Recursive Calls
F(4)
F(3)
F(2)
F(2)
F(0)
F(1)
F(1)
1
1
1
F(1)
F(0)
1
1
13
Dynamic Programming

• Save intermediate results from which the final
  result can be computed
• Order the computations so you have all saved
  results before you need them
• Example our Fibonacci Algorithm

14
Dynamic Programming For Fibonacci

• Generate answers from smallest to largest
• Use previous answers in calculations
• fibonacci (n) O(n) where n is input
• f0 lt- 1
• f1 lt- 1
• for i lt- 2 to n-1 n times
• fi lt- fi-1fi-2 1 step
• return fn-1

15
Recursion Isnt Always Slower

• Recursive US Change (M)
• if (Mgt0)
• c lt- value of largest coin with value lt
  M
• give c coin to customer
• Recursive US Change(M-c)
• Only one recursive call, at end
• This is tail recursion, just as fast as the
  original loop

16
Greedy Algorithms

• Order your algorithm to solve the biggest chunk
  first
• Example always give the biggest coin possible in
  the change problem
• Not every greedy algorithm is correct (e.g.
  generalized change)

17
Branch and Bound

• Consider a brute force problem solution as a
  series of choices
• When a single choice is avoided (pruned), all
  future choices that depend on it are also avoided
• AI calls this heuristic search

18
Randomized

• Make each choice randomly until the problem is
  solved
• Not guaranteed to solve the problem
• Technically not even algorithms, if not
  guaranteed to halt (e.g. by time limit)
• Sometimes we can prove that a good enough
  solution is likely enough and these are simple
  to implement

19
Find the Phone (Brute Force)

• Find the phone
• if the phone is at your location
• return true (phone is found)
• else
• for each direction (left, right, forw,
  back)
• take a step
• if (find the phone)
• return true
• return false (phone not found here)

20
Find the Phone (Branch Bound)

• Find the phone
• if the phone is at your location
• return true (phone is found)
• else
• listen to the phone rule out opposite
  direction
• for each remaining direction
• take a step
• if (find the phone)
• return true
• return false (phone not found here)

21
Find the Phone (Greedy)

• Find the phone
• if the phone is at your location
• return true (phone is found)
• else
• listen pick best direction (left, right,
  forw, back)
• take a step
• if (find the phone)
• return true
• return false (phone not found here)
• What if theres an irreplaceable Ming vase
  between you and the phone?

22
Find the Phone (Randomized)

• Find the phone
• if the phone is at your location
• return true (phone is found)
• else
• repeat forever
• pick a random direction (left, right,
  forw, back)
• take a step
• if (find the phone)
• return true

23
Divide and Conquer

• If the problem is small enough, solve it directly
• Else
• Break the problem down into 2 or more subproblems
• Solve each one separately (recursive step)
• Combine the solutions
• Works best when subproblems are equal size

24
Divide Conquer Example

• Quick Sort
• Pick a value from the list (k)
• Divide the input into ltk list and gtk list
• Sort each list separately
• Build the result
• Sorted ltk list followed by k followed by sorted
  gtk list

25
Machine Learning

• Given many instances of input and output
• Discover a function that relates them
• Generalize to previously unseen inputs
• Generally need large amounts of representative
  data
• ML algorithms are fairly time consuming

26
Summary of Techniques

• Brute Force
• Try every possibility
• Rarely practical for reasonable size problems
• Dynamic programming
• Solve subproblems in the right order and save
  solutions to build the final answer

27
Summary (contd)

• Greedy Algorithms
• Take the biggest step first each time. Not
  always correct.
• Branch and Bound
• Rule out impossible choices and their successors,
  but try everything else
• Divide and Conquer
• Combine results from smaller subproblems

28
Summary (contd)

• Machine learning
• Use large amounts of data to automatically
  generate an appropriate solution algorithm
• Randomized algorithms
• Try solutions at random until one is found (or
  good enough)
• Not always completely random (e.g. genetic
  algorithms)