Introduction to Greedy Algorithms

1
Introduction to Greedy Algorithms

• The greedy technique
• Problems explored
• The coin changing problem
• Activity selection

2
Optimization problems

• An optimization problem
• Given a problem instance, a set of constraints
  and an objective function.
• Find a feasible solution for the given instance
  for which the objective function has an optimal
  value
• either maximum or minimum depending on the
  problem being solved.
• A feasible solution satisfies the problems
  constraints
• The constraints specify the limitations on the
  required solutions.
• For example in the knapsack problem we require
  that the items in the knapsack will not exceed a
  given weight

3
The Greedy Technique(Method)

• Greedy algorithms make good local choices in the
  hope that they result in an optimal solution.
• They result in feasible solutions.
• Not necessarily an optimal solution.
• A proof is needed to show that the algorithm
  finds an optimal solution.
• A counter example shows that the greedy algorithm
  does not provide an optimal solution.

4
Pseudo-code for Greedy Algorithm

• set Greedy (Set Candidate)
• solution new Set( )
• while (Candidate.isNotEmpty())
• next Candidate.select() //use selection
  criteria,
• //remove from Candidate and
  return value
• if (solution.isFeasible( next)) //constraints
  satisfied
• solution.union( next)
• if (solution.solves()) return solution
• //No more candidates and no solution
• return null

5
Pseudo code for greedy cont.

• select() chooses a candidate based on a local
  selection criteria, removes it from Candidate,
  and returns its value.
• isFeasible() checks whether adding the selected
  value to the current solution can result in a
  feasible solution (no constraints are violated).
• solves() checks whether the problem is solved.

6
Coin changing problem

• Problem Return correct change using a minimum
  number of coins.
• Greedy choice coin with highest coin value
• A greedy solution (next slide)
• American money
• The amount owed 37 cents.
• The change is 1 quarter, 1 dime, 2 cents.
• Solution is optimal.
• Is it optimal for all sets of coin sizes?
• Is there a solution for all sets of coin sizes?

(12,D,N,P/15)
7
A greedy solution

• Input Set of coins of different denominations,
  amount-owed
• change
• while (more coin-sizes valueof(change)ltamount-o
  wed)
• Choose the largest remaining coin-size //
  Selection
• // feasibility check
• while (adding the coin does not make the
  valueof(change) exceed the amount-owed ) then
• add coin to change
• //check if solved
• if ( valueof(change) equals
  amount-owed)return change
• else delete coin-size
• return failed to compute change

8
Elements of the Greedy Strategy

• Cast problem as one in which we make a greedy
  choice and are left with one subproblem to solve.
• To show optimality
• Prove there is always an optimal solution to
  original problem that makes the greedy choice.

9
Elements of the Greedy Strategy

• 2. Demonstrate that what remains is a subproblem
  with property
• If we combine the optimal solution of the
  subproblem with the greedy choice we have an
  optimal solution to original problem.

10
Activity Selection

• Given a set S of n activities with start time si
  and finish time fi of activity i
• Find a maximum size subset A of compatible
  activities (maximum number of activities).
• Activities are compatible if they do not overlap
• Can you suggest a greedy choice?

11
Example
Activities 1 2 3 4 5 6 7
11 13
2 12
3 10
11 15
3 7
1 4
0 2
Time
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
12
Counter Example 1

• Select by start time

Activities 1 2 3
11 15
1 4
0
15
Time
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
13
Counter Example 2

• Select by minimum duration

Activities 1 2 3
8 15
1 8
7 9
Time
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
Select by finishing time
Activities 1 2 3 4 5 6 7
11 13
2 12
3 10
11 15
3 7
1 4
0 2
Time
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
15
Activity Selection

• Assume without loss of generality that we number
  the intervals in order of finish time. So
  f1...fn.
• Greedy choice choose activity with minimum
  finish time
• The following greedy algorithm starts with A1
  and then adds all compatible jobs. (Theta(n))
• Theta(nlogn) when including sort

16
Greedy-Activity-Selector(s,f)

• n lt- lengths // number of activities
• A lt- 1
• j lt- 1 //last activity added
• for i lt- 2 to n //select
• if si gt fj then //compatible (feasible)
• add i to A
• j lt- i //save new last activity
• return A

17
Proof that greedy solution is optimal

• It is easy to see that there is an optimal
  solution to the problem that makes the greedy
  choice.
• Proof of 1.
• Let A be an optimal solution. Let activity 1 be
  the greedy choice. If 1 ? A the proof is done. If
  1 ?A, we will show that AA-a1 is another
  optimal solution that includes 1.
• Let a be the activity with minimum finish time
  in A.
• Since activities are sorted by finishing time in
  the algorithm, f(1)?? f(a). If f(1)?? s(a) we
  could add 1 to A and it could not be optimal. So
  s(1) lt f(a), and 1 and a overlap. Since f(1)??
  f(a), if we remove a and add 1 we get another
  compatible solution AA-a1 and AA

18
Proof that greedy solution is optimal

• 2. If we combine the optimal solution of the
  remaining subproblem with the greedy choice we
  have an optimal solution to the original problem.
• Proof of 2.
• Let activity 1 be the greedy choice.
• Let S be the subset of activities that do not
  overlap with 1.
• Sii 1,,n and si?
  f(1).
• Let B be an optimal solution for S.
• From the definition of S, A1B is
  compatible, and a solution to the original
  problem.

19
Proof that greedy solution is optimal

• 2. If we combine the optimal solution of the
  remaining subproblem with the greedy choice we
  have an optimal solution to the original problem.
• Proof of 2 continued.
• The proof is by contradiction.
• Assume that A is not an optimal solution to the
  original problem.
• Let A be an optimal solution that contains 1.
• So AltA, and A-1gtA-1B.
• But A-1 is also a solution to the problem of
  S, contradicting the assumption that B is an
  optimal solution to S.