Greedy technique

1

• LECTURE 9
• Greedy technique

2
Outline

• Optimization problems
• Basic idea of greedy technique
• Examples
• Correctness verification and efficiency analysis
• Some classical applications

3
Optimization problems

• The general structure of a optimization problem
  is
• Find x in X such that
• x satisfies some constraints
• x optimizes (minimizes or maximizes) a criterion
• Particular case
• X is a finite set the problem is a
  combinatorial optimization problem
• At the first sight such a problem should be easy
  to solve. However it is not so simple.

4
Optimization problems

• Example.
• Let Aa1,,an and Clta1an
• Find S subset of A such that
• The sum of elements of S is less than C
  (constraint)
• The cardinal of S is minimal (optimization
  criterion)
• Remark. X the set of all 2n subsets of A
• A brute force approach is of O(2n). A more
  efficient method should be found

5
Outline

• Optimization problems
• Basic idea of greedy technique
• Examples
• Correctness verification and efficiency analysis
• Some classical applications

6
Basic idea of greedy technique

• Lets reformulate the optimization problem as
  follows
• Let A(a1,,an) be a multiset (a set of not
  necessarily distinct elements). Find
  S(s1,,sk)?A such that S satisfies some
  constraints and optimizes a criterion.
• Basic idea of greedy technique
• S is constructed successively starting with the
  first element
• At each step a new element (that element which
  seems to be the best at that moment) is selected
  from A.
• Once a choice is made it is final (the greedy
  approach do not back track)

7
Basic idea of greedy technique

• The general structure of a greedy algorithm
• Greedy(A)
• S?
• WHILE S is not completed AND there exists
  unselected elements in A DO
• choose the best currently available
  element a from A
• IF by adding a to S the constraints are
  still satisfied
• THEN add a to S
• RETURN S

8
Basic idea of greedy technique

• The most important element of a greedy algorithm
  is the selection of an element at each step.
• The elements are selected based on a selection
  criterion which is established depending on the
  problem.
• The selection criterion is frequently based on
  some heuristics ( technique based on
  experiential data and intuition rather than on
  theoretical analysis)

9
Outline

• Optimization problems
• Basic idea of greedy technique
• Examples
• Correctness verification and efficiency analysis
• Some classical applications

10
Examples

• Problem 1(maximal sum subset of a given cardinal)
• Find a subset S of a finite multiset A such that
• S has mltcard A elements
• The sum of elements in S is maximal
• Example
• Let A5,1,7,5,4 and m3.
• Then S5,5,7

11
Examples

• Greedy approach sort decreasingly the elements
  of A and select the first m elements

Subset(A1..n,m) A1..ndecreasing_sort(A1..n
) FOR i1,m DO SiAi RETURN S1..m
Subset(A1..n,m) FOR i1,m DO ki FOR
ji1,n DO IF AkltAj THEN kj IF
kltgti THEN Ak?Ai SiAi RETURN S1..m
Remark. One can prove that for this problem the
greedy strategy always produces an optimal
solution
12
Examples

• Problem 2 (coin changing problem)
• By using coins of values v1,v2,,vn we want
  to find a subset of coins that total the amount C
  and the number of used coins is minimal
• Lets denote by ki the number of coins of
  denomination vi
• Constraint k1v1knvnC
• Optimization criterion k1k2kn is minimal
• Greedy approach starting from the coin of the
  largest denomination try to cover as much as
  possible from the initial amount C continues by
  choosing the next largest value and so on

13
Examples

• Coins(v1..n,C)
• v1..ndecreasing_sort(v1..n)
• FOR i1,n DO Si0
• i1
• WHILE Cgt0 and iltn DO
• SiC DIV vi
• CC MOD vi
• ii1
• IF C0 THEN RETURN S1..n
• ELSE the problem has no solution

14
Examples

• Remarks
• Sometimes the problem has not a solution
• Example V(20,10,5) and C17
• However, if we have coins of value 1, then the
  problem has always a solution
• Sometimes the greedy approach doesnt give an
  optimal solution
• Example V(25,20,10,5,1), C40
• Greedy approach (1,0,1,1,0)
• Non-greedy approach (0,2,0,0,0)
• A sufficient condition for optimality is
  v1gtv2gtgtvn 1and vi-1di-1vi

15
Examples

• So the greedy approach leads to
• simple and intuitive algorithms
• efficient algorithms
• But
• it does not always lead to an optimal solution (a
  locally greedy choice can have negative global
  consequen-ces)
• sometimes the non-optimal solutions are close to
  optimal ones (an algorithm which gives
  near-optimal solutions is called approximation
  algorithms)
• Since the greedy approach doesnt assure the
  optimality of the solution we have to analyze for
  each particular problem if the algorithm leads to
  an optimal solution

16
Outline

• Optimization problems
• Basic idea of greedy technique
• Examples
• Correctness verification and efficiency analysis
• Some classical applications

17
Correctness verification

• There is not a general method to prove that a
  greedy method yields an optimal solution (in fact
  in most of practical situations it gives only a
  sub-optimal solutions).
• However most problems for which the greedy
  solution is optimal share the following
  properties
• the optimal substructure
• the greedy choice

18
Optimal substructure property

• When can we say that a problem has the optimal
  substructure property ?
• When for an optimal solution S(s1,,sk) of a
  problem of dimension n the set S(2)(s2,,sk) is
  an optimal solution of the subproblem of
  dimension (n-1).
• How can we verify if a problem has this property
  ?
• Using a proof by contradiction

19
Greedy choice property

• When can we say that a problem has the greedy
  choice property ?
• When an optimal solution either is constructed
  by a greedy strategy or can be transformed into
  an optimal solution whose elements are chosen by
  a greedy strategy
• How can we verify if a problem has this property
  ?
• First we prove that by replacing the first
  element of an optimal solution with an element
  selected by the greedy strategy the solution
  remains optimal. Then we proves the same for the
  other element by using the mathematical induction
  or by using the optimal substructure property.

20
Correctness verification

• Example (maximal sum subset of fixed cardinal)
• Let A(a1gta2gtgtan). The greedy solution is
  (a1,,am).
• Let O(o1,,om) be an optimal solution.
• Greedy choice property. Let us suppose that
  o1ltgta1. Then O(a1,o2,,om) has the property
• a1o2omgt o1o2om
• This means that O is better than O. This
  is in contradiction with the fact that O is
  optimal. Thus o1 have to be a1
• b) Optimal substructure property. We shall
  prove by contradiction. Let us suppose that
  (o2,,om) is not an optimal solution for
  A(2)(a2,,an). Let us consider that O(2)
  (o2,,om) is an optimal solution of the
  subproblem. Then O(a1, o2,,om) is a better
  solution than O. Contradiction Thus the problem
  has the optimal substructure property.

21
Correctness verification

• Example coin changing problem
• Let V(v1gtv2gtgtvn1) the values of the coins and
  vidi vi1. The greedy solution, (g1,,gm), is
  characterized by g1 C DIV v1
• Let O(o1,,om) be an optimal solution.
• Greedy choice property. Let us suppose that
  o1ltg1. Then the amount C(C DIV v1-o1)v1 is
  covered by smaller coins. Due to the property of
  coins values by replacing o1 with g1 one obtains
  a better solution (the same value is covered by
  fewer coins of higher value). Thus the problem
  has the greedy choice property.
• b) Optimal substructure property. Easy to
  prove.

22
Efficiency analysis

• Greedy algorithms are efficient
• Usually the dominant operation is that of
  selection of a new element or of the sorting of
  the set A
• Thus the efficiency class of greedy analysis is
• O(n2) or O(nlgn) or even O(n)

23
Outline

• Optimization problems
• Basic idea of greedy technique
• Examples
• Correctness verification and efficiency analysis
• Some classical applications

24
Some classical applications

• The knapsack problem
• Let us consider a set of objects. Each object is
  characterized by its weight (or dimension - d)
  and a value (or profit - p). We want to fill in a
  knapsack of capacity C such that the total value
  of selected objects is maximal.
• Variants
• Continuous variant entire objects or part of
  objects can be selected. The components of the
  solution are from 0,1.
• Discrete variant (0-1) an object either is
  entirely transferred into the knapsack or is not
  transferred.

25
The knapsack problem

• Example
• Value Weight Relative profit
  (value per weight)
• 6 2 3
• 5 1 5
• 12 3 4
• C5
• Selection criteria
• Increasing order of the weight (put as many
  objects as possible) 56122/5114.815.8
• Decreasing order of the value (put the most
  valuable objects) 12618
• Decreasing order of the relative profit (put
  objects which are small and valuable first)
• 51261/5171.218.2

26
The knapsack problem

• Knapsack(d1..n,p1..n)
• sort d and p by the relative profit
• FOR i1,n DO Si0
• i1
• WHILE Cgt0 AND iltn DO
• IF Cgtdi THEN Si1
• CC-di
• ELSE Sidi/C
• C0
• ii1
• RETURN S1..n

27
The knapsack problem

• Correctness verification
• for the continuous variant of the knapsack
  problem the greedy approach leads to an optimal
  solution
• Remarks
• A greedy solution satisfies S(1,1,,1,f,0,..,0)
  
• s1d1sndnC (the equality restriction can be
  always satisfied)
• The objects are decreasingly sorted by the
  relative profit
• p1/d1gtp2/d2gtgtpn/dn
• Proof.
• Let O(o1,o2,,on) an optimal solution. We shall
  prove by contradiction that it is a greedy
  solution. Let us suppose that O is not a greedy
  solution and let us consider a greedy solution
  O(o1,o2,,on)

28
The knapsack problem

• Let B ioigtoi and B_ioiltoi, k
  the smallest index for which oiltoi.
• Due to the structure of a greedy solution it
  follows that any index i from B is less than any
  j from B_.
• On the other hand both solutions satisfy the
  restrictions, thus
• o1d1ondno1d1ondn

This is in contradiction with the hypothesis that
O is optimal. QED
29
Activities selection problem

• Let Aa1,,an a set of activities which share
  the same resource. Each activity ai needs a time
  interval si,fi) to be executed. Two activities
  are considered compatible if their associated
  time intervals are disjoint.
• The problem asks us to select as much as possible
  compatible activities.
• Remarks. There are different criteria of
  selecting the activities
• In increasing order of the ending time (optimal)
• In increasing order of the interval length
• In increasing order of the starting time

30
Activities selection problem

• Activity_selection(a1..n)
• a1..n decreasing_sorting_by_f(a1..n)
• x1a1
• k1
• FOR i2,n DO
• IF ai.sgtxi.f THEN
• kk1
• xkai
• RETURN x1..k

31
Next lecture will be on

• dynamic programming strategy
• and its applications