0-1 Knapsack Problem

1
0-1 Knapsack Problem

• A burglar breaks into a museum and finds n
  items
• Let v_i denote the value of ith item, and let w_i
  denote the weight of the ith item
• The burglar carries a knapsack capable of holding
  a total weight W
• The burglar wishes to carry away the most
  valuable subset of items subject to the weight
  constraint
• Want to steal diamonds before gold
• We assume that the burglar cannot take a fraction
  of an object, so he/she must make a decision to
  take the object entirely or leave it behind
• If the burglar can take a fraction of an object
  for a fraction of the value and weight, the
  problem very easily solved

2
0-1 Knapsack Formal Definition

• Given ltv1, v2, , vngt and ltw1, w2, , wngt, and W
  gt 0, we wish to determine the subset T lt 1, 2,
  .., n (of objects to take) that maximizes
• Sum_i e T vi
• subject to the constraint
• Sum_i e T wi lt W

3
0-1 Knapsack Brute-Force Algo

• Brute-force algorithm would be to try all
  possible combinations of items that fit in the
  knapsack and see which one results in the highest
  total gain
• How many possible sets?
• Size of the superset of n items
• Set of items containing 1 items n
• Set of items containing 2 items (n 2)
• Set of items containing 3 items (n 3)
• Set of items containing n items (n n)
• Sum_k 1n (n k) ? Exponential of
  alternatives

4
0-1 Knapsack Greedy Algorithm

• The greedy thing to do will be to compute the
  gain per unit of weight (vi/wi) for each item
  i, and fill in the knapsack starting with the
  item having the highest gain and continuing this
  way until the knapsack is filled.
• Example next

5
0-1 Knapsack Greedy Algorithm
60
60
35 -- 40
60
60
40
30
40
140
160
90
30
20
20
20
20
100
100
100
10
5
5
30
5
30
-----
-----
-----
90
160
20
100
30
Knapsack
270
220
260
Greedy solution to Fractional problem
6.0 2.0 5.0 3.0 4.0
Gain
Greedy solution to 0-1 problem
Optimal solution to 0-1 problem
6
0-1 Knapsack Greedy Algorithm

• Greedy Algorithm does not work because we cannot
  take fraction of an item. If that was allowed,
  the greedy algorithm would have found the optimum
  solution
• How do we solve 0-1 knapsack then?
• It turns out this problem is NP-complete, so we
  cannot hope to find an efficient algorithm
• However, if we make the same sort of assumptions
  that we made in counting sort, we can come up
  with an efficient algorithm

7
Formulation of the Problem

• We have object 1, 2, 3, , n., Knapsack of
  size W
• Here is how we formulate the problem
• Assume you have a function Knapsack, that that
  computes the optimal Value Knapsack(v, w, n, W).
• Leave the Last Object
• If we choose to not take object n, then the
  optimal value will come about by considering how
  to fill a knapsack of size W with the remaining
  objects 1, 2, , i-1. This is Knapsack(v, w,
  n-1, W)
• Take the Last Object
• If we take object n, then we gain a value of
  vn, but have used up wn of our capacity. With
  remaining W-wn capacity in the knapsack, we can
  fill it up to the best possible way with objects
  1, 2, , i-1. This is vnKnapsack(v, w, n-1,
  W-wn). This is only possible if wn lt W

8
0-1 Knapsack Divide-Conquer Algorithm

• Knapsack(v1..n, w1..n, n, W)
• if (n lt 0) return 0
• if (W lt 0) return 0
• // Consider leaving object n
• leave_val Knapsack(v, w, n-1, W)
• // Consider taking object n
• take_val -INFINITY
• if (wn lt W) take_val vn Knapsack(v, w,
  n-1, W-wn)
• return max(leave_val, take_val)
• //end-Knapsack

• Running Time? O(2n) in the worst case
• Notice that the function makes 2 calls to itself.
  
• At each call, n is decremented by 1. So the
  height of the function call tree is potentially
  n, giving rise to potentially 2n function
  invocations

9
0-1 Knapsack DP Solution

• The problem with the divide and conquer algorithm
  is that it solves the same subproblems over and
  over again.
• This is what DP is all about. To make divide and
  conquer algorithms more efficient
• The idea is to store the solutions to subproblems
  in a table and retrieve the solutions from that
  table the next time we need to solve the same
  subproblem. This way each subproblem is solved
  just once, giving rise to efficient algorithms

10
0-1 Knapsack DP Solution

• How to store solutions to subproblems?
• Construct an array V0..n, 0..W
• For 1 lt i lt n, 0 lt j lt W, the entry Vi, j
  will store the maximum value of any subset of
  objects 1, 2, .., i that can fit into a
  knapsack of weight j
• If we can compute all the entries of this array,
  then the entry Vn, W will contain the max.
  value of all n objects that can fit into the
  entire knapsack of weight W

11
Computing Vi, j

• Observe that V0, j 0 for all 0 lt j lt W
• No items, no value
• We consider 2 cases
• Leave Object
• If we choose to not take object i, then the
  optimal value will come about by considering how
  to fill a knapsack of size j with the remaining
  objects 1, 2, , i-1. This is just Vi-1, j
• Take Object
• If we take object i, then we gain a value of
  vi, but have used up wi of our capacity. With
  remaining j-wi capacity in the knapsack, we can
  fill it up to the best possible way with objects
  1, 2, , i-1. This is viVi-1, j-wi. This is
  only possible if wi lt j

12
0-1 Knapsack Final Formulation

• Combining all observations, we have the following
  rules
• 0
  if i0 and 0ltjltW
• vi,j vi-1,j
  if wi gt j
• maxvi-1,j, vivi-1,j-wi
  if wiltj

13
0-1 Knapsack Algorithm

• Knapsack(v1..n, w1..n, n, W)
• Allocate V0..n0..W
• For j0 to W do V0, j 0 //
  Initialization
• For i1 to n do
• For j0 to W do
• leave_val Vi-1, j // Total val.
  If we leave i
• if (j gt wi) // Enough
  capacity to take i?
• take_val vi Vi-1, j-wi // Tot.
  value if
• else
  // we take i
• take_val -INFINITY // cannot
  take i
• Vi, j maxleave_val, take_val //
  final value
• //end-for
• //end-for
• return Vn, W
• //end-Knapsack

14
0-1 Knapsack Running Time

• It is easy to see that the algorithm takes
  (n1)(W1) O(nW) steps
• The algorithm computes the maximum attainable
  knapsack value in Vn, W, but does not describe
  which items are taken
• But that can be added by recording for each entry
  Vi, j in the matrix whether we got this entry
  by taking the ith item, or leaving it.
• This this information it is possible to
  reconstruct the optimum knapsack contents

15
0-1 Knapsack Example

• Values of the objects are lt10, 40, 30, 50gt
• Weights of the objects are lt5, 4, 6, 3gt
• The capacity of the knapsack, W 10

Capacity?
1
2
3
4
5
6
7
8
9
10
J 0
Item
Value
Weight
0
0
0
0
0
0
0
0
0
0
0
1
10
5
0
0
0
0
10
10
10
10
10
10
0
2
40
4
0
0
0
40
40
40
40
40
50
50
0
3
30
6
0
0
0
40
40
40
40
40
50
70
0
4
50
3
0
0
50
50
50
50
90
90
90
90
0
Final result is V4, 10 90 (for taking items 2
and 4)