Brute Force Approaches

1
Brute Force Approaches

• Techniques for finding optimal solutions to hard
  problems relatively quickly
• Backtracking
• Branch and Bound
• Combining efficient solutions with brute force
  approaches
• Other issues

2
The Knapsack Problem

• Input
• Capacity K
• n items with weights wi and values vi
• Goal
• Output a set of items S such that
• the sum of weights of items in S is at most K
• and the sum of values of items in S is maximized

3
Greedy Does Not Work

• What are possible greedy strategies?
• Highest Density First
• Highest Value First
• Lowest Weight First
• Prove these are not optimal for the 0-1 knapsack
  problem.

4
Multi-constraint Knapsack

• Input
• Capacity K1 and K2
• n items with integer weights wi, size si, and
  values vi
• Goal
• Output a set of items S such that
• the sum of weights of items in S is at most K1
• the sum of sizes of items in S is at most K2
• and the sum of values of items in S is maximized
• Can we use dynamic programming?

5
Situation where dynamic programming does not work
What if our problem cant be described with
integers?
wA 2 vA 40 wB ? vB 50 wC 1.98 vC
100 wD 5 vD 95 wE 3 vE 30
We have to resort to brute force.
6
Brute Force

• Generate all possible solutions
• With n items, there are 2n solutions to be
  generated
• Check each to see if they satisfy the constraint
• Save maximum solution that satisfies constraint
• Can be represented as a tree

7
Brute Force Branching
In
Out
Weight 15.12 Value 315
Weight 8.98 Value 235
Weight 9.98 Value 225
8
Backtracking

• In the tree representation, we can think of the
  previous algorithm doing a DFS in the tree
• If we reach a point where a solution no longer is
  feasible, there is no need to continue exploring
• We can backtrack from this point and
  potentially cut off much of the tree and many of
  the solutions
• In the given example, backtracking would be much
  more effective if we had even more items or a
  smaller knapsack capacity

9
Backtracking
2, 40 ?, 50 1.98, 100 5, 95 3, 30
In
Out
B
B
E
E
E
Weight 8.98 Value 235
Weight 9.98 Value 225
10
Branch and Bound

• We can backtrack if we know the best possible
  solution in current subtree is worse than current
  best solution obtained so far.
• Estimates for improvement given current ordering
• A down can give 315
• B down -gt 275
• C down -gt 225
• D down -gt 125
• E down -gt 30

11
Branch and Bound
2, 40 ?, 50 1.98, 100 5, 95 3, 30
In
Out
B
B
C
D
D
E
E
E
E
E
E
Weight 7.12 Value 190
Weight 8.98 Value 235
12
Generating a good bound

• The key to branch and bound is having a good
  initial bound.
• How might we generate a good initial bound?

13
Order of items
2, 40 ?, 50 1.98, 100 5, 95 3, 30

• Since there is no fixed ordering of items, is
  there a better way to order the input items?
• Highest weight first
• Generate infeasible solutions quickly
• Highest density first
• Generate good solution quickly
• Which is better depends on input

14
Heaviest on Top
D 5, 95 B ?, 50 E 3, 30 A 2, 40 C 1.98,
100
In
Out
B
B
(Max remaining 220)
A
C
C
Weight 8.14 Value 145
Weight 10.0 Value 165
Weight 9.98 Value 225
Weight 8.98 Value 235
15
Best Density on Top
C 1.98, 100 A 2, 40 D 5, 95 B ?, 50 E
3, 30
In
Out
A
A
B
B
E
E
E
Weight 8.98 Value 235
16
Greedy and Brute Force

• Suppose half the inputs to our knapsack problem
  all have the same weight.
• If all inputs had the same weight, we could
  implement a greedy solution
• However, since half do not, we cannot use greedy
  alone to find optimal solution
• Combine brute-force approach with greedy to find
  optimal solution more quickly.

17
The Breakdown...
In
Out
F2
F2
F3
F3
F3
F3
F4
F4
F4
F4
F4
F5
F5
F5
F5
F5
F5
F5
F5
F5
F5
Greedy on half!
18
Algorithm

• Backtrack on half the inputs
• At leafs, apply greedy strategy on the other half
  of the inputs
• Comparison
• Pure brute force O(2n)
• Combination O(n2n/2)

19
How much better?

• Assumptions
• Suppose n50
• We can test 1,000,000 solutions/second.
• 250 would take over 35 years
• 225 can be generated in half an hour
• Plus marginal time to generate greedy solution
  for each of the 225 solutions

20
Another combination

• Suppose half the inputs to our knapsack problem
  have small integral weights/values while the
  other half have real weights/values.
• How can we combine approaches to solve this
  efficiently?

21
The n-Queens Problem

• Input
• Positive integer n
• Task
• Place n queens on an n by n chessboard so that no
  two queens attack each other (on same row,
  column, diagonal), or report that this is
  impossible
• Solve the n-queens problem for n 1, 2, 3

22
n4

• Pure brute force search
• 16 squares on a 4 by 4 chessboard
• Leads to 16 15 14 13 43,680 possible
  solutions
• Improvements
• At most one queen per row 44 256 possible
  solutions
• Backtracking If two queens already attack before
  final queen placed, backtrack

23
Larger values of n

• n8
• At most one queen per row 88 16,777,216
• Early backtracking 2057 nodes total
• Time to find first solution 114 nodes
• n12
• At most one queen per row 1212
• Early backtracking 856,189 nodes total
• Time to find first solution 262 nodes

24
The Problem
In the U.S. navy, the SEALS are each specially
trained in a wide variety of skills so that small
teams can handle a multitude of missions. If
there are k different skills needed for a
mission, and n SEAL members that can be assigned
to the team, find the smallest team that will
cover all of the required skills. Andersen knows
hand-to-hand, first aid, and camouflage Butler
knows hand-to-hand and snares Cunningham knows
hand-to-hand Douglas knows hand-to-hand, sniping,
diplomacy, and snares Eckers knows first-aid,
sniping, and diplomacy
25
Greedy Algorithm

• What is the obvious greedy algorithm for this
  problem?
• Find a counter-example to the optimality of
  greedy for this problem.

26
Brute Force Approach

• What is the brute-force approach?
• How can we simplify the problem as far as
  possible in polynomial time?
• How can we set bounds on the solutions?
• When will we need to do backtracking?
• What order should we test the potential members
  in when branching?