AAAI00

1
Generating Satisfiable Problem InstancesDimitris
AchlioptasMicrosoftCarla P. Gomes Cornell
University Henry KautzUniversity of
WashingtonBart SelmanCornell University

• AAAI00
• Austin, Texas

2
Introduction

• An important factor in the development of search
  methods is the availability of good benchmarks.
• Sources for benchmarks
• Real world instances
• hard to find
• too specific
• Random generators
• easier to control (size/hardness)

3
Random Generators of Instances

• Understanding threshhold phenomena lets us tune
  the hardness of problem instances
• At low ratios of constraints -
• most satisfiable, easy to find assignments
• At high ratios of constraints -
• most unsatisfiable easy to show inconsistency
• At the phase transition between these two regions
• roughly half of the instances are satisfiable and
  we find a concentration of computationally hard
  instances.

4
Limitation of Random Generators

• PROBLEM evaluating incomplete local search
  algorithms
• Filtering out Unsat Instances - use a complete
  method and throw away unsat instances.
• Problem want to test on instances too large for
  any complete method!
• Forced Formulas
• Problem the resulting instances are easy have
  many satisfying assignments

5
Outline

• I Generation of only satisfiable instances
• II New phase transition in the space of
  satisfiable instances
• III Connection between hardness of satisfiable
  instances and new phase transition
• IV Conclusions

6
Generation of only satisfiable instances

7
Quasigroup or Latin Squares
Given an N X N matrix, and given N colors, color
the matrix in such a way that -all cells are
colored - each color occurs exactly once in
each row - each color occurs exactly once
in each column
Quasigroup or Latin Square
8
Quasigroup Completion Problem (QCP)
Given a partial assignment of colors (10 colors
in this case), can the partial quasigroup (latin
square) be completed so we obtain a full
quasigroup? Example
32 preassignment
9
QCP A Framework for Studying Search

• NP-Complete.
• Random instances have structure not found in
  random k-SAT
• Closer to real world problems!
• Can control hardness via preassignment
• BUT problem of creating large, guaranteed
  satisfiable instances remains

(Anderson 85, Colbourn 83, 84, Denes Keedwell
94, Fujita et al. 93, Gent et al. 99, Gomes
Selman 97, Gomes et al. 98, Shaw et al. 98, Walsh
99 )
10
Quasigroup with Holes(QWH)

• Given a full quasigroup, punch holes into it

Difficulty how to generate the full quasigroup,
uniformly.
Question does this give challenging instances?
11
Markov Chain Monte Carlo (MCMM)

• We use a Markov chain Monte Carlo method (MCMM)
  whose stationary (egodic) distribution is uniform
  over the space of NxN quasigroups (Jacobson and
  Matthews 96).
• Start with arbitrary Latin Square
• Random walk on a sequence of Squares obtained via
  local modifications

12
Generation of Quasigroup with Holes (QWH)

• Use MCMM to generate solved Latin Square
• Punch holes - i.e., uncolor a fraction of the
  entries
• The resulting instances are guaranteed
  satisfiable
• QWH is NP-Hard
• Is there holes where instances truly hard on
  average?

13
Easy-Hard-Easy Pattern in Backtracking Search
QWH peaks near 32 (QCP peaks near 42)
Computational Cost
holes
14
Easy-Hard-Easy Pattern in Local Search
Computational Cost
holes
First solid statistics for overconstrainted area!
15
Phase Transition in QWH?

• QWH - all instances are satisfiable - does it
  still make sense to talk about a phase
  transition?
• The standard phase transition corresponds to the
  area with 50 SAT/UNSAT instances
• Here all instances SAT
• Does some other property of the wffs show an
  abrupt change around hard region?

16
Backbone
Backbone is the shared structure of all solutions
to a given instance (not counting preassigned
cells)
Number sols 4
17
Phase Transition in the Backbone

• We have observed a transition in the size of
  backbone
• Many holes backbone close to 0
• Fewer holes backbone close to 100
• Abrupt transition coincides with hardest
  instances!

18
New Phase Transition in Backbone
Backbone
of Backbone
Computational cost
holes
19
Why correlation between backbone and problem
hardness?

• Intuitions Local Search
• Near 0 Backbone many solutions easy to find
  by chance
• Near 100 Backbone solutions tightly clustered
  all the constraints vote in same direction
• 50 Backbone solutions in different clusters
  different clauses push search toward different
  clusters

(Current work verify intuitions!)
20
Why correlation between backbone and problem
hardness?

• Intuitions Backtracking search
• Bad assignments to backbone variables near root
  of search tree cause the algorithm to deteriorate
• For the algorithm to have a significant chance of
  making bad choices, a non-negligible fraction of
  variables must appear in the backbone

21
Reparameterization of Backbone
Backbone for different orders (30 - 57)
of Backbone
22
ReparameterizationComputational Cost
Computational Cost different orders (30, 33, 36)
of Backbone
Local Search (normalized)
Local Search (normalized reparameterized)
23
Summary

• QWH is a problem generator for satisfiable
  instances (only)
• Easy to tune hardness
• Exhibits more realistic structure
• Well-suited for the study of incomplete search
  methods (as well as complete)
• Confirmation of easy-hard-easy pattern in
  computational cost for local search
• New kind of phase transition in backbone
• Reparameterization
• GOAL new insights into practical complexity of
  problem solving

24
QWH generator, demos, available soon (lt one
month)www.cs.cornell.edu/gomeswww.cs.washingto
n.edu/home/kautzSATLIBCSPLIB
25
Parameterization
Backbone for different orders (30 - 57)
of Backbone