Tradeoffs in Backdoors: Inconsistency Detection, Dynamic Simplification, and Preprocessing

1
Tradeoffs in BackdoorsInconsistency Detection,
DynamicSimplification, and Preprocessing

• Bistra Dilkina, Carla Gomes, Ashish Sabharwal
• Cornell University
• ISAIM 2008
• Fort Lauderdale, FL

Note Part of this work was presented at CP-07
2
Context

• Constraint Satisfaction Problems (CSPs)
• In particular, Boolean Satisfiability or SAT
• Given a Boolean formula F in conjunctive normal
  forme.g. F (a or b) and (?a or ?c or d) and
  (b or c)determine whether F is satisfiable
• NP-complete
• widely used in practice, e.g. in hardware
  software verification, design automation, AI
  planning,

3
SAT Gap between theory practice

• From annual SAT competitions
• Large industrial benchmarks (10K vars) are
  solved within seconds by state-of-the-art
  complete SAT solvers
• Good scaling behavior seems to defy
  NP-completeness !
• How can we explain this gap between theory and
  practice?
• Real-world problems have
• tractable sub-structure

Backdoors explain how solvers can get smart
and solve very large instances
4
Backdoors to tractability

• Informally Gomes et al 03
• A backdoor is a critical set of variables for a
  givenproblem such that, once assigned values,
  theremaining instance simplifies to a
  tractable class

How is tractability captured formallyin the
notion of backdoors?
sub-solver capturing poly-time mechanisms(not
necessarily syntactically defined) e.g. Unit
Propagation, Arc Consistency, LP.
Note concept of backdoor applicable to CSPs,
MIP,
5
Tractability sub-solver

• Sub-solver S An algorithm that, given formula F,
  satisfies four properties
• Efficiency S runs in polynomial time(often
  linear or quadratic time)
• Trivial solvability S can determine whether F is
  trivially True (has no clauses) or trivially
  False (has an empty clause, or empty domain)
• Trichotomy reject, declare sat, declare unsat
• Self-reducibility also solves Fv/x

6
Strong Backdoors

• A subset B of variables is a strong backdoor
  fora formula F w.r.t. a sub-solver S if
• for every truth assignment ? to BS solves the
  simplified formula F?/B
• Note have provided powerful insights, leading to
  techniques like randomization, restarts, and
  algorithm portfolios for SAT
• Two key features
• A. Dynamic simplification based on truth values
  and constraint semantics (e.g. unit
  propagation)
• B. Trivial solvability / inconsistency detection

7
Our Results

1. Relaxing these two key features leads to
   exponentially larger backdoors
2. Finding strong backdoors can become harder than
   NP just by adding inconsistency detection to
   tractable class(static notions trivially
   within NP, thus weaker)
3. State-of-the-art solvers do find very small
   strong backdoors (with inconsistency detection)
4. Preprocessing small mixed effect on backdoor
   size
5. Satisfiable instances backdoors smaller w.r.t.
   PL than UP

8
Our Results

1. Relaxing these two key features leads to
   exponentially larger backdoors
2. Finding strong backdoors can become harder than
   NP just by adding inconsistency detection to
   tractable class(static notions trivially
   within NP, thus weaker)
3. State-of-the-art solvers do find very small
   strong backdoors (with inconsistency detection)
4. Preprocessing small mixed effect on backdoor
   size
5. Satisfiable instances backdoors smaller w.r.t.
   PL than UP

9
From algorithmic backdoors to syntactically
defined tractable classes
10
Horn/2CNF backdoors

• Horn formula every clause has at most one
  positive literal
• 2CNF formula every clause is binary
• Strong Backdoors w.r.t. Horn/2CNF
• a subset B of variables such that
• for every value assignment to B,
• the simplified sub-formula is Horn/2CNF (not
  including trivial inconsistency)
• Nishimura, Ragde, Szeider 04 Strong
  backdoors for Horn/2CNF equivalent to deletion
  backdoors

11
Horn/2CNF backdoors

• Horn formula every clause has at most one
  positive literal
• 2CNF formula every clause is binary
• Strong Backdoors w.r.t. Horn/2CNF
• a subset B of variables such that
• for every truth assignment to B,
• the simplified sub-formula is Horn/2CNF (not
  including trivial inconsistency)
• Nishimura, Ragde, Szeider 04 Strong
  backdoors for Horn/2CNF equivalent to deletion
  backdoors
• Aside Dechter 92 Cycle cutset is a deletion
  backdoor w.r.t. the tractable class of acyclic
  CSPs

Deletion
once deleted from F,
Chandru, Hooker 92
12
Deletion backdoors for Horn/2CNF

• Deciding whether a formula has Horn/2CNF deletion
  backdoor of size k is fixed-parameter tractable
  (FPT)
• O(f(k)nc) possibly exponential in k,but
  polynomial in n independent of k
• Why do we care about deletion backdoors?
• In general
• Often easier to reason about and characterize
  (syntactic)
• Deletion backdoors for a class are also strong
  backdoors
• BUT, are deletion backdoors always as small as
  strong backdoors? (for Horn/2CNF yes
  what about richer classes?)

Nishimura, Ragde, Szeider 04
13
Not The Case in General!

• Renamable Horn (RHorn) formulas
• The formula is Horn up to renaming of the
  variables
• Rename(v) flip the sign of all literal
  occurrences of v
• Theorem. Strong backdoors w.r.t. RHorn can be
  exponentially smaller than deletion backdoors
  w.r.t. RHorn
• Proof idea
• given a strong backdoor set B,
• for each assignment ? to B, the simplified
  formula F?/B is RHorn
• However, different ? require different and
  mutually incompatible renamings to convert to Horn

14
Lesson?
Ignoring truth value assignmentsmay not always
be a good idea

• Deletion backdoors ignore the interplay between
  semantics of the constraints and value
  assignments
• Thus, they capture only static tractable
  sub-structure
• Dynamic structure analysis also much more
  powerful in other contexts, e.g.
• dynamic caching for constraint reasoning beyond
  SATBacchus CP-07 invited talk

15
How about inconsistency detection?

• What if the simplified formula is not Horn/2CNF
  butcontains an empty clause?
• Clearly unsatisfiable due to x0
• But smallest Horn backdoor has size n !!
• Missing trivial solvability empty clause
  detection

16
Empty clause detection

• C class of formulas containing an empty
  clause
• Inconsistency detection is a key ingredient of
  backtrack solvers (distinguishes DPLL from naïve
  search)
• size of C backdoors correlated with search
  effort by zChaff Lynce et al 04
• Proposition Strong backdoors w.r.t. C can
  bearbitrarily smaller than strong backdoors
  w.r.t.pure Horn/2CNF

from the example on previous slide
17
Our Results

1. Relaxing these two key features leads to
   exponentially larger backdoors
2. Finding strong backdoors can become harder than
   NP just by adding inconsistency detection to
   tractable class(static notions trivially
   within NP, thus weaker)
3. State-of-the-art solvers do find very small
   strong backdoors (with inconsistency detection)
4. Preprocessing small mixed effect on backdoor
   size
5. Satisfiable instances backdoors smaller w.r.t.
   PL than UP

18
Backdoors w.r.t. Horn/2CNF ? C

• Horn/2CNF ? C ? sub-solver
• Deletion backdoors ? Strong backdoors
• Deciding the existence of strong backdoor of size
  k w.r.t. pure Horn/2CNF is NP-complete Nishimura
  et al 04
• Intuitively speaking, not surprising since
  harder-to-find objects often capture richer
  structure more succinctly

Theorem. Deciding the existence of a strong
backdoor of size k w.r.t. Horn/2CNF ? C is both
NP-hard and coNP-hard.
19
Our Results

1. Relaxing these two key features leads to
   exponentially larger backdoors
2. Finding strong backdoors can become harder than
   NP just by adding inconsistency detection to
   tractable class(static notions trivially
   within NP, thus weaker)
3. State-of-the-art solvers do find very small
   strong backdoors (with inconsistency detection)
4. Preprocessing small mixed effect on backdoor
   size
5. Satisfiable instances backdoors smaller w.r.t.
   PL than UP

20
What matters in practice

• Backdoors characterize hidden structure in
  interesting combinatorial problems
• It is a key notion to understand the behavior of
  state-of-the-art solvers and their sophisticated
  propagation mechanisms
• Backdoors provide powerful insights motivating
  new strategies for algorithm design (e.g.,
  randomization, restarts, algorithm portfolios)

Do problems exhibit small backdoors? Do SAT
solvers exploit them?
21
An Experimental Study of Backdoors
Finer grained insights into backdoors
22
Experimental study

• Syntactic classes Horn, Rhorn
• Algorithmic sub-solversUP (unit propagation),
  PL (pure literal elimination), UPPL,UPPLprobin
  g (branch-free Satz, failed lits)
• Preprocessors2-Simplify, HyPre, SatELite,
  3-Resolution
• Various problem domains
• Graph coloring, logistics planning,car
  configuration, game theory (finding pure Nash eq.)

Kilby et al. 05 Satz-backdoor sizecorrelated
with problem hardness
23
Finding minimum deletion backdoors

• Decision version within NP (unlike strong
  backdoors)
• For Horn (same as strong backdoors) and RHorn
• Use Integer Programming
• E.g for Horn, one 0,1 variable yi for each
  variable i in F
• Minimize size
• Every clause should be Horn

24
Finding small strong backdoors

• Not within NP ? rely on indirect methods for
  finding a strong backdoor
• Gives upper bound on the minimum strong backdoor
  size
• Randomized version of Satz Li and Anbulagan
  97
• a complete backtrack search solver that employs
  UP, PL, probing
• can be easily configured to use selective
  propagation (e.g. only PL)
• Using Satz to find (several) strong backdoors
• Run Satz-Rand multiple times without restarts
• Record the set B of vars not fixed by propagation
  mechanism S
• Can verify B is a strong backdoor w.r.t. S
  for unsat instances

25
Results graph coloring
optimal
upper bounds

• 3 binary variables per graph node
• Planted clique backdoor of size 4x312 w.r.t.
  C

26
Results graph coloring
optimal
upper bounds

• 3 binary variables per graph node
• Planted clique backdoor of size 4x312 w.r.t.
  C
• Propagation sub-solvers find even smaller
  backdoors
• Horn and RHorn backdoors ignore constraint
  semantics
• backdoor size linear in number of variables

27
Results logistics planning

• SATZ backdoors of size 0 !!
• UPPLprobing solves these problems without
  branching
• UP and UPPL also excellent at finding the core
  inconsistencies
• Horn and RHorn backdoors quite large ? 48

28
Results car configuration

• RHorn deletion backdoors are also very small
• The original formulas are almost Horn
• Again SATZ is excellent at finding small backdoor
  sets

29
Results game theory

• Games on random graphs G(n,p) with avg. degree 3
• SATZ backdoors again almost all of size 0
• UP backdoors of size 6 to 8 critical players
• Horn and RHorn backdoors quite large ? 66

30
Our Results

1. Relaxing these two key features leads to
   exponentially larger backdoors
2. Finding strong backdoors can become harder than
   NP just by adding inconsistency detection to
   tractable class(static notions trivially
   within NP, thus weaker)
3. State-of-the-art solvers do find very small
   strong backdoors (with inconsistency detection)
4. Preprocessing small mixed effect on backdoor
   size
5. Satisfiable instances backdoors smaller w.r.t.
   PL than UP

31
Effect of Preprocessors

• How does preprocessing affect backdoor size?
• Refer to paper for details
• Considered 2-Simplify, HyPre, SatELite,
  3-Resolution
• Compared to backdoor size without preprocessing
• Key observations
• Backdoor size in the same order of
  magnitude(e.g. 156-219 vars for UPPL on an
  instance)
• Mixed effect overall often decreases but can
  increase
• SatELite usually gave the smallest backdoors
• Preprocessing often simplifies formula but
  sometimesobfuscates hidden structure

32
Backdoors for Satisfiable Instances

• How do strong backdoors behave for satisfiable
  instances?
• Satisfiable formulas mostly studied w.r.t. weak
  backdoors, although strong backdoors are also
  applicable
• Refer to paper for details
• Considered satisfiable car configuration
  instances
• Also tested with various preprocessing techniques
• Key observations
• PL yields smaller strong backdoors than UP
• Perhaps because pure literal elimination is a
  streamliner
• Preprocessing does not have any significant effect

33
Summary

• The notion of backdoor sets takes into account
  two key features of state-of-the-art constraint
  solvers
• interplay between variable assignments and
  constraint semantics
• inconsistency detection
• These features are critical for small backdoors
• In the worst-case, backdoor detection is not in
  NP(unless PH collapses to NP)
• Despite this, in practice, state-of-the art
  solvers do find surprisingly small backdoors

34
Future Directions

• Study of backdoors in CSPs, MIP,
• Semantics of backdoors
• Backdoors in the context of learning/caching,and
  restarts
• Finer theoretical characterization of the
  complexity of backdoor detection