Title: CSPs: Adding Structure to SAT
1CSPs Adding Structure to SAT
- George Katsirelos Fahiem Bacchus
- University of Toronto
2Introduction
- Finite domain Constraint Satisfaction Problems
(CSPs). - Formally equivalent to SAT
- Important practical differences.
- Different algorithmic techniques have been
developed in the two areas. - Understanding these can help cross fertilize both
fields.
3BackgroundThe SAT and CSP Formalisms
4Formalism
5Formalism
- SAT h V, Ci
- V V1, V2, , Vn is a set of Boolean
variables - C c1, c2, , ck a set of clauses.
- CSP h V, D, C i
- V V1, V2, , Vn is a set of multi-valued
variables - D D1, D2, , Dn is a set of value domains,
with Di being the domain of values for variable
Vi - C C1, C2, , Ck is a set of constraints.
- In both CSP and SAT the aim is to find an
assignment of values for all of the variables - In SAT these values must satisfy the clauses
- In CSPs these values must satisfy the
constraints.
6Constraints
- A constraint C(X1,X2, , Xk) over the variables
X1, , Xk is a Boolean function - It maps assignments to these variables to 0,1
C(X1,X2, , Xk) DX1 ? DXk ? 0,1 - If a tuple of assignments maps to 1, then these
assignments satisfy the constraint, otherwise
these assignments the falsify the constraint.
7Extensionally vs Intensionally Represented
Constraints
- We can specify the constraint with a table
- C(X,Y,Z) with DX DY DZ 1, 2, 3
8Extensionally vs Intensionally Represented
Constraints
- Thus we can represent the constraint as a set of
satisfying assignment tuples
9Extensionally vs Intensionally Represented
Constraints
- Or as a set of falsifying assignment tuples
10Extensionally vs Intensionally Represented
Constraints
- Extensional representations specify the
constraint as an explicit list of satisfying
assignments (or falsifying assignments). - Extensional representations were used in the 2005
CSP solver competition. But are almost never used
in practice. - The extensional representation becomes very
large, growing exponentially with the number of
variables the constraint is over.
11Extensionally vs Intensionally Represented
Constraints
- Constraint could also be represented
intensionally as an algorithm for computing the
Boolean function.
12Extensionally vs Intensionally Represented
Constraints
X Y Z
13Extensionally vs Intensionally Represented
Constraints
- Intensional representations are typical in
practice. - To specify a CSP problem in a CSP solver one
supplies subroutines to implementing the
constraints of the problem. - Commercial CSP solvers supply a large library of
predefined common constraints. - You then simply specify the variables of the CSP,
their domains, and the constraints that are over
them.
14Translating between SAT and CSPs
- Further insight into the relation between SAT and
CSPs is provided by looking at how we can
translate between the formalisms.
15SAT ? CSP
- Translating in this direction is trivial
- Each SAT variable becomes a CSP variable, with
0,1 as its domain of values. - Each clause is equivalent to a Boolean function
from the variables it is over - (x, y, -z) A function mapping (x0,y0,z1) to
0, all other assignments of x,y,z to 1.
16CSP ? SAT
- The other direction requires two steps
- Converting the multi-valued variables into a set
of Boolean assignment variables. - Converting the constraints into clauses over the
assignment variables.
17CSP ? SATConverting the Multi-Valued Variables
- Let X be a CSP variable with Dx d1, , dm
- We create m Boolean assignment variables x1, x2,
, xm these have the the interpretation - xi is true iff Xdi.
-
18CSP ? SAT
- The CSP variable X must have a value and it must
have a unique value. - Hence the Boolean assignment variables x1, x2, ,
xm associated with a particular CSP variable are
mutually exclusive and exhaustive. - This is captured by adding the clauses
- (x1, x2, , xm) X must have a value
- (-xi, -xj) for all (i ? j) X has a unique value
19CSP ? SAT Converting the Constraints into Clauses
- Now we convert the constraints to clauses.
- Each falsifying assignment tuple in the
constraints extensional representation is
equivalent to a clause. - So a constraint becomes a set of clauses, one for
each falsifying assignment.
20CSP ? SAT
- Each falsifying tuple is a set of assignment
variables that cannot be simultaneously true. -
- E.g.. (x1 y2 z1)
- Pushing the negation in we get a clause(-x1 _
-y2 _ -z1)
21CSP ? SAT
- There are various optimizations that can be
applied to this basic translation. - Specific constraints admit more compact encodings.
22Modeling with CSPs
23Modeling with CSPs
- CSPs offer
- Multi valued variables more natural for modeling
real problems. - Constraints over groups of variables that permit
a more natural encoding of the constraints of the
problem. - Industrial applications are much easier to
formalize using CSPs, and the range of
application of CSP technology in industry far
exceeds that of SAT.
24N-Queens
- Place N queens on an NxN chess board so that no
queen can attack any other queen.
25N-Queens
- Place N queens on an NxN chess board so that
queen can attack any other queen.
- N, Queen variables, one for each column
26N-Queens
- Place N queens on an NxN chess board so that
queen can attack any other queen.
- N values for each variable
- The row we place that columns queen on.
27N-Queens
- Constraints
- AllDiff(Q1, , QN) each Queen has a unique value
(cant be in the same row) - Cij(Qi,Qj) Qi Qj ? i-j (for each i ? j)
- cant be on same diagonal
28Modeling with CSPs
- A SAT encoding of N-Queens more complex to
specify. - SAT encodings almost impossible to generate by
hand.
29Modeling with CSPs
- Modeling using the richer language of CSPs,
translate to SAT (automatically), solve using
standard SAT solver. - Understanding the pros and cons of this approach
gives us further insight into the algorithmic
differences between CSP and SAT solvers.
30Solving CSPs
31Backtracking Search
- SAT and CSP backtracking solvers differ in the
three main parts of backtracking - Propagation as we descend the search tree
- Learning as we ascend from failed subtrees
- Heuristics for guiding the branching decisions
32Translation to SAT
- The clause learning in SAT solvers can be
exploited. - The mutually exclusive and exhaustive clauses for
the multi-valued variables are not fully
exploited. - Branching heuristics insensitive to CSP
structure. - Unit propagation weaker than propagation methods
employed in CSP solvers.
33Disadvantages(a) Clauses for Multi-valued
Variables
34Disadvantages(a) Clauses for Multi-valued
Variables
- (x1, x2, , xm) X must have a
value - (-xi, -xj) for all (i ? j) X has a unique
value
- These clauses impose a useful structure on the
assignment variables.
35Disadvantages(a) Clauses for Multi-valued
Variables
- In general, the disjunction of any subset of
positive literals is equivalent to the
conjunction of the complimentary set of negative
literals. E.g., if m4 - x1 _ x2 x3 x4
- x3 -x1 -x2 -x4
36Disadvantages(a) Clauses for Multi-valued
Variables
- This structure could be exploited in various
ways. For example, - Two negative assignment literals ? clause is
redundant - (y1, y2, -x1, -x2, -z3) subsumed by (-x1,-x2)
37Disadvantages(a) Clauses for Multi-valued
Variables
- Negative assignment literal ? remove all positive
literals from same variable. - (y1, y2, -y3, x1, -x2, -z3)
- Resolve with (-y1, -y3) and (-y2, -y3) to obtain
subsuming clause (-y3, -x2, -z3).
38Disadvantages(a) Clauses for Multi-valued
Variables
- Sets of clauses can be replaced by a single
clause. - Dx Dy 1, 2, 3, 4(R, -x1, -y1) (R, -x1,
-y2) (R, -x2, -y1) (R, -x2, -y2) - (R, -x1, -y1) (R, -x1, -y2) (R, -x2, -y1) (R,
-x2, -y2)Equivalent to single clause(R, x3,
x4, y3 , y4).
39Disadvantages(a) Clauses for Multi-valued
Variables
- (R, x3, x4, y3 , y4) (R, (-x2 -x1),
(-y2 -y1)) - Multiply this out and you get 8 clauses.
40Disadvantages(b) Heuristics
41Disadvantages (b) Heuristics
- Under unit propagation contradictions arise when
x is inferred in a context where x is already
true - This causes some clause to be falsified (conflict
clause).
42Disadvantages (b) Heuristics
- With multi-valued variables we always have
- xi -x1 ? xi-1 xi1 ? -xm
- Hence conflicts arise only from refuting all
values from some CSP variables domain - -x1 ? ? -xm
43Disadvantages (b) Heuristics
- In CSP solvers the number of unrefuted values of
a variable is always considered in the branching
heuristic. - In a SAT solver we shouldnt choose to branch on
xi without considering the status of other
associated assignment variables.
Ansótegui1 et al 2003.
44Disadvantages(c) Propagation
45Disadvantages (c) Propagation
- Unit Prop in a SAT solver on the clauses
generated by a constraint is equivalent to
Forward Checking in CSPs. - Forward Checking. Wait until all but one variable
of the constraint is instantiated, and then prune
incompatible values from the domain of the sole
remaining uninstantiated variable.
46Disadvantages (c) Propagation
- (-x1 _ -y2 _ -z1)
- (-x1 _ y3 _ z1)
47Disadvantages (c) Propagation
- Each clause contains one negated assignment
literal from each CSP variable in the constraint. - To make the clause unit one has to make all but
of these assignment variables true - Equivalent to assigning the corresponding CSP
variable - x1 X1, y2 Y2
48Disadvantages (c) Propagation
- Then unit propagation will falsify all
assignments to the remaining unassigned CSP
variable that would violate the constraint - (-x1 _ y3 _ z1), (-x1 _ y3 _ z2)X1 Y3 ?
Z ? 1 Z ? 2
49Disadvantages (c) Propagation
- However, in practice, FC does not perform
particularly well. - A superior form of propagation is GAC.
50GAC (Macworth Freuder 1977-79)
- Given a constraint C(X1,X2, , Xk)
- di 2 DXi is supported (in C) if there exists a
set of assignmentsX1 d1, , Xi di ,, Xk
dk that satisfies CC(X1d1, , Xk dk) 1. - This set is called a support for di.
51GAC (Macworth Freuder 1977-79)
- Supports for X 1 (x1)
- Supports for x3If 3 is removed from the
domain of Z, i.e., -z3 becomes true, x3 will
loose its only support.
52GAC (Macworth Freuder 1977-79)
- The constraint C(X1,X2, , Xk) is said to be GAC
if for all of its variables Xi every value in
their domain is supported (in C). - We can make C(X1,X2, , Xk) GAC by removing all
unsupported values from the domains of its
variables.
53GAC (Macworth Freuder 1977-79)
- GAC propagation is the dynamic process of making
all of the constraints GAC. - If d 2 DX is pruned from the domain of X while
making C1 GAC. - Then the other constraints over X must have GAC
reestablished. - This might prune values of other variables, and
their constraints in turn must be made GAC once
again.
54GAC (Macworth Freuder 1977-79)
- During search we make all constraints GAC at the
root. - The assignment X1 means X ? 2, X ? 3,
- Thus constraints over X have to have GAC
reestablished by GAC propagation. - Reestablishing GAC at every node is called
Maintaining GAC.
55GAC Propagation in the SAT encoding
- By using extra variables in the SAT encoding we
can establish GAC with Unit Propagation. - But Unit Propagation on the standard encoding is
less powerful than GAC.
Bessière et al. 2003
56GAC Propagation in the SAT encoding
- The power of GAC can be characterized using the
notion of prime implicates. - If T is a set of clauses, then the clause c is a
prime implicate if - c is non-tautological
- T ² c
- T ? c for any c that is a subset of c
57GAC Propagation in the SAT encoding
- Let TC be the set of clauses of the constraint C,
along with the mutually exclusive and exhaustive
clauses for each of the Cs variables. - Now we replace TC by the prime implicates of TC,
PIc - Theorem Unit prop over PIc achieves precisely
GAC propagation on the constraint C.
58GAC Propagation in the SAT encoding
- Note that GAC is local to the constraint.
Communication between constraints occurs only
through unit implicants (pruned values). - So GAC is complete local inference for units.
-
59GAC Propagation in the SAT encoding
- Achieving GAC over a generic constraint C(X1,X2,
, Xk) requires time exponential in K. - However, there is however a huge body of
knowledge in the CSP literature on how to achieve
GAC on particular constraints in time polynomial
in K. - These methods (called propagators) exploit the
special structure of the constraint.
60Translation to SAT Summary
- The fundamental problems
- is the size of the encoding
- The vast body of knowledge about propagators for
GAC cannot be exploited. - Exploiting propagators GAC has much in common
with exploiting specialized theories in SMT.
61CSP Solvers
62Using a CSP solver
- Constraints can be represented intensionally and
propagators can be exploited. - The multi-valued variable structure, and
information about the constraints can be
exploited for branching decisions. - Learning in CSP solvers is much weaker than
clause learning in Sat solvers, and it doesnt
integrate well with GAC propagation.
63Learning
- We can improving learning in CSPs and achieve a
better integration with GAC. - We can also integrate GAC propagators with
learning using ideas that are essentially
identical to those used in SMT - These ideas were developed independently.
64Learning
- In CSPs learning from failed subtrees has a long
tradition. Learning is typically called nogood
recording.
65NoGoods
- A NoGood in CSPs is a set of assignments that
cannot be extended to a solution. - (X3, Y2, Z1)
- Translating this to SAT we get -(x3 y2 z1)
(-x3, -y2, -z1) - ? Nogoods are negative clauses
- (clauses containing only negative literals).
66Negative Resolution
- Restricting learning to NoGoods (negative
clauses) restricts the solvers resolution
power to Negative Resolution. - Negative resolution every resolution step
involves a clause negative clause. - Negative resolution not as powerful as general
resolution - CSP solvers sometimes suffer a super-polynomial
slowdown over SAT solvers running on the SAT
encoding.
Mitchell 2003Katsirelos PhD thesis
67Negative Resolution
- As a result of this restriction to learning
negative clauses learning is hardly ever used CSP
solvers in practice. - Learning negative clauses is also produces
particularly ineffective clauses from GAC.
68Integrating SAT style Clause Learning
- GAC prunes domain values. It forces negated
assignment literals. - Like SMT all we need to do is to label those
literals with clauses.
69Clause Learning in CSP solvers
X 1 ? CHOICEX ? 2 ? (X ? 1, X ? 2) (variable
can only have one value)Y ? 1 ? (Y ? 1, X2)
(Non-negative clause reason from GAC)Z ? 1 ? (Z
? 1, X2) A 2 ? CHOICEA ? 1 ? A ? 3 ? X ?
1 ? (X ? 1, A ? 2) (conflict clause
from constraint over X, and A)
70Clause Learning in CSP solvers
- We can resolve backwards from a conflict along
the implication trail from a conflict to learn
various types of new clauses - I.e., we can apply standard SAT clause learning
techniques.
71Computing Clauses from GAC
- With Unit Prop each literal is implied by a
specific clause that became unit, so the clause
for labeling an implied literal is obvious. - With GAC a value is pruned (an assignment
variable is made false) as the result of many
different clauses of the constraint. - In particular, a value is pruned by GAC when it
looses all of its supports. - Each support (which is a tuple of assignments to
the variables of the Constraint) is lost when one
of its assignments is made false.
72Computing Clauses from GAC
- E.g., (X1 Y1 Z1). This support can be
lost if 1 is pruned from the domain of Y or from
Z. (Y?1, or Z?1)
73Computing Clauses from GAC
- If GAC on this constraint prunes X1, a reason
for this pruning is a set of currently true
non-assignments that hits all of X1s supports.
74Computing Clauses from GAC
- E.g.,Y?1, Z?2, Z?3 covers all of X1s supports
in this constraint - Y?1Æ Z?2 Æ Z?3!X?1 (Y1, Z2, Z3, X?1)
75Computing Clauses from GAC
- We put this implication on the trailX?1 ? (X?1,
Y1, Z2, Z3) - Note that
- this is a non-negative clause
- We can compute this clause on the fly from an
extensional representation of the constraint. - There is no need to precompute and store all such
possible pruning clauses.
76Computing clauses from Intensional Constraints
- How do we obtain clausal reasons from GAC
propagators? - We can no longer find a hitting set for the
supports of the value, there is no explicit
representation of these supports.
77Example All Different
- AllDiff(X1, , Xn) is satisfied only by tuples of
assignments to the Xi that are all different,
i.e., - i?j ! Xi?Xj
78All Different
- A way of enforcing GAC on AllDiff in poly-time
was the probably the first propagator developed
in the CSP literature. Regin 1994. - The method utilizes maximum matchings in
bipartite graphs. - Since then dozens of propagators have been
developed.
79The power of propagators
- DPLL must take exponential time on the pigeon
hole problem PHP this problem is hard for
general resolution. - PHP can be encoded as a single AllDiff(P1, ,
Pn) each with domain of values 1, , n-1. - This constraint has no satisfying tuples so every
value will be pruned by GAC. - A CSP solver can solve this problem in polynomial
time. GAC propagation at the root, no search.
80Clausal Reasons from AllDiff
- How do we obtain a clausal reason for a value
pruned by AllDiff? - For Alldiff a value is pruned from a variable
domain only when that value is consumed by some
other variables. That is, the value must be used
by some other variable.
81Clausal Reasons from AllDiff
- DX 1, 2, 3,4, DY 1, 2, 3,4, DZ 1, 2,
3, DW 1, 4, 5
DX 1, 2, 3,4, DY 1, 2, 3,4, DZ 1, 2,
3, DW 1, 4, 5
Prune 4 from domain of X and Y
Initially all values are supported
82Clausal Reasons from AllDiff
DX 1, 2, 3,4, DY 1, 2, 3,4, DZ 1, 2,
3, DW 1, 4, 5
DX 1, 2, 3,4, DY 1, 2, 3,4, DZ 1, 2,
3, DW 1, 4, 5
1, 2, 3 are consumed by X, Y, Z. (Hall interval)
W cannot be assigned 1 since that value is
consumed.
83Clausal Reasons from AllDiff
DX 1, 2, 3,4, DY 1, 2, 3,4, DZ 1, 2,
3, DW 1, 4, 5
- X?4 Æ Y?4 ! W?1
- Clause reason for W?1
- (W?1,X4,Y4)
W cannot be assigned 1 since that value is
consumed.
84Clausal Reasons from All Diff
- If allDiff ? X?d the reason is
- First find the set of variables that still have d
in their domain. - These variables must be consuming d.
- And the reason is the set of pruned values in
their domains.
85Computing clauses from Intensional Constraints
- Katsirelos has found ways to compute clausal
reasons for a variety of known propagators. - He has implemented this in a CSP solver called
EFC, which contains - a number of built in intensional constraints
- GAC propagators for them.
- Clausal reasons supplied by these propagators.
86Computing clauses from Intensional Constraints
- This solver can be thought of as a multi-valued
variable SMT solver, where T includes a set of
constraints known in the CSP literature. - Like SMT solvers it can display very impressive
performance.
87Solving CSP via CSP solvers
- More work still needs to be done
- There are many more propagators that we dont yet
know how to get clausal reasons from. - Heuristics remain poorly understood, and still
need improvement.
88Some Empirical Results
89Logistics (AI Planning)
- From Katsirelos Bacchus, Generalized NoGoods
in CSPs
90Social Golfer
91Other ideas from CSP solvers for SAT/SMT
- CSP solvers use propagation Queues
- Sequences the propagators, as some propagators
are more expensive. - Flexible ways of specifying the level of
propagation for each constraint. - Only forward checking is preformed on some, GAC
on others, we delay GAC until all but 3 variables
of the constraint are instantiated etc.
92Other ideas from CSP solvers for SAT/SMT
- Bounds propagation.
- Order the domain values and instead of pruning
all unsupported values, we maintain upper and
lower bounds on the possible domain values. - Possible to do more efficiently than GAC in many
cases. - This would correspond to generating only a subset
of the implied literals. - Once we get to a solution all literals are set
anyway.
93Other ideas from CSP solvers for SAT/SMT
- Multi-valued variables are very useful. But not
fully exploited in SAT solvers. - Huge body of known constraints and algorithms for
propagating them.
94Conclusions
- CSPs add structure to SAT.
- This structure can be exploited to make modeling
easier. - Can be used to identify groups of clauses over
which - higher levels of propagation can be profitable
- Specialized non-resolution based algorithms can
be exploited. - Connecting these extra kinds of reasoning with
clause learning adds considerable extra power.