Constraint Solving: Problems, Domains and Search Methods

1
Constraint SolvingProblems, Domains and Search
Methods

• Jacques Robin

2
Outline

• What is constraint solving?
• Constraint domains
• Constraint solving inference services
• Practical applications of CSP
• Finite domain Constraint Solving Problem (CSP)
  solving through search
• CSP search algorithms

3
What is Constraint Solving?

• A versatile paradigm for symbolic, numerical and
  hybrid symbolico-numerical automated reasoning
• Relies on hybrid logical-numerical knowledge
  representation formalism
• Relies on AI search, term rewriting, operation
  research and mathematical inference algorithms
• Allows reasoning with incomplete information
• Takes as input intentional and extensional
  knowledge
• When input knowledge is consistent and complete,
  returns as output extensional knowledge
• When input knowledge is consistent but
  incomplete, returns as output intentional and
  extensional knowledge that is more concise and
  easy to understand than the input
• Identifies inconsistent input knowledge
• Most other automated reasoning paradigms
  (monotonic deduction, belief revision, belief
  update, abduction, planning, optimization) can be
  reformulated as one form of constraint solving

4
Constraint Solving Problems (CSP)

• Input
• Set of variables, each one associated with an
  associated domain of possible values (constants)
• Set of functions defining mapping between these
  domains
• Set of relations (called primitive constraints),
  including equations and inequations, over these
  domains
• A logical conjunction of such relations (called
  a compound constraints)
• Output
• Composed of same elements as input
• If the input is just rightly constrained, the
  output is one complete consistent variable
  valuation, i.e., a logical conjunction of
  equations of the form ltvariablegt ltconstant
  valuegt.
• If the input is underconstrained, the output is
  a simplification of the input, i.e., a logically
  equivalent conjunction of primitive constraints
  containing fewer constraints and/or functions
  and/or variables.
• If the input is overconstrained, the output is
  fail for there exists no variable valuation
  that simultaneously satisfy all constraints.

5
CSP Example Analog Circuit Modeling

• Particular circuit instance data sets
• PD1 ( V 10 ? R1 10 ? R2 5 ),
  extensional
• PD2 ( V 10 ? R2 10 ? I1 5 ),
  extensional
• PD3 ( R1 10 ? R2 5 ), extensional
• PD4 ( V 10 ? R1 5 ? I 1 ? 0 ? R2
  ), intensional
• Solving particular circuit instance
  model PM1 GM ? PD1 yields extensional
  consistent solution
• V 10 ? V1 10 ? V2 10 ? R1 10 ? R2
  5 ?
• I 6 ? I1 5 ? I2 1
• Solving particular circuit instance model PM2
  GM ? PD2yields extensional consistent solution
• V 10 ? V1 10 ? V2 10 ? R1 2 ? R2 10 ?
• I 6 ? I1 5 ? I2 1
• Solving particular circuit instance model PM2
  GM ? PD2yields intentional consistent
  solutionV x 3 I x 10
• Solving particular circuit instance model PM4
  GM ? PD4
• yields fail (inconsistent input)

6
CSP Example Building Scheduling

• Particular query Q1
• TS 0 ? Tm min(TE)
• Solution to particular problem GM ? Q1
• TS 0 ? TA 7 ?
• TB 11 ? TC 10 ?
• TD 12 ? TE 15 ?
• Tm 15
• Particular query Q2
• TE ? 14
• Solution to particular problem GM ? Q2 fail

7
CSP Example Map Coloring

• Generic Australia map coloring model AMCM
• WT ? SO ? WT ? NT ?NT ? SO ? NT ? Q ?
• Q ? SO ? Q ? NSW ?
• NSW ? SO ? NSW ? V ? V ? SO
• Color set instance BGR
• WT ? blue, green, red ? SO ? blue, green, red
  ?
• NT ? blue, green, red ? Q ? blue, green, red
  ?
• NSW ? blue, green, red ? V ? blue, green, red
  ?
• T ? blue, green, red
• Solving specific Australian map coloring
  problemAMCM ? BGR yields complete consistent
  solutionSO blue ? WA red ? NT green ? Q
  red ?NSW green ? V red ? T green
• Solving specific Australian map coloring problem
  with any set instance with two-color domains for
  all variables yields fail

8
The Language of CSP MOF Metamodel
FOL Formula
9
The Language of CSP MOF Metamodel
ltltenumgtgt FOL Connective ? ? ? ? ?
10
CSP Domains MOF Metamodel
Constraint Domain (CD)
, ?, true, false
?, ?
11
CSP Solving Services

• Substitution
• Satisfaction
• Absolute Implication
• Absolute Equivalence
• Normalization
• Absolute Simplification
• Projection
• Relative Implication
• Relative Equivalence
• Relative Simplification
• Local Propagation
• Optimization
• Labeling

12
CSP Solving Services Substitution

• Substitute(?Valuation, CCompoundConstraint)Comp
  oundConstraint
• returns result of substituting in C the
  variables in ? by their value in ?.
• Examples
• C B P I x P ? B2 B I x B
• ?1 B 1200 ? P 1000 ? I 20/100 ? B2
  1440
• ?1(C) 1200 1000 20/100 x 1000 ? 1440
  1200 20/100 x 1200
• ?2 B 1 ? I 1
• ?2(C) 1 P 1 x P ? B2 1 1 x 1
• ?3 B 1 ? P 0 ? I 1 ? B2 1
• ?3(C) 1 0 1 x 0 ? 1 1 1 x 1

13
CSP Solving Services Satisfaction

• Satisfiable(CCompoundConstraint)Boolean
• result true iff ??Valuation Substitute(?,
  C) holds
• if result true, also returns ?.
• Examples
• C1 B P I x P ? B2 B I x B
• Satisfiable(C1) true,
• since ??1 (B 1200 ? P 1000 ? I
  20/100 ? B2 1440)
• ?1(C1) (1200 1000 20/100 x 1000 ? 1440
  1200 20/100 x 1200) ? (1200 1000 20000/100
  ? 1440 1200 24000/100) ? (1200 1000
  200 ? 1440 1200 240) ? (1200 1200 ?
  1440 1440) ? (true ? true) ? true
• C2 X Y 1 ? Y X 1
• Satisfiable(C2) false, since
• C2 ? (X X 1 1 ? Y X 1) ? (X X 2
  ? Y X 1) ? (0 2 ? Y X 1) ?
  (false ? Y X 1) ? false

14
CSP Solving Services Absolute Implication and
Equivalence

• Implies(C1CompoundConstraint, C2CompoundConstrai
  nt)Boolean
• result true iff ??Valuation, ?(C1)
  satisfiable ? ?(C2) satisfiable
• Examples
• C1 (TS ? 0 ? TA ? TS 7 ? TB ? TA 4 ?
  TC ? TA 3 ? TD ? TC 2 ? TE ? TB 2
  ? TE ? TC 3 ? TE ? TD 3)
• C2 TB ? TC
• Implies(C1,C2) false
• Since ? (TS 0 ? TA 7 ? TB 11 ? TC
  12 ? TD 14 ? TE 17) satisfies C1 but
  not C2
• C3 C1 ? TE 15
• Implies(C3,C2) true
• Since C3 ? 12 ? TD ? 10 ? TC ? TA ? 7 ?
  TB ? 11
• Equivalent(C1CompoundConstraint,
  C2CompoundConstraint) Boolean
• result true iff ??Valuation, ?(C1)
  satisfiable ? ?(C2) satisfiable
• C1 ? C2 iff (C1 ? C2) and (C2 ? C1)

15
CSP Solving Services Normalization

• Solved form compound constraint
• X1 e1 ? ... ? XN eN such that
• none of the variables X1 ... XN occur in any of
  the expressions e1 ... eN.
• Normalize(CCompoundConstraint)CompoundConstraint
  
• result S is in solved form and verifies S ? C
  ? C
• Examples
• C (X 2 Y ? 2Y X T Z ? X Y 4
  ? Z T 5)
• S Normalize(C) (X 3 ? Y 1 ? Z 5
  T)
• C ? (X 2 Y ? 2Y 2 Y T Z ? 2 Y
  Y 4 ? Z 5 - T)
• ? (X 2 Y ? 3Y 2 T 5 - T ?
  2Y 4 - 2 ? Z 5 - T)
• ? (X 2 Y ? 3Y 2 5 ? Y 1 ?
  Z 5 - T)
• ? (X 2 1 ? 31 2 5 ? Y 1 ?
  Z 5 - T)
• ? (X 3 ? 5 5 ? Y 1 ? Z 5 - T)
• ? (X 3 ? true ? Y 1 ? Z 5 - T)
• ? (X 3 ? Y 1 ? Z 5 - T) S

16
CSP Solving ServicesAbsolute Simplification

• Simplify(CCompoundConstraint)CompoundConstraint
  
• result S is equivalent, simpler constraint,
  i.e.,
• S ? C and
• S has fewer primitive constraints than C and/or
• S has more constraints in solved form than C
  and/or
• S has fewer function symbols than C and/or
• S has fewer variables than C
• Examples
• C (X ? Y Z ? U V ? X V ? U Z Y
  ? V V 0 ?
• U,V,X,Y,Z ? N)
• S (X Y Z ? U Z Y ? V 0 ?
  U,V,X,Y,Z ? N)
• Since
• C ? (X ? Y Z ? U V ? X V ? U Z Y
  ? V 0 ? U,V,X,Y,Z ? N)
• ? (X ? Y Z ? 0 U ? X 0 ? U Z
  Y ? U,V,X,Y,Z ? N ? V 0)
• ? (X ? Y Z ? U ? X ? U Z Y ?
  U,V,X,Y,Z ? N ? V 0)
• ? (X ? Y Z ? Z Y ? X ? U Z Y
  ? V 0 ? U,X,Y,Z ? N)
• ? S

17
CSP Solving Services Projection

• Valuation extension
• Given a valuation ?B of the form (X1 v1 ?...?
  XB vB)
• Any valuation ?E of the form (X1 v1 ?...? XB
  vB ? XB1 vB1 ?...? XE vE )is an extension
  of ?B.
• Partial solution
• A valuation ?P is a partial solution of a
  constraint C iff??F Valuation, ?F extends ?P
  and ?P is a solution of C
• Notation vars(C) set of variables occurring in
  constraint C
• Project(CCompoundConstraint,VsVariableSet)Compo
  undConstraint
• precondition Vs ? vars(C)
• result P verifies
• vars(P) ? Vs
• C ? P
• ??P Valuation over Vs, ?P solution of P ? ?P
  partial solution of C

18
Projection Examples

• C1 (X ? Y ? Y ? Z ? Z ? T ? T ? 0)
• Project(C1,X) X ? 0
• C2 (f(Y,Y) f(X,Z) ? s(Z) s(T) ? f
  bijection)
• Project(C2,X,Z) (X Z)
• C3 (X Y ? 1 ? X - Y ? 1 ? - X Y ? 1 ?
  - X - Y ? 1)
• Project(C3,X) (- 1 ? X ? X ? 1)
• Counter-example C4 (X f(Y,Z))
• Project(C4,X) fail
• there is no primitive constraint in C1that
  either do not contain X or can be simplified

19
CSP Solving Services Local Propagation

• Determined solved form of compound constraint
• X1 v1 ? ... ? XN vN where X1 ... XN are
  variables and v1 ... vN are constants
• Propagate(CdCompoundConstraint,
  CCompoundConstraint)CompoundConstraint
• preconditions
• Cd sub-conjunction of C
• Cd in determined solved form
• result Propagate(Cd(C),choose(Cd,determines(Cd
  ,C))), i.e.,
• apply Cd as valuation substitution on C
• find which other sub-conjunctions of C become
  determined by this substitution --
  determines(Cd,C)
• choose one of them Cd
• recursively propagate Cd on Cd(C)
• stop when propagation fails to determine new
  member of C

20
Local Propagation Example

• C (V V1 ? V V2 ? V1 I1 x R1 ? V2 I2 x
  R2 ? I I1 I2)
• Cd (V 10 ? R1 5 ? R2 10)
• Propagate(Cd,C) (V 10 ? V1 10 ? V2 10 ?
  I1 2 ? I2 1 ? I 3)
• Since
• Cd(C) (10 V1 ? 10 V2 ? V1 I1 x 5 ? V2
  I2 x 10 ? I I1 I2)
• Cd (V1 10 ? V2 10)
• Cd(Cd(C)) (10 10 ? 10 10 ? 10 I1 x 5 ?
  10 I2 x 10 ? I I1 I2)
• Cd (I1 2 ? I2 1)
• Cd(Cd(Cd(C))) (10 V1 ? 10 V2 ? 10 2 x
  5 ? 10 1 x 10 ? I 2 1)

21
CSP Solving Services Optimization

• Optimize(CCompoundConstraint, FCostFunction)Val
  uation
• if C overconstrained result fail
• if C just rightly constrained result unique ?u
  such that ?u(C) satisfiable
• if C underconstrained result one of the
  lowest-cost solutions, i.e., ?o such that ?o(C)
  satisfiable and ?? such that
  ?(C) satisfiable, F(?o) ? F(?)
• if there is no such lower-cost solution, result
  none
• Examples
• C1 (X Y ? 4)
• F(X,Y) X2 Y2
• Optimize(C1,F) (X 2 ? Y 3)
• G(X,Y) X Y
• Optimize(C1,F) any solution to C2 (X Y
  4)
• C3 X ? 0
• H(X) X
• Optimize(C3,H) none

22
CSP Solving Services Labeling

• Label(CCompoundConstraint)ValuationSet
• precondition C over finite domain
• result ?Valuation ?(C) satisfiable
• Example
• C1 (WT ? SO ? WT ? NT ? NT ? SO ? NT ? Q
  ? Q ? SO ? Q ? NSW ? NSW ? SO ?
  NSW ? V ? V ? SO)
• C2 (WT ? blue, green, red ? SO ? blue,
  green, red ? NT ? blue, green, red
  ? Q ? blue, green, red ? NSW ? blue,
  green, red ? V ? blue, green, red ?
  T ? blue, green, red)
• Label(C1 ? C2) (SO blue ? WA red ? NT
  green ? Q red
  ? NSW green ? V red ? T green)

23
Constraint Solvers

• Constraint solver software providing one CSP
  service
• Many CSP services can be implemented through
  judicious assembly and reuse of other CSP
  services
• Properties
• Correct
• Complete
• Normalizing
• Set-based
• Variable name independent
• Monotonic (falsity preserving)
• Projecting
• Weakly projecting

24
Constraint Solvers Properties

• Correct guaranteed to return only correct
  solutions
• Complete guaranteed to return all existing
  solutions
• Possible only for small instances of CSP over
  specific domains
• Most CSP are NP-Hard, some are semi-decidable or
  even undecidable
• Satisfiable service returns unknown when it
  can neither conclude that the input constraint is
  satisfiable nor that it is unsatisfiable
• Normalizing return results in solved form
• Directly legible, usable result
• No need for simplification or projection
  post-processing
• Set-based returns same solution for two
  equivalent compound constraints differing only in
  primitive constraint order and/or repetitions
• Variable-name independent returns same solution
  for two equivalent compound constraints different
  only in terms of variable names
• Monotonic ?C1,C2 satisfies(C1) false ?
  satisfies(C1 ? C2) false

25
Soft Constraints

• Define preference order over valuations
  consistent with hard constraints
• Hard constraint example a professor cannot teach
  two courses with overlapping time slots
• Soft constraint example a professor prefers its
  undergraduate and graduate course to be scheduled
  on the same day
• Most satisfiability problems with hard and soft
  constraints can be transformed into optimization
  problems
• The preference defined by the soft constraints
  is captured by the cost function to optimize

26
Primitive Constraint Arity

• Arity primitive constraint argument number
• Zero-ary true, false
• Unary boolean negation, , ?, ?, ?, ?, ? with
  one variable and one constant
• Binary , ?, ?, ?, ?, ? with two variables
• Primitive high-order FD constraints
• Alldiff(V1 ?D, ... , Vn ?D), no pair of
  variables from V1, ... , Vn can share the same
  value in finite domain D
• Atmost(T?D,V1 ?D, ... , Vn ?D), T ? V1 , ... ,
  Vn
• Element(I ?1, ... ,n, V1, ... , Vn, X), if I
  i, then X Vi
• Any primitive high-order FD constraints can be
  converted to an equivalent conjunction of binary
  primitive FD constraints by the introduction of
  additional, auxiliary variables
• But, special-purpose propagation techniques
  handle primitive high-order FD constraints far
  more efficiently than general purpose propagation
  techniques can handle their conversion as a
  conjunction of binary constraints

27
CSP Domainsand Algorithms

• Chronological Backtracking (CBT)
• Simple, w/ Forward Checking
• Conflict-Directed Backjunping (CDBJ)
• Simple, w/ Forward Checking
• k-Consistency Propagation
• CBT w/ k-Consistency Propagation
• CDBJ w/ k-Consistency Propagation
• Min-Conflict

Constraint Domain (CD)
Local Propagation
Symbolic CD
Numeric CD
Finite CD
Infinite CD
Real Equations Inequalities
Integer FD Linear Equations Inequalities
Symbolic FD

• Bounds Consistency Propagation (BCP)
• CBT w/ BCP
• CDBJ w/ BCP

Real Polynomial Equations Inequalities
Gauss-Jordan Elimination
Simplex Optimization
Interval FD
Real Linear Equations
Real Linear Equations Inequalities
Ordinal FD
String CD
Rational Trees CD
Real Linear Inequalities
Nominal FD
Fourier Elimination
Integer Linear Equations Inequalities
Infinite Symbolic CD
Boolean CD
Unification
28
FD Constraint Satisfaction as Search

• FD CSP D v1, ... , vm ? X1 ? D ? ... ? Xn ?
  D ? Cswhere Cs c1(Xi1,Xj1) ? ... ? cp(Xip,Xjp)
• Finite domain allows solving by enumeration of
  possible valuations (search)
• Formulation as incremental search
• Initial state no variable assigned, i.e., X1 ?
  D ? ... ? Xn ? D ? Cs
• Successor function add Xi vj to current
  valuation Xk vl ? ... ? Xq vr such that Xk
  vl ? ... ? Xq vr ? Xi vj ? Cs satisfiable
• Goal test X1 vi1 ? ... ? Xn vin ? Cs
  satisfiable
• Path cost 1 per step
• Formulation as complete-state search
• Initial state all variables randomly assigned,
  i.e., X1 vi1 ? ... ? Xk
  vik ? ... ? Xn vin
• Successor function change assignment of one
  variable in current valuation resulting in X1
  vi1 ? ... ? Xk vil ? X1 vi1 ? ... ? Xn
  vinwhere vil ? vik.
• Goal test same as incremental search

29
General Problem-Solving Searchvs. FD CSP Search

• General problem-solving search
• State representation
• Problem-specific
• Black-box data structure
• Successor function
• Problem-specific
• Arbitrary black-box
• Goal test function
• Problem-specific
• Arbitrary black-box
• Heuristic functions
• All domain-specific

• FD CSP search
• State representation
• Standard for all CSP
• Compositional logical knowledge representation
  language
• Successor function
• Standard for all CSP
• Instantiation of one piece of intentional
  knowledge into extensional knowledge
• Goal test function
• Standard for all CSP
• Satisfaction of all instantiated primitive
  constraints
• Heuristic functions
• Standard for all CSP
• Some domain-independent!

30
FD CSP Graphs

• Summarize dependencies between variables through
  constraints
• Node variable
• Arc constraint
• Useful for
• Computing FD CSP search heuristic functions
• Complexity analysis of FD CSP search algorithm

31
Backtracking Search for FD CSP

• General algorithm
• At each step, choose one variable to assign one
  value to it and choose that value from the
  variables associated domain
• If the resulting partial valuation satisfies all
  the primitive constraints, recur to choose next
  variable-value assignment pair
• Otherwise, backtrack to a earlier assignment
  pair choice, choose an alternative pair and
  resume forward search from that point
• Notes
• Path to solution state irrelevant
• Base, uninformed version
• Chooses variable to assign randomly among
  remaining options
• Chooses value to assign randomly among remaining
  options
• Always backtracks to last choice point
  (chronological backtracking)
• Does not perform any pruning of future options
  based on the propagation of the consequences of
  its last assignment Xi vi to the domains of
  variables adjacent to Xi in the constraint graph

32
Chronological Backtracking Example

• (X1r ? X1b ? X1g) ? (X2b ? X2g) ? (X3r
  ? X3b) ? (X4r ? X4b) ? (X5b ? X5g) ?
  (X6r ? X6g ? X6t) ? (X7r ? X7b) ?
  ?(X1X2) ? ?(X1X3) ? ?(X1X4) ? ?(X1X7) ?
  ?(X2X6) ? ?(X3X7) ? ?(X4X5) ? ?(X4X7) ?
  ?(X5X6) ? ?(X5X7)

33
Chronological Backtracking Example
X1
X7
X2
X6
X3
X4
X5
34
Chronological Backtracking Example
X1
X7
X2
X6
X3
X4
X5
35
Chronological Backtracking Example
X1
X7
X2
X6
X3
X4
X5
36
Chronological Backtracking Example
X1
X7
X2
X6
X3
X4
X5
37
Chronological Backtracking Example
X1
X7
X2
X6
X3
X4
X5
38
Chronological Backtracking Example
X1
X7
X2
X6
X3
X4
X5
X1
X7
X2
X6
X3
X4
X5
39
Chronological Backtracking Example
X1
X7
X2
X6
X3
X4
X5
X1
X7
X1
X7
bt
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
40
Chronological Backtracking Example
X1
X7
X2
X6
X3
X4
X5
X1
X7
X1
X7
X1
X7
bt
x
X2
X6
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
X3
X4
X5
41
Chronological Backtracking Example
X1
X7
bt
x
X2
X6
bt
X3
X4
X5
42
Chronological Backtracking Example
X1
X7
X1
X7
bt
x
X2
X6
X2
X6
bt
X3
X4
X5
X3
X4
X5
bt
x
43
Chronological Backtracking Example
X1
X7
X1
X7
bt
X2
X6
X2
X6
bt
X3
X4
X5
X3
X4
X5
bt
X1
X7
X2
X6
X3
X4
X5
x
bt
44
Chronological Backtracking Example
X1
X7
X1
X7
bt
X2
X6
X2
X6
bt
X3
X4
X5
X3
X4
X5
bt
X1
X7
X1
X7
X2
X6
X2
X6
bt
X3
X4
X5
X3
X4
X5
x
bt
45
Chronological Backtracking Example
X1
X7
X1
X7
bt
X2
X6
X2
X6
bt
X3
X4
X5
X3
X4
X5
bt
X1
X7
X1
X7
X1
X7
x
X2
X6
X2
X6
X2
X6
bt
X3
X4
X5
X3
X4
X5
X3
X4
X5
bt
46
Chronological Backtracking Example
X1
X7
X1
X7
bt
X2
X6
X2
X6
bt
X3
X4
X5
X3
X4
X5
bt
X1
X7
X1
X7
X1
X7
x
X2
X6
X2
X6
X2
X6
bt
X3
X4
X5
X3
X4
X5
X3
X4
X5
bt
X1
X7
X2
X6
X3
X4
X5
47
Chronological Backtracking Example
X1
X7
X1
X7
bt
X2
X6
X2
X6
bt
X3
X4
X5
X3
X4
X5
bt
X1
X7
X1
X7
X1
X7
x
X2
X6
X2
X6
X2
X6
bt
X3
X4
X5
X3
X4
X5
X3
X4
X5
bt
X1
X7
X1
X7
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
48
Chronological Backtracking Example
X1
X7
X1
X7
bt
X2
X6
X2
X6
bt
X3
X4
X5
X3
X4
X5
bt
X1
X7
X1
X7
X1
X7
x
X2
X6
X2
X6
X2
X6
bt
X3
X4
X5
X3
X4
X5
X3
X4
X5
bt
X1
X7
X1
X7
X1
X7
X2
X6
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
X3
X4
X5
49
Chronological Backtracking Example
X1
X7
X2
X6
X3
X4
X5
50
Chronological Backtracking Example
X1
X7
X1
X7
bt
x
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
51
Chronological Backtracking Example
X1
X7
X1
X7
...
bt
x
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
x
X1
X7
X2
X6
X3
X4
X5
52
Chronological Backtracking Example
X1
X7
X1
X7
...
bt
x
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
x
X1
X7
X1
X7
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
53
Chronological Backtracking Example
X1
X7
X1
X7
...
bt
x
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
x
X1
X7
X1
X7
...
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
X1
X7
bt
X2
X6
X3
X4
X5
54
Chronological Backtracking Example
X1
X7
X1
X7
...
bt
x
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
x
X1
X7
X1
X7
...
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
x
x
...
X1
X7
X1
X7
bt
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
55
Chronological Backtracking Example
X1
X7
X1
X7
...
X2
X6
X2
X6
X3
X4
X5
X3
X4
X5
56
Heuristic Improvements of Backtracking Search for
FD CSP

• How to choose next variable to assign at each
  forward step?
• How to choose next value to assign to that
  variable?
• Where to backtrack when current valuation fails?
• What to record when current valuation fails to
  avoid repeating following such unsuccessful path
  in the future?
• Domain-independent, general-purpose heuristics
  for each decision
• Whether or not to combine it with
  consistency-based constraint propagation to prune
  the domains of the values
• Such propagation can be done either as a
  pre-processing step
• Or after each variable assignment

57
FD CSP BT SearchForward Phase Heuristics

• Next variable choice
• Most constrained by current partial valuation
• a.k.a., MRV (Minimum Remaining Values) heuristic
• Why? Speed-ups failure detection, avoids BT in
  hopeless search space regions
• Most constrained with currently unassigned
  variables
• a.k.a., Highest Degree (HD) heuristic
• Why? Reduces future effective branching factor
• Common combination HD as tie breaker for MRV
• Next value choice
• Least Constraining Value (LCV)
• Why? Leaves options open, avoids BT triggered by
  early commitment with insufficient knowledge
• Instance of general AI heuristic of
  least-commitment

58
FD CSP BT SearchForward Phase Heuristics Examples

• Next variable choice most constrained by
  current partial valuation
• Next variable choice most constrained with
  currently unassigned variables
• Next value choice least constraining value
• Improves scalability from25-Queens for
  uninformed BTto 1000-Queens

59
FD CSP BT Searchwith Forward Checking

• In forward phase
• After each new variable assignment Xi vi
• Add step to delete vi from the domains of
  adjacent(Xi) in the constraint graph

60
FD CSP BT Searchwith Forward Checking

• In forward phase
• After each new variable assignment Xi vi
• Add step to delete vi from the domains of
  adjacent(Xi) in the constraint graph

61
FD CSP BT Searchwith Forward Checking

• In forward phase
• After each new variable assignment Xi vi
• Add step to delete vi from the domains of
  adjacent(Xi) in the constraint graph

62
FD CSP BT Searchwith Forward Checking

• In forward phase
• After each new variable assignment Xi vi
• Add step to delete vi from the domains of
  adjacent(Xi) in the constraint graph

63
FD CSP BT Searchwith Forward Checking

• Forward checking does not provide early detection
  for all failures
• After 3 steps, domains of adjacent variablesNT
  and SA are both reduced to blue which leads to
  failure
• Systematic early detection requires multi-step
  constraint propagation after each assignment

64
k-Consistency

• CSP P1 (D v1, ... , vm ? X1 ? D ? ... ? Xn
  ? D ? Cs) where Cs is a compound constraint on X1
  ... Xn
• CSP P2 is a sub-problem of P1 iff it is of the
  form
• D v1, ... , vm ? X1 ? D ? ... ? Xn-1 ? D ?
  Cs
• k-Consistency
• An FD CSP is k-consistent iff any consistent
  partial valuation involving k-1 variables can be
  extended into a consistent valuation assigning
  anyone of the remaining unassigned variables
• 1-Consistency a.k.a. Node Consistency
• Every variable has a consistent assignment in
  any non-empty sub-domain of D
• 2-Consistency a.k.a. Arc Consistency
• Every consistent single variable assignment can
  be extended into a consistent variable assignment
  pair for any other variable
• 3-Consistency a.ka. Path Consistency
• Every consistent variable assignment pair can be
  extended into a consistent variable assignment
  triple for any third variable
• Strong k-Consistency
• An FD CSP is strongly k-consistent iff it is
  k-consistent, k-1 consistent, ...
  path-consistent, arc-consistent and
  node-consistent

65
k-Consistency Examples

• CSP1 X ? Y ? Y ? Z ? Z ? 2 ? X ? D ? Y ?
  D ? Z ? D ? D 1,2,3,4 is not
  node consistent
• Primitive constraint Z ? 2 rules out any
  consistent assignment for Z over sub-domain 3,4
• CSP2 X ? Y ? Y ? Z ? Z ? 2 ? X ? D ? Y ?
  D ? Z ? D ? D 1,2 is node
  consistent
• CSP2 is not arc-consistent
• Y 1 ? X ? Y ? X ? 1,2 is unsatisfiable
• Australia map coloring problem with two colors is
  not globally satisfiable but still arc-consistent

66
Node and Arc-Consistency Propagation

• Node consistency
• For each unary constraints U(X)
• Delete all the values from Domain(X) that
  violate U
• Arc-consistency
• For each binary constraint B(X,Y)
• Delete all the values from Domain(X) and
  Domain(Y) that violates the arc-consistency of
  the CSP

67
Node and Arc Consistency Example
Colouring Australia with constraints
Node consistency
68
Node and Arc Consistency Example
Colouring Australia with constraints
Arc consistency
69
Node and Arc Consistency Example
Colouring Australia with constraints
Arc consistency
70
Node and Arc Consistency Example
Colouring Australia with constraints
Arc consistency
71
Backjumping

• Backjumping or dependency-directed backtracking
  algorithms
• improve on chronological backtracking
• by attempting to backtrack directly to the deep
  cause of the failure up the proof tree.
• Cache and update additional constraint dependency
  information
• derived from constraint graph and current
  partial valuation
• during both the forward and backward phases of
  the search.

72
Conflict Sets

• Given constraint graph G,
• a partial valuation A is a conflict set for
  variable X
• if ?V?Dom(X), ?C?G, A ? XV ? C false,
• i.e., all domain values of X are ruled out by
  some constraint involving A
• e.g., valuation X1r ? X2b ? X3b ? X4b ? X5g
  ? X6r
• is a conflict set for X7 since
• X7r conflicts with X1r ? ?X1X7
• and X7b conflicts with X3b ? ?X3X7 and with
  X4b ? ?X4X7
• A conflict set is minimal if it does not contain
  any conflict subset
• e.g., valuations X1r ? X3b and X1r ? X4b
• are minimal conflict sets for X7
• X1r ? X3b is the Earliest Minimal Conflict Set
  (EMCS) for X7 with variable ordering X1, , X7

73
Dead-end Variablesand Jump-back Sets

• A dead-end valuation is partial valuation
• X1r ? ? Xib that is a conflict set
• for the dead-end variable Xi1
• The jump-back set of a dead-end variable is its
  EMCS with the current variable ordering
• When failure occurs at a dead-end variable,
• Conflict-Directed Back-Jumping (CDBJ) backtracks
  directly (or jumps back) to the latest variable
  of its jump-back set with the current variable
  ordering
• Jump-Back Sets are Accumulated (AJBS) during the
  forward search phase as follow
• When choosing XiVk add it to the conflict sets
  of all variables
• Xjs such that ?C(, Xi,, Xj,)?G and XiVk ? C
  false
• Jump-Back Sets are Updated (UJBS) during the
  backtracking phase as follows
• After jumping back from Xd to Xb,
• UJBS(Xb) UJBS(Xb) ? (AJBS(Xd) XbVb)

74
Simple CDBJ Run Example
75
Simple CDBJ Run Example
76
Simple CDBJ Run Example
77
Simple CDBJ Run Example
78
CDBJ with Forward Checking
79
CDBJ with Forward Checking
80
CDBJ with Forward Checking,HD and LCV Heuristics

• Variable ordering tie-breaker domain with least
  potentially conflicting value
• e.g., X1 before X7 because g?dom(X1) can only
  conflict with X2 g,
• whereas all values of dom(X7) may conflict with
  values of 3 or 4 variables

81
CDBJ with Forward Checking,HD and LCV Heuristics

• Variable ordering tie-breaker domain with least
  potentially conflicting value
• e.g., X1 before X7 because g?dom(X1) can only
  conflict with X2 g,
• whereas all values of dom(X7) may conflict with
  values of 3 or 4 variables
• No need to backtrack!
• Variable and value ordering heuristics and
  forward checking eliminate the need to backtrack
  for many problem instances

82
Min-Conflict Example

• States 4 queens in 4 columns (44 256 states).
• Actions move queen in column.
• Goal test no attacks.
• Evaluation h(n) number of attacks
• Given random initial state, can solve n-queens in
  almost constant time for arbitrary n with high
  probability (e.g., n 10,000,000)

83
Performance Experiments
Problem Backtracking BTMRV Forward Checking FCMRV Minimum Conflicts
USA (4-colors) (gt1.000K) (gt1.000K) 2K 60 64
n-Queens (2 lt n lt 50) (gt40.000K) 13.500K (gt40000K) 817K 4K
Zebra (ex. 5.13) 3.859K 1K 35K 0.5K 2K
Random 1 415K 3K 26K 2K Not run
Random 2 942K 27K 77K 15K Not run