Constraintbased problem solving

1
Constraint-based problem solving

• Model problem
• specify in terms of constraints on acceptable
  solutions
• define variables (denotations) and domains
• define constraints in some language
• Solve model
• define search space / choose algorithm
• incremental assignment / backtracking search
• complete assignments / stochastic search
• design/choose heuristics
• Verify and analyze solution

2
Constraint-based problem solving

• Model problem
• specify in terms of constraints on acceptable
  solutions
• define variables (denotations) and domains
• define constraints in some language
• Solve model
• define search space / choose algorithm
• incremental assignment / backtracking search
• complete assignments / stochastic search
• design/choose heuristics
• Verify and analyze solution

Constraint Satisfaction Problem
3
Constraint satisfaction problem

• A CSP is defined by
• a set of variables
• a domain of values for each variable
• a set of constraints between variables
• A solution is
• an assignment of a value to each variable that
  satisfies the constraints

4
Example Crossword puzzles
5
Options

• CSP, binary CSP, SAT, 3-SAT, ILP, ...
• Model and solve in one of these languages
• Model in one language, translate into another to
  solve

6
Article of faith

• Constraints arise naturally in most areas of
  human endeavor. They are the natural medium of
  expression for formalizing regularities that
  underlie the computational and physical worlds
  and their mathematical abstractions.
• P. Van
  Hentenryck and V. Saraswat
• 
  Constraint Programming Strategic
  Directions

7
Reducibility
NP-Complete
8
Options

• CSP, binary CSP, SAT, 3-SAT, ILP, ...
• Model and solve in one of these languages
• Model in one language, translate into another to
  solve

9
Importance of the model

• In integer programming, formulating a good
  model is of crucial importance to solving the
  model.
• 
  G. L. Nemhauser and L. A. Wolsey
• 
  Handbook in OR MS, 1989

Same for constraint programming.

Folk Wisdom, CP practitioners
10
Measures for comparing models

• How easy is it to
• write down,
• understand,
• modify, debug,
• communicate?
• How computationally difficult is it to solve?

11
Constraint systems/languages
12
Abstractions

• Capture commonly occurring constraints
• special propagation algorithms
• e.g. alldifferent,
• cardinality,
• cumulative, ...
• User-defined constraints
• full power of host language

13
CSP versus ILP

• OO databases versus Relational databases
• C versus C
• C versus Fortran
• C versus Assembly language

14
Computational difficulty?

• What is a good model depends on algorithm
• Choice of variables defines search space
• Choice of constraints defines
• how search space can be reduced
• how search can be guided

15
Improving model efficiency

• Given a model
• Add/remove variables, values, constraints (keep
  the denotation of the variables)
• Use/translate to a different representation (chang
  e the denotation of the variables)

16
Adding redundant constraints

• Improve computational efficiency of model by
  adding right constraints
• symmetries removed
• dead-ends encountered earlier in search process
• Three methods
• add hand-crafted constraints during modeling
• apply a consistency algorithm before solving
• learn constraints while solving

17
Examples

• Adding hand-crafted constraints
• crossword puzzles
• Applying a consistency algorithm
• dual representation

18
Adding redundant variables

• Variables that are abstractions of other
  variables
• e.g, decision variables
• Suppose x has domain 1,,10
• Add Boolean variable to represent
  decisions
• (x
• Variables that represent constraints

19
Translations between models

• Improve computational efficiency by completely
  changing the model
• change denotation of variables
• e.g, convert from non-binary to binary
• aggregate variables
• e.g, timetabling multiple sections of a course
• Not much as has been done on either the theory or
  the practice side

20
Conversion to binary

• Any CSP can be converted into one with only
  binary constraints
• Two techniques known
• dual graph method (Dechter Pearl, 1989)
• hidden variable method (Peirce, 1933 Dechter
  1990)
• Translations are polynomial if constraints are
  represented extensionally

21
3-SAT example
(x1 ? x2 ? x6) ? (?x1 ? x3 ? x4) ?
(?x4 ? ?x5 ? x6) ? (x2 ? x5 ? ?x6)

• Non-binary CSP
• Boolean variables x1, , x6
• constraints one for each clause

C1(x1, x2, x6) (0,0,1), (0,1,0), (0,1,1),
(1,0,0),
(1,0,1), (1,1,0), (1,1,1) C2(x1, x3, x4) ...
22
Dual graph representation
x2, x6
y1 (x1, x2, x6)
y4 (x2, x5, x6)
x6
x5, x6
x1
y2 (x1, x3, x4)
y3 (x4, x5, x6)
x4
23
Hidden variable representation
h1
h4
x1
x2
x3
x4
x5
x6
h3
h2
24
Hidden variable representation

• Non-binary constraint
• Augmented constraint
• Binary constraints

C1(x1, x2, x6) (0,0,1), (0,1,0), (0,1,1),
(1,0,0), (1,0,1),
(1,1,0), (1,1,1)
C1(x1, x2, x6, h1) (0,0,1,0), (0,1,0,1),
(0,1,1,2), (1,0,0,3),
(1,0,1,4), (1,1,0,5), (1,1,1,6)
R1(x1, h1) (0,0), (0,1), (0,2), (1,3), (1,4),
(1,5), (1,6) R2(x2, h1) (0,0), (1,1), (1,2),
(0,3), (0,4), (1,5), (1,6) R3(x6, h1) (1,0),
(0,1), (1,2), (0,3), (1,4), (0,5), (1,6)
25
Dual exponentially worse
x1, ?x1 ? x2, ?x1 ? ?x2 ? x3, ..., ?x1 ? ?
?xn-1 ? xn

• FC on dual O(2n)
  consistency checks

• FC on non-binary O(n) consistency checks

x1
h1
x2
h2
x3
h3
26
Dual exponentially better
x1 ? ? xn-1, x1 ? ? xn-2 ? xn, , x2 ? ? xn

• FC on dual O(n2)
  consistency checks

• FC on non-binary O(n2n) consistency checks

x1
h1
x2
h2
x3
h3
27
Bounds on performance

• Worst case
• Best case

FC(non-binary) ? FC(hidden)
dk
FC(hidden) ? FC (non-binary)
dn
28
Crossword puzzles
1
2
3
4
5
a aardvark aback abacus abaft abalone abandon ...
Mona Lisa monarch monarchy monarda ... zymurgy zyr
ian zythum
6
7
8
9
10
11
12
13
14
17
18
16
15
20
22
21
23
19
29
Crossword puzzles
30
Crossword puzzles
31
Model of random non-binary CSP

• n variables
• each with domain size d
• m constraints
• each with k variables, chosen at random
• each with t tuples, chosen at random

32
Order of magnitude curves
n 20 d 2 k 3
n 20 d 10 k 3
(number of constraints) / (number of variables)
n 20 d 10 k 5
n 20 d 2 k 5
(number of tuples in constraints) / (maximum
tuples)
33
Random 3-SAT
34
Project Planning

• Given start state, goal state, and actions
  determine a plan (a sequence of actions)

Box1
Box2
Box2
35
Contrasts

• CP philosophy/methodology
• emphasis on modeling, domain knowledge
• general purpose search algorithm
• backtracking with constraint propagation
• Successful
• e.g., can solve practical scheduling problems

36
Contrasts

• Planning philosophy/methodology
• emphasis on minimal model
• just representation of actions
• special-purpose search algorithms
• Not as successful
• has not solved many practical problems

37
Tradeoffs

• Robust CSP model needed for each new domain
• can require much intellectual effort
• Less work needs to be done on algorithms
• many general purpose constraint solvers available

38
CSP model of planning

• State-based model
• model each state by a collection of variables
• constraints enforce valid transitions between
  states
• Example logistics world
• variable for each package, truck, plane
• domains of packages all locations, trucks,
  planes
• domains of trucks, planes all locations

39
CSP model constraints (I)

• Action constraints
• model the effects of actions
• patterned after explanation closure axioms
• State constraints
• variables within a state must be consistent

40
Improving model efficiency

• Can add/remove/aggregate/decompose
• variables
• domain values
• constraints
• Here
• added hidden variables
• added redundant symmetry-breaking constraints

41
CSP model constraints (II)

• Symmetric values constraints
• break symmetries on values variables can be
  assigned
• Action choice constraints
• break symmetries on equivalent permutations of
  actions

42
CSP model constraints (III)

• Domain constraints
• restrictions on original domains of variables
• Capacity constraints
• bounds on resources
• Distance constraints
• bounds on steps needed for a variable to change
  from one value to another

43
Solving the CSP model

• Given an instance of a planning problem
• generate a model with one step in it
• instantiate variables in the initial and goal
  states
• search for a solution (GACCBJ)
• repeat, incrementing number of steps, until
  plan is found
• Properties
• forwards, backwards, or middle out planner
• sound, complete, guaranteed to terminate

44
Experiments

• Five test domains from AIPS98
• Five planners
• CPlan
• Blackbox (Kautz Selman)
• HSP (Bonet Geffner)
• IPP (Koehler Nebel)
• TLPlan (Bacchus Kabanza)
• Setup
• machines 400MHz Pentium IIs
• resources 1 hour CPU time, 256 Mb memory

45
Gripper problems
46
Logistics problems
47
Mystery problems
48
MysteryPrime problems
49
Grid problems
50
Related work

• Planning as a CSP
• satisfiability (Kautz Selman)
• ILP (e.g., Bockmayr Dimopoulos)
• Adding declarative knowledge to improve
  efficiency
• hand-coded (e.g, Kautz Selman, Bacchus
  Kabanza)
• automatically derived (e.g., Fox Long, Nebel et
  al.)

51
Project Resource-constrained instruction
scheduling (I)
a j ? i h b k? i g c l ? j 1
1 a, b 2 c

• Modern architectures (VLIW) allow instruction
  level parallelism
• multiple functional units
• Compiler to generate code that takes advantage of
  parallelism
• resource-constrained scheduling task

52
Project Resource-constrained instruction
scheduling (II)

• Current methods
• heuristic
• recent integer linear programming approach
• Project
• investigate constraint programming approach
• GNU compiler, MERCED chip

53
Project Theoretical analysis of alternative
models

• Improve computational efficiency by completely
  changing the model
• change denotation of variables
• e.g, convert from non-binary to binary
• aggregate variables
• e.g, timetabling multiple sections of a course
• Not much as has been done on either the theory or
  the practice side

54
Conclusions

• Importance of the model
• ease of modeling is important
• form of model is important
• Constraint programming advantages
• succinctness, declarativeness of models
• flexibility in specifying model
• on some classes of problems speed

55
Conclusions

• Advantages shared with CSP-like approaches
• expressiveness of modeling language
• declarativeness of models
• independence of model and solving algorithm
• Advantages over other CSP-like approaches
• succinctness of models
• robustness scales well, not as brittle
• speed

56
Future Work

• Approximate planning
• heuristic CSP models
• solve same CSP models using local search
• Alternative CSP models
• action-based models vs state-based models
• finding the right model can be key to solving
  difficult combinatorial problems