Foundations of Constraint Processing

1
Constraint Satisfaction 101

• Foundations of Constraint Processing
• CSCE421/821, Fall 2005
• www.cse.unl.edu/choueiry/F05-421-821/
• Berthe Y. Choueiry (Shu-we-ri)
• AVH 123B
• choueiry_at_cse.unl.edu
• Tel [phone]

2
Outline

• Motivating example, application areas
• CSP Definition, representation
• Some simple modeling examples
• More on definition and formal characterization
• Basic solving techniques
• (Implementing backtrack search)
• Advanced solving techniques
• Issues research directions

3
Motivating example

• Context You are a senior in college
• Problem You need to register in 4 courses for
  the Spring semester
• Possibilities Many courses offered in Math, CSE,
  EE, CBA, etc.
• Constraints restrict the choices you can make
• Unary Courses have prerequisites you have/don't
  have Courses/instructors you
  like/dislike
• Binary Courses are scheduled at the same time
• n-ary In CE 4 courses from 5 tracks such as at
  least 3 tracks are covered
• You have choices, but are restricted by
  constraints
• Make the right decisions!!

4
Motivating example (contd)

• Given
• A set of variables 4 courses at UNL
• For each variable, a set of choices (values)
• A set of constraints that restrict the
  combinations of values the variables can take at
  the same time
• Questions
• Does a solution exist? (classical decision
  problem)
• How two or more solutions differ? How to change
  specific choices without perturbing the solution?
• If there is no solution, what are the sources of
  conflicts? Which constraints should be
  retracted?
• etc.

5
Practical applications
Adapted from E.C. Freuder

• Radio resource management (RRM)
• Databases (computing joins, view updates)
• Temporal and spatial reasoning
• Planning, scheduling, resource allocation
• Design and configuration
• Graphics, visualization, interfaces
• Hardware verification and software engineering
• HC Interaction and decision support
• Molecular biology
• Robotics, machine vision and computational
  linguistics
• Transportation
• Qualitative and diagnostic reasoning

6
Constraint Processing

• is about ...
• Solving a decision problem
• While allowing the user to state arbitrary
  constraints in an expressive way and
• Providing concise and high-level feedback about
  alternatives and conflicts
• Related areas
• AI, OR, Algorithmic, DB, TCS, Prog. Languages,
  etc.

7
Power of Constraints Processing

• Flexibility expressiveness of representations
• Interactivity
• users can
  constraints

8
Outline

• Motivating example, application areas
• CSP Definition, representation
• Some simple modeling examples
• More on definition and formal characterization
• Basic solving techniques

9
Defining a problem

• General template of any computational problem
• Given
• Example a set of objects, their relations, etc.
• Query/Question
• Example Find x such that the condition y is
  satisfied
• How about the Constraint Satisfaction Problem?

10
Definition of a CSP

• Given P (V, D, C )
• V is a set of variables,
• D is a set of variable domains (domain values)
  
• C is a set of constraints,
• Query can we find a value for each variable such
  that all constraints are satisfied?

11
Other queries

• Find a solution
• Find all solutions
• Find the number of solutions
• Find a set of constraints that can be removed so
  that a solution exists
• Etc.

12
Representation I

• Given P (V, D, C ), where
• Find a consistent assignment for
  variables

• Constraint Graph
• Variable ? ? node (vertex)
• Domain ? node label
• Constraint ? arc (edge) between nodes

13
Representation II

• Given P (V, D, C ), where
  

• Example I
• Define C ?

14
Outline

• Motivating example, application areas
• CSP Definition, representation
• Some simple modeling examples
• More on definition and formal characterization
• Basic solving techniques

15
Example II Temporal reasoning

• Give one solution .
• Satisfaction, yes/no decision problem

16
Example III Map coloring

• Using 3 colors (R, G, B), color the US map such
  that no two adjacent states have the same color
• Variables?
• Domains?
• Constraints?

17
Example III Map coloring (contd)

• Using 3 colors (R, G, B), color the US map such
  that no two adjacent states have the same color

18
Example IV Resource Allocation

• What is the CSP formulation?

19
Example IV RA (contd)
Interval Order
R1, R3
T1
R1, R3
T2
R1, R3, R4
R1, R3
T2
T4
T3
R1, R2, R3
T5
R2, R4
T6
R2, R4
T7
20
Example V Puzzle

• Given
• Four musicians Fred, Ike, Mike, and Sal, play
  bass, drums, guitar and keyboard, not
  necessarily in that order.
• They have 4 successful songs, Blue Sky,
  Happy Song, Gentle Rhythm, and Nice
  Melody.
• Ike and Mike are, in one order or the other,
  the composer of Nice Melody and the
  keyboardist.
• etc ...
• Query Who plays which instrument and who
  composed which song?

21
Example V Puzzle (contd)

• Formulation 1
• Variables Bass, Drums, Guitar, Keyboard, Blue
  Sky, Happy Song
• Gentle Rhythm and Nice
  Melody.
• Domains Fred, Ike, Mike, Sal
• Constraints
• Formulation 2
• Variables Fred's-instrument, Ike's-instrument,
  ,
• Fred's-song, Ikes's-song, Mikes-song,
  , etc.
• Domains
• bass, drums, guitar, keyboard
• Blue Sky, Happy Song, Gentle Rhythm, Nice
  Melody
• Constraints

22
Example VI Product Configuration

• Train, elevator, car, etc.
• Given
• Components and their attributes (variables)
• Domain covered by each characteristic (values)
• Relations among the components (constraints)
• A set of required functionalities (more
  constraints)
• Find a product configuration
• i.e., an acceptable combination of components
• that realizes the required functionalities

23
Example VII Cryptarithmetic puzzles

• DX1 DX2 DX3 0,1
• DFDTDUDWDRDO0,9
• OO R10X1
• X1WW U10X2
• X2 TT O 10X3
• X3F
• Alldiff(F,D,U,V,R,O)

24
Constraint types examples

• Example I algebraic constraints
• Example II
• (algebraic) constraints
• of bounded difference
• Example III IV coloring, mutual exclusion,
  difference constraints
• Example V VI elements of C must be made
  explicit

25
More examples

• Example VII Databases
• Join operation in relational DB is a CSP
• View materialization is a CSP
• Example VIII Interactive systems
• Data-flow constraints
• Spreadsheets
• Graphical layout systems and animation
• Graphical user interfaces
• Example IX Molecular biology (bioinformatics)
• Threading, etc

26
Outline

• Motivating example, application areas
• CSP Definition, representation
• Some simple modeling examples
• More on definition and formal characterization
• Basic solving techniques

27
Representation (again)

• Macrostructure G(P)
• - constraint graph for
• binary constraints
• - constraint network for
• non-binary constraints
• Micro-structure ? (P)
• Co-microstructure co-?(P)

(V1, a ) (V1, b)
(V2, a ) (V2, c)
(V3, b ) (V3, c)
no goods
(V1, a ) (V1, b)
(V2, a ) (V2, c)
(V3, b ) (V3, c)
28
Constraint arity I

• Given P (V, D, C ) , where
  

• How to represent the constraint V1 V2
  V4 lt 10 ?

29
Constraint arity II

• Given P (V, D, C ) , where
  

Constraints universal, unary, binary, ternary,
, global. A Constraint Network
30
Domain types

• P (V, D, C ) where
  
• Domains
• Restricted to 0,1 Boolean CSPs
• Finite (discrete), enumeration works
• Continuous, sophisticated algebraic techniques
  are needed
• Consistency techniques on domain bounds

31
Constraint terminology

• Arity
• universal, unary, binary, ternary, , global
• Scope
• The set of variables to which the constraint
  applies
• Definition
• Extension all allowed tuples are listed
• Constrained Decision Diagrams (Cheng Yap,
  AAAI 05)
• Intention when it is not practical (or even
  possible) to list all tuples
• Define types/templates of common constraints to
  be used repeatedly linear constraints, All-Diff
  (mutex), Atmost, TSP-constraint,
  cycle-constraint, etc.)
• Implementation
• predicate, set of tuples (list or table), binary
  matrix (bit-matrix), etc.

32
Complexity of CSP

• Characterization
• Decision problem
• In general, NP-complete
• by reduction from 3SAT

33
Proving NP-completeness

• Show that ?1 is in NP
• Given a problem ?1 in NP, show that an known
  NP-complete problem ?2 can be efficiently reduced
  to ?1
• Select a known NP-complete problem ?2 (e.g.,
  SAT)
• Construct a transformation f from ?2 to ?1
• Prove that f is a polynomial transformation
• (Check Chapter 3 of Garey Johnson)

34
What is SAT?

• Given a sentence
• Sentence conjunction of clauses
• Clause disjunction of literals
• Literal a term or its negation
• Term Boolean variable
• Question Find an assignment of truth values to
  the Boolean variables such the sentence is
  satisfied.

35
CSP is NP-Complete

• Verifying that an assignment for all variables is
  a solution
• Provided constraints can be checked in polynomial
  time
• Reduction from 3SAT to CSP
• Many such reductions exist in the literature
  (perhaps 7 of them)

36
Problem reduction

• Example CSP into SAT (proves nothing, just
  an exercise)
• Notation variable-value pair vvp
• vvp ? term
• V1 a, b, c, d yields x1 (V1, a), x2 (V1,
  b), x3 (V1, c), x4 (V1, d),
• V2 a, b, c yields x5 (V2, a), x6 (V2, b),
  x7 (V2,c).
• The vvps of a variable ? disjunction of terms
• V1 a, b, c, d yields
• (Optional) At most one VVP per variable
  

37
CSP into SAT (cont.)

• Constraint
• Way 1 Each inconsistent tuple ? one disjunctive
  clause
• For example
  how many?
• Way 2
• Consistent tuple ? conjunction of terms
• Each constraint ? disjunction of these
  conjunctions
• ? transform into conjunctive normal form (CNF)
• Question find a truth assignment of the Boolean
  variables such that the sentence is satisfied

38
Outline

• Motivating example, application areas
• CSP Definition, representation
• Some simple modeling examples
• More on definition and formal characterization
• Basic solving techniques
• Modeling and consistency checking
• Constructive, systematic search
• Iterative improvement, local search

39
How to solve a CSP?

• Search
• 1. Constructive, systematic
• 2. Iterative repair, local search

40
Before starting search!

• Consider
• Importance of modeling/formulation
• To control the size of the search space
• Preprocessing
• A.k.a. constraint filtering/propagation,
  consistency checking
• reduces size of search space

41
Importance of modeling

• N-queen formulation 1
• Variables?
• Domains?
• Size of CSP?
• N-queens formulation 2
• Variables?
• Domains?
• Size of CSP?

0,1
42
Constraint checking

• Arc-consistency

B
A lt B
A
5.... 18
B lt C
1.... 10
2 lt C - A lt 5
3- B 5 .. 13
4.... 15
C
43
Constraint checking

• Arc-consistency every combination of two
  adjacent variables
• 3-consistency, k-consistency (k ? n)
• Constraint filtering, constraint checking, etc..
• Eliminate non-acceptable tuples prior to search
• Warning arc-consistency does not solve the
  problem

still is not a solution!
44
Systematic search

• Starting from a root node
• Consider all values for a variable V1
• For every value for V1, consider all values for
  V2
• etc..
• For n variables, each of domain size d
• Maximum depth? fixed!
• Maximum number of paths? size of search
  space, size of CSP

45
Systematic search (I) Back-checking

• Systematic search generates dn possibilities
• Are all possibilities acceptable?
• Expand a partial solution only when it is
  consistent
• This yields early pruning of inconsistent paths

46
Systematic search (II) Chronological backtracking

• What if only one solution is needed?
• Depth-first search Chronological backtracking
• DFS Soundness? Completeness?

47
Systematic search (III) Intelligent backtracking

• What if the reason for failure was higher up in
  the tree?
• Backtrack to source of conflict !!
• Backjumping, conflict-directed backjumping,
  etc.

48
Systematic search (IV) Ordering heuristics

• Which variable to expand first?
• Heuristics
• most constrained variable first (reduce branching
  factor)
• most promising value first (find quickly first
  solution)

49
Systematic search (V) Back-checking

• Search tree with only backtrack search?

Root node
Q
50
Systematic search (VI) Forward checking

• Search Tree with domains filter by Forward Check

Root node
51
Improving BT search

• General purpose methods for
• Variable, value ordering
• Improving backtracking intelligent backtracking
  avoids repeating failure
• Look-ahead techniques propagate constraints as
  variables are instantiated

52
Search-tree branching

• K-way branching
• One branch for every value in the domain
• Binary branching
• One branch for the first value
• One branch for all the remaining values in the
  domain
• Used in commercial constraint solvers ILOG,
  Eclipse
• Have different behaviors (e.g., WRT value
  ordering heuristics Smith, IJCAI 95)

53
Summary of backtrack search

• Constructive, systematic, exhaustive
• In general sound and complete
• Back-checking expands nodes consistent with past
• Backtracking Chronological vs. intelligent
• Ordering heuristics
• Static
• Dynamic variable
• Dynamic variable-value pairs
• Looking ahead
• Partial look-ahead
• Forward checking (FC)
• Directional arc-consistency (DAC)
• Full (a.k.a. Maintaining Arc-consistency or MAC)

54
CSP a decision problem (NP-complete)

• Modeling abstraction and reformulation
• Preprocessing techniques
• eliminate non-acceptable tuples prior to search
• Systematic search
• potentially dn paths of fixed lengths
• chronological backtracking
• intelligent backtracking
• variable/value ordering heuristics
• Search hybrids
• Mixing consistency checking with search
  look-ahead strategies

55
Non-systematic search

• Iterative repair, local search modifies a global
  but inconsistent solution to decrease the number
  of violated constraints
• Examples Hill climbing, taboo search, simulated
  annealing, GSAT, WalkSAT, Genetic Algorithms,
  Swarm intelligence, etc.
• Features Incomplete not sound
• Advantage anytime algorithm
• Shortcoming cannot tell that no solution exists

56
Outline of CSP 101

• We have seen
• Motivating example, application areas
• CSP Definition, representation
• Some simple modeling examples
• More on definition and formal characterization
• Basic solving techniques
• We will move to
• (Implementing backtrack search)
• Advanced solving techniques
• Issues research directions