General Principles of Constraint Programming

1
General Principles of Constraint Programming

Jean-Charles REGIN Director of Constraint
Programming ILOG, Sophia Antipolis,
France regin_at_ilog.fr
2
Plan

• General Principles
• CP vs other techniques
• Filtering algorithms
• An example sports scheduling
• Strength of CP
• Weakness of CP
• New research Area
• Recent advances in CP at ILOG
• Conclusion

3
History

• General Problem Solvers in 70s
• ALICE J-F Lauriere, AIJ 78, phD in Paris VI, 76
• Prolog CHIP, ECRC Munich 86 (Alice in Prolog)
• Colmerauer, Gallaire, Van Hentenryck
• Constraint Satisfaction Problems
• Waltz 72, Mackworth 74, Freuder 76
• Industry Bull (Charme), ILOG (Solver) 92

4
Constraint Programming

• In CP a problem is defined from- variables with
  possible values (domain)- constraints
• Domain can be discrete or continuous, symbolic
  values or numerical values
• Constraints express properties that have to be
  satisfied

5
Problem conjunction of sub-problems

• In CP a problem can be viewed as a conjunction of
  sub-problems that we are able to solve
• A sub-problem can be trivial x lt y or complex
  search for a feasible flow
• A sub-problem a constraint

6
Constraints

• Predefined constraints arithmetic (x lt y, x y
  z, x-y gt k, alldiff, cardinality, sequence
• Constraints given in extension by the list of
  allowed (or forbidden) combinations of values
• user-defined constraints any algorithm can be
  encapsulated
• Logical combination of constraints using OR, AND,
  NOT, XOR operators. Sometimes called
  meta-constraints

7
Filtering

• We are able to solve a sub-problem a method is
  available
• CP uses this method to remove values from domain
  that do not belong to a solution of this
  sub-problem filtering or domain-reduction
• E.g x lt y and D(x)10,20, D(y)5,15gt
  D(x)10,14, D(y)11,15

8
Filtering

• A filtering algorithm is associated with each
  constraint (sub-problem).
• Can be simple (x lt y) or complex (alldiff)
• Theoretical basics arc consistency, remove all
  the values that do not belong to a solution of
  the underlined sub-problem.

9
Propagation

• Domain Reduction due to one constraint can lead
  to new domain reduction of other variables
• When a domain is modified all the constraints
  involving this variable are studied and so on ...

10
Propagation

• D(x)D(y)0,1, D(z)0,1,2, D(u)3,4,5,
  D(v)4,5,6
• Alldiff(x,y,z), uz3, vgtu, z-vlt6

11
Propagation

• D(x)D(y)0,1, D(z)0,1,2, D(u)3,4,5,
  D(v)4,5,6
• Alldiff(x,y,z), uz3, vgtu, z-vlt6
• Alldiff(x,y,z) FAgt z2

12
Propagation

• D(x)D(y)0,1, D(z)0,1,2, D(u)3,4,5,
  D(v)4,5,6
• Alldiff(x,y,z), uz3, vgtu, z-vlt6
• Alldiff(x,y,z) FAgt z2
• uz3 FAgt u5

13
Propagation

• D(x)D(y)0,1, D(z)0,1,2, D(u)3,4,5,
  D(v)4,5,6
• Alldiff(x,y,z), uz3, vgtu, z-vlt6
• Alldiff(x,y,z) FAgt z2
• uz3 FAgt u5
• vgtu FAgt v6

14
Propagation

• D(x)D(y)0,1, D(z)0,1,2, D(u)3,4,5,
  D(v)4,5,6
• Alldiff(x,y,z), uz3, vgtu, z-vlt6
• Alldiff(x,y,z) FAgt z2
• uz3 FAgt u5
• vgtu FAgt v6
• z-vlt6 FA gt y1

15
Propagation

• D(x)D(y)0,1, D(z)0,1,2, D(u)3,4,5,
  D(v)4,5,6
• Alldiff(x,y,z), uz3, vgtu, z-vlt6
• Alldiff(x,y,z) FAgt z2
• uz3 FAgt u5
• vgtu FAgt v6
• z-vlt6 FA gt y1
• Alldiff(x,y,z) FAgt x0
• Solution x0, y1, z2, u5, v6

16
Why Propagation?

• Idea problem conjunction of easy sub-problems.
  
• Sub-problems local point of view. Problem
  global point of view. Propagation tries to obtain
  a global point of view from independent local
  point of view
• The conjunction is stronger that the union of
  independent resolution

17
Search

• Backtrack algorithm with strategiestry to
  successively assign variables with values. If a
  dead-end occurs then backtrack and try another
  value for the variable
• Strategy define which variable and which value
  will be chosen.
• After each domain reduction (I.e assignement
  include) filtering and propagation are triggered

18
Constraint Programming

• 3 notions- constraint network variables,
  domains constraints filtering (domain
  reduction)- propagation- search procedure
  (assignments backtrack)
• The structure of every constraint is exploited

19
Plan

• General Principles
• CP vs other techniques
• Filtering algorithms
• An example sports scheduling
• Strength of CP
• Weakness of CP
• New research Area
• Recent advances in CP at ILOG
• Conclusion

20
CP vs other techniques

• CP vs greedy algorithms
• CP vs local search
• CP vs inference engines
• CP vs MIP

21
CP vs greedy algorithm

• Greedy algorithm a search strategy which
  guarantees that there is no backtrack
• CP is a generalization your strategy can be non
  necessarily greedy. Solver manages for you the
  errors of your strategy
• Some problems are solved with few backtracks

22
CP vs local search

• Local search it is hard to respect hard
  constraints. With CP no problem
• Combination of CP and local search is used quite
  often to solve problems find a first solution
  with CP then re-optimize it iteratively.

23
CP vs Inference engines

• (A implies B) represented by (NOT A OR B)
• (x lt 5 implies z gt 4)
• Propagation from Left to Right if x lt 5 then z gt
  4
• Propagation from Right to Left if z lt 5 then
  necessarily x gt4

24
CP vs MIP
MIP approach
CP approach
25
CP vs MIP
MIP approach
CP approach

• Relax the problem floats instead of integers
• 1) Use the Simplex algorithm (polynomial)
• 2) Set float to integer value. Go to 1) and
  backtrack if necessary

26
CP vs MIP
MIP approach
CP approach

• Relax the problem floats instead of integers
• 1) Use the Simplex algorithm (polynomial)
• 2) Set float to integer value. Go to 1) and
  backtrack if necessary

• Identify sub-problems that are easy (called
  constraints)
• 1) Use specific algorithm for solving these
  sub-problems and for performing domain-reduction
• 2) Instantiate variable. Go to 1) and backtrack
  if necessary

27
CP vs MIP
MIP approach
CP approach

• Relax the problem floats instead of integers
• 1) Use the Simplex algorithm (polynomial)
• 2) Set float to integer value. Go to 1) and
  backtrack if necessary
• Global point of view on a relaxation of the
  problem

• Identify sub-problems that are easy (called
  constraints)
• 1) Use specific algorithm for solving these
  sub-problems and for performing domain-reduction
• 2) Instantiate variable. Go to 1) and backtrack
  if necessary
• Local point of view on sub-problems. Global
  point of view by propagation of domain reductions

28
CP vs MIP

• In CP constraints can be non-linear
• Structure of the problem is used in CP
• Semantic of the constraints are used much easier
  to add a global constraint with a filtering
  algorithm than defining a new cut
• First solution given by CP is generally good

29
Plan

• General Principles
• CP vs other techniques
• Filtering algorithms
• An example sports scheduling
• Strength of CP
• Weakness of CP
• New research Area
• Recent advances in CP at ILOG
• Conclusion

30
Arc consistency

• All the values which do not belong to any
  solution of the constraint are deleted.
• Example Alldiff(x,y,z) with D(x)D(y)0,1,
  D(z)0,1,2the two variables x and y take the
  values 0 and 1, thus z cannot take these
  values.FA by AC gt 0 and 1 are removed from D(z)

31
Alldiff constraint
The value graph
1 2 3 4 5 6 7
x1 x2 x3 x4 x5 x6
D(x1)1,2 D(x2)2,3 D(x3)1,3 D(x4)3,4 D(
x5)2,4,5,6 D(x6)5,6,7
32
Value network
(0,1)
1 2 3 4 5 6 7
Default orientation
x1 x2 x3 x4 x5 x6
(0,1)
(1,1)
t
s
(6,6)
33
A feasible flow
(0,1)
1 2 3 4 5 6 7
Default orientation
x1 x2 x3 x4 x5 x6
(0,1)
(1,1)
t
s
(6,6)
34
Residual graph
1 2 3 4 5 6 7
orientation
x1 x2 x3 x4 x5 x6
s
35
Residual graph
1 2 3 4 5 6 7
orientation
x1 x2 x3 x4 x5 x6
s
36
Arc consistency
The value graph
1 2 3 4 5 6 7
x1 x2 x3 x4 x5 x6
D(x1)1,2 D(x2)2,3 D(x3)1,3 D(x4)4 D(x5
)5,6 D(x6)5,6,7
37
Alldiff constraint

• Compute a feasible flow
• Compute the strongly connected components
• Remove every arc of flow value 0 for which the
  ends belong to two different components
• Linear algorithm achieving arc consistency
• Idem for global cardinality constraints
• work well due to (0,1) arcs

38
Plan

• General Principles
• CP vs other techniques
• Filtering algorithms
• An example sports scheduling
• Strength of CP
• Weakness of CP
• New research Area
• Recent advances in CP at ILOG
• Conclusion

39
The problem

• n teams and n-1 weeks and n/2 periods
• every two teams play each other exactly once
• every team plays one game in each week
• no team plays more than twice in the same period

40
The problem

• n teams and n-1 weeks and n/2 periods
• every two teams play each other exactly once
• every team plays one game in each week
• no team plays more than twice in the same period

• Problem 10teams of the MIPLIB
• (n10 and the objective function is dummy)
• MIP is not able to find a solution for n14
• CP finds a solution for n10 in 0.06s, n14 in
  0.2, n40 in 6h

41
The problem

• n teams and n-1 weeks and n/2 periods
• every two teams play each other exactly once
• every team plays one game in each week
• no team plays more than twice in the same period

For 40 teams 800 variables with 39 possible
values for each variable.
42
CP model variables
For each slot 2 variables represent the teams
and 1 variable represents the match are defined
1 vs 6
M33 variable (M3312)
Mij1 ltgt 0 vs 1 or 1 vs 0 Mij12 ltgt 1 vs 6 or 6
vs1
T33a variable (T33a6) T33h variable (T33h1)
43
CP model T variables
D(Tija)1,n-1 D(Tijh)0,n-2
Tijh lt Tija
44
CP model M variables
D(Mij)1,n(n-1)/2
45
CP model constraints

• n teams and n-1 weeks and n/2 periods
• every two teams play each other exactly once
• every team plays one game in each week
• no team plays more than twice in the same period

46
CP model constraints

• n teams and n-1 weeks and n/2 periods
• every two teams play each other exactly once
• every team plays one game in each week
• no team plays more than twice in the same period

Alldiff constraints defined on M variables
47
CP model constraints

• n teams and n-1 weeks and n/2 periods
• every two teams play each other exactly once
• every team plays one game in each week
• no team plays more than twice in the same period

48
CP model constraints

• n teams and n-1 weeks and n/2 periods
• every two teams play each other exactly once
• every team plays one game in each week
• no team plays more than twice in the same period

For each week w Alldiff constraint defined on
Tpwh, p1..4 U Tpwa, p1..4
49
CP model constraints

• n teams and n-1 weeks and n/2 periods
• every two teams play each other exactly once
• every team plays one game in each week
• no team plays more than twice in the same period

50
CP model constraints

• n teams and n-1 weeks and n/2 periods
• every two teams play each other exactly once
• every team plays one game in each week
• no team plays more than twice in the same period

For each period p Global cardinality constraint
defined on Tpwh, w1..7 U Tpwa, w1..7 every
team t is taken at most 2
51
CP model constraints

• For each slot the two T variables and the M
  variable must be linked together exampleM12
  game T12h vs T12a

52
CP model constraints

• For each slot the two T variables and the M
  variable must be linked together exampleM12
  game T12h vs T12a
• For each slot we add Cij a ternary constraint
  defined on the two T variables and the M
  variable exampleC12 defined on T12h,T12a,M12

53
CP model constraints

• For each slot the two T variables and the M
  variable must be linked together exampleM12
  game T12h vs T12a
• For each slot we add Cij a ternary constraint
  defined on the two T variables and the M
  variable exampleC12 defined on T12h,T12a,M12
• Cij are defined by the list of allowed tuples
  for n4 (0,1,1),(0,2,2),(0,3,3),(1,2,4),(1,3,5)
  ,(2,3,6)(1,2,4) means game 1 vs 2 is the game
  number 4

54
CP model constraints

• For each slot the two T variables and the M
  variable must be linked together exampleM12
  game T12h vs T12a
• For each slot we add Cij a ternary constraint
  defined on the two T variables and the M
  variable exampleC12 defined on T12h,T12a,M12
• Cij are defined by the list of allowed tuples
  for n4 (0,1,1),(0,2,2),(0,3,3),(1,2,4),(1,3,5)
  ,(2,3,6)(1,2,4) means game 1 vs 2 is the game
  number 4
• All these constraints have the same list of
  allowed tuples
• Efficient arc consistency algorithm for this kind
  of constraint is known

55
First model
Introduction of a dummy column
56
First model
Introduction of a dummy column

• We can prove that
• each team occurs exactly twice for each period

57
First model
Introduction of a dummy column

• We can prove that
• each team occurs exactly twice for each period

58
First model
Introduction of a dummy column

• We can prove that
• each team occurs exactly twice for each period

59
First model
Introduction of a dummy column

• We can prove that
• each team occurs exactly twice for each period
• each team occurs exactly once in the dummy column

60
First model
Introduction of a dummy column

• The problem is exactly the same
• The solver is helped by such constraint. It can
  deduce someinconsistencies more quickly

61
First model strategies

• Break symmetries 0 vs w appears in week w

62
First model strategies

• Break symmetries 0 vs w appears in week w
• Teams are instantiated- the most instantiated
  team is chosen- the slots that has the less
  remaining possibilities (Tijh or Tija is minimal)
  is instantiated with that team

63
First model results
MIPLIB
MIP solver limit
64
Second model

• Break symmetry 0 vs 1 is the first game of the
  dummy column

65
Second model

• Break symmetry 0 vs 1 is the first game of the
  dummy column
• 1) Find a round-robin. Define all the games for
  each column (except for the dummy)- Alldiff
  constraint on M is satisfied- Alldiff constraint
  for each week is satisfied

66
Second model

• Break symmetry 0 vs 1 is the first game of the
  dummy column
• 1) Find a round-robin. Define all the games for
  each column (except for the dummy)- Alldiff
  constraint on M is satisfied- Alldiff constraint
  for each week is satisfied
• 2) set the games in order to satisfy constraints
  on periods. If no solution go to 1)

67
Second model strategy
M variables are instantiated
68
Second model strategy
M variables are instantiated
69
Second model strategy
M variables are instantiated
70
Second model strategy
M variables are instantiated
71
Second model strategy
M variables are instantiated
72
Second model results
MIPLIB
MIP limit
First model limit
73
Assume P ? NP

• Ok, we cannot avoid an exponential behavior
• For some instances, an NP Complete Problem will
  required an exponential time to be solved
• So, our only hope is to shift the exponential
  such that the problem is solvable for a size and
  a time that are acceptable

74
Shifting the exponential
75
Shifting the exponential
76
Why does CP perform well?

• Pure discrete problem. You can give any number to
  the teams
• This is a feasibility problem (no objective
  function).
• No arithmetic symbol , -, is used
• A global point of view on the global cardinality
  constraints (i.e. group these constraints into
  only one) does not help

77
Plan

• General Principles
• CP vs other techniques
• Filtering algorithms
• An example sports scheduling
• Strength of CP
• Weakness of CP
• New research Area
• Recent advances in CP at ILOG
• Conclusion

78
Strength of CP

• Very flexible (easy to take into account new
  constraints)
• The system is open you can define you own
  constraints, your own search mechanism.
• CP allows the use of sophisticated strategies,
  you can use the knowledge of the domain of
  application.
• You just have to respect a protocol given by a
  solver. The solver manages the propagation and
  provides you with a lot of predefined things

79
Strength of CP

• CP is exact no solution is lost even for float
  variables
• Any existing algorithm can be integrated in CP as
  a filtering algorithm of a constraints
• Concepts are simple
• A first model can be defined and tested quickly
• For optimization problems the first solution is
  a good one
• Easy to introduce your new idea in the system
• Cooperation is easy thanks to constraints

80
Plan

• General Principles
• CP vs other techniques
• Filtering algorithms
• An example sports scheduling
• Strength of CP
• Weakness of CP
• New research Area
• Recent advances in CP at ILOG
• Conclusion

81
Weakness of CP

• Must be improved for optimization problems spend
  too much time in proving sub-optimality
• First step integration of cost in the
  constraints
• Sometimes lack of global point of view
• Dark zones press Enter key then ?
• Relaxation is not good for CP. We learn
  relaxation at school!

82
Plan

• General Principles
• CP vs other techniques
• Filtering algorithms
• An example sports scheduling
• Strength of CP
• Weakness of CP
• New research Area
• Recent advances in CP at ILOG
• Conclusion

83
New research areas

• New point of view CP is based on filtering
  algorithm, i.e. Given a property P defining a
  necessary condition for an element to be in a
  solutionFind as quickly as possible ALL elements
  that do not satisfy P
• Ex alldiff constraint and matching, cardinality
  constraint and flows etc
• Close to sensitivity analysis, but also different
  (for instance we only have monotonic
  modifications).

84
New research areas

• Consider a Minimization problem, and OBJ the
  objective. Suppose that we found a solution
  with OBJ25.
• We will reject any solution with OBJ gt 24. So
  if xa leads to an OBJ gt 24 then value a must be
  removed from D(x).
• If we have a lower bound of OBJ then we can use
  itif lb(OBJ,xa) gt 24 then remove a from D(x)
• Problem literature mainly gives upper bound for
  minimization problems and lower bound for
  maximization problems.
• Not always easy to get lb of good quality

85
New research areas

• The very same algorithm is called thousand times
  (million sometimes)
• The incremental aspect of the algorithm becomes
  really important.

86
Plan

• General Principles
• CP vs other techniques
• Filtering algorithms
• An example sports scheduling
• Strength of CP
• Weakness of CP
• New research Area
• Recent advances in CP at ILOG
• Conclusion

87
Recent advances in CP at ILOG

• Lets start with a real world example that
  involved a lot of people in ILOG The ROCOCO
  project
• From this project we learnt a lot of things

88
The ROCOCO Problem (1)

• Routing of Communications
• Mono-routing each demand from a point p to a
  point q must follow a unique path
• Dimensioning of Links
• The capacity of each link must exceed the sums of
  the demands going through the link
• Additional Constraints
• Depend on the customer for whom the network is
  designed

89
The ROCOCO Problem (2)

• Data
• Customer traffic demands
• Possible links, capacities and costs

27Kb/s
S2
S1
S4
115Kb/s
S3

• Result
• Minimal cost network able to simultaneously
  respond to all the demands
• Route for each demand

S2
S1
S4
Rented capacity 256Kb/s
S3
90
The ROCOCO Problem (3)

• Cost minimization principle
• Traffic demands share link capacities

S2
S1
512Kb/s
128Kb/s
S3
91
The Problem (4)

• Demands share links
• ? demandsi?j ? capacityi?j
• Technological constraints

92
The Problem (5)
64Kb/s
64Kb/s
64Kb/s
64Kb/s

• Side constraints
• Quality of service
• Reuse of existing equipment (limit on the number
  of ports, maximal traffic at a node)
• Commercial and legal constraints
• Possible future network evolution
• Network management (e.g., traffic concentration)

93
Optional Constraints

• Security some commodities to be secured cannot
  go through unsecured nodes and links
• No line multiplication at most one line per arc.
• Symmetric routing demands from node p to node q
  and demands from node q to node p are routed on
  symmetric paths.
• Number of bounds (hops) the number of arcs of
  the path used to route a given demand is limited.
• Number of ports the number of links entering
  into or leaving from a node is limited.
• Maximal traffic the total traffic managed by a
  given node is limited.

64Kb/s
64Kb/s
64Kb/s
64Kb/s
94
Numerical Characteristics (1)
95
We worked a lot!
96
The key idea path variable!
97
General considerations

• When solving a problem in CP
• Potential performance gain
• data structure optimization (code) x 10
• search strategies x 1 000
• model x 1 000 000
• Repartition of effort for ROCOCO
• data structure optimization (code) 65
• search strategies 25
• model 10

98
General considerations

• When solving a problem in CP
• Potential performance gain
• data structure optimization (code) x 10
• search strategies x 1 000
• model x 1 000 000
• Repartition of effort for ROCOCO
• data structure optimization (code) 65
• search strategies 25
• model 10
• Objective of CP Optimizer
• data structure optimization (code) 0
• search strategies 10
• model 90

99
CP at ILOG
Focus

• Our current focus is on simplifying the solving
  process for the user
• The user can then concentrate on modeling
• Why?
• Huge ROI a good model may lead to x1000000
  improvement
• The goal of CP is to be close to the natural
  description of the problem
• Usability of our product will be improved

100
CP at ILOG
Focus

• Our current focus is on simplifying the solving
  process for the user
• The user can then concentrate on modeling
• How?
• Introduce a built-in intelligent search strategy
• Control the strategy only through higher-level
  mechanisms
• Increase speed of the search
• Increase inference strength (constraint
  propagation)
• Increase efficiency of the engine

101
ILOG CP

• ILOG Solver is a mature product which has been on
  the market for fifteen years
• We are embarking on a new product with a
  redesigned engine which we call ILOG CP

102
Conclusion

• CP is a general technique can encapsulate a lot
  of work
• CP is an efficient method for solving some
  combinatorial problems small or large
• Filtering algorithms are quite important for non
  binary constraints
• CP allows the use of sophisticated strategies
• If you want to use CP think CP (avoid Boolean
  (0-1) variables). CP allows the use of symbolic
  representation

103
More information

• Go to www.ilog.com
• Go to my webpagewww.constraint-programming.com/p
  eople/reginor google my name Jean Charles Regin