Constraints and Optimisation

1
Constraints and Optimisation

• Mark Wallace with Kim Marriott,
• Maria Garcia de la Banda and Reza Rafeh

2
Overview

• Introduction
• Three Optimisation Paradigms
• Mapping Conceptual Models to Design Models
• Conclusion

3
Constraints and Optimisation in Action

• Truck, ship, train and plane transportation
• Personnel rostering from call centre staff, to
  surgeons
• Optimal design of artifacts from cars to
  integrated circuits
• Production of everything from steel to chocolate
• Network planning and optimisation for water,
  electricity, gas and the internet itself.
• Financial portfolio optimisation
• and many more...

4
Precursors and Foundation of Constraint
Programming

• AI
• 1975 Waltz scene labelling,
• 1981 Stefik Molgen
• OR
• 1947 Dantzig linear programming
• 1994 Carlier Pinson scheduling
• CS
• 1982 Colmerauer Prolog I-IV
• 1989 Van Hentenryck CHIP
• 1987 Jaffar, Lassez the CLP scheme
• "Constraint Logic Programming", Technical Report
  86-74, Monash University, Victoria, Australia,
  June 1986.

5
The Holy Grail

• Model Problems Naturally
• constraints
• solution properties
• Solve them efficiently
• overcome combinatorial explosion
• Compile
• Natural models to efficient solutions
• Conceptual Model to Design Model

6
Conceptual Modelling

• A set of decision variables
• numeric continuous/discrete
• boolean
• symbolic
• A set of constraints on the variables
• a constraint only admits specified combinations
  of values
• extensional cext(X,Y) iff ltX,Ygt in lta,1gt,
  lta,3gt,ltb,1gt
• intensional X gt Y
• An optimisation function
• a constraint relating decision variables to a
  numeric cost variable
• copt(X,Y,Cost) iff Cost XY

7
Terms

• Soft / hard constraints
• Constraints that can / cant be violated
• Solution
• An assignment of values to the variables that
  satisfies all the hard constraints
• Optimal Solution
• A solution for which no other solution has a
  better cost value
• Partial solution
• An assignment of values to some of the variables
  that satisfies all the hard constraints

8
Perfect Rectangle Dissection
15
1
4
32
7
14
8
18
18
33
9
10
10
9
8
7
4
1
14
32X33 Rectangle
9 Squares
9
Problem Statement

• Each green square is inside the blue rectangle
• No two green squares overlap
• Check - the area of the blue rectangle is the sum
  of the areas of the green squares
• Find a solution

10
Conceptual Model Precise problem statement

• Structured Data
• Square (var 0..100 x, y PosInt size)
• list of Square squares (x_,y_,sizes) s
  in 1,4,7,8,9,10,14,15,18
• Each green square is inside the blue rectangle
• forall(s in squares)
• s.x s.size lt 32 /\ s.y s.size lt 33
• No two green squares overlap
• forall(i,j in 1..9 where iltj) nonOverlap(squares
  i, squaresj)
• nonOverlap(Square s,t)
• s.x s.size lt t.x \/
• t.x t.size lt s.x \/
• s.y s.size lt t.y \/
• t.y t.size lt s.y

Zinc a language for conceptual modelling
11
Efficient Problem Solution

• Add redundant constraints
• Each column must be completely covered
• onCol(Squares, int r) s.x lt r /\ r lt s.x
  s.size
• forall(p in 1..32)
• sum(s in Squares) (s.size holds(onCol(s,p)))
  33
• Search cleverly
• For each column, in turn, choose which squares
  start on that column
• Research Question
• A language for specifying search

12
Perfect Rectangle Dissection
15
1
4
32
7
14
18
8
18
33
9
4
10
15
7
10
1
9
8
14
32X33 Rectangle
9 Squares
13
Results
Problem 1 32X33, 9 Squares
Problem 2 112X112, 21 squares
CPU time to find all solutions
No extra constraints Simple search Extra
constraints Simple search Extra
constraints Clever search
Timeout Timeout
1 second Timeout
0.1 seconds 2 minutes
14
Overview

• Introduction
• Three Optimisation Paradigms
• Mapping Conceptual Models to Design Models
• Conclusion

15
Three Optimisation Paradigms

• Mathematical Programming
• Meta-heuristics
• Constraint solving and optimisation (CSOP)

16
Mathematical Programming

• Linear inequations
• 2X 3Y lt 10
• X Y lt 4
• maximise(X)
• efficient algorithms available
• Integer constraints
• integer(X)
• for expressing NP complete problems
• solve by tree search
• Non-linear constraints
• X2 Y2 23

Solution X 22/5, Y 2/5
Solution X4, Y 2/3
Solution X2, Y2
17
Meta-heuristics

• Scheme
• generate an initial state
• repeatedly explore neighbourhood, and modify
  state
• restart
• Improving one solution
• hill climbing
• simulated annealing
• tabu search
• Combining multiple solutions
• genetic algorithms
• ant colony optimisation
• Improving one algorithm
• neural nets

Arbitrary constraints, but discrete states
18
Constraint Solving and Optimisation (CSOP)

• Search then propagate
• Search
• post additional constraints, narrowing the search
  space
• Propagate
• infer more constraints from those added so far
• Prune
• when constraints are inconsistent / conflict with
  optimality

Arbitrary constraints but needs propagation
algorithm
19
Mathematical Programming Constraint Handling

• Easy linear constraints enforce
• optimally satisfy a (linear) subset of the
  constraints
• Difficult other constraints - relax
• fix conflicts by adding more linear constraints

20
Meta-heuristics constraint handling

• Easy built into clever move operator
• 3-swap for travelling salesman problem
• Hard punish violations by a penalty

21
CSOP Constraint Handling

• Easy send to complete constraint solver
• Difficult apply constraint propagation
• X,Y,Z1..10, X2 Y Z2
• X2 in 1..99 gt X in 1..9
• Z2 in 2..100 gt Z in 2..10,
• Z2 in 1..81 1..10 gt Z in 2..9
• X2 in 4..81- 1..10 gt X in 1..8
• Z2 in 1..641..10 gt Z in 2..8
• X2 in 4..64- 1..10 gt X in 1..7
• Z2 in 1..491..10 gt Z in 2..7
• X2 in 4..49- 1..10 gt X in 1..6
• Z2 in 1..361..10 gt Z in 2..6
• X2 in 4..36- 1..10 gt X in 1..5
• Z2 in 1..251..10 gt Z in 2..5
• X2 in 4..251..10 gt X in 1..4

22
A Toy Example

• The N-queens problem
• Given any integer N,
• place N queens on an NxN chessboard,
• so that no two queens threaten each other

23
N-Queens Problem Formulation
N variables (one for each queen) Qi Each
variable has domain 1..N
24
Tree search with passive constraint checking
1
2
3
4
5
6
7
8
A
B
C
Fail and try another branch
D
E
F
G
H
25
Tree search with active constraints
1
2
3
4
5
6
7
8
A
B
C
Fail and try another branch
D
E
F
G
H
26
Ships and Berths
10 ships
5 Berths
One choice per millisecond -gt Search in 3 minutes
27
More Ships and Berths
20 ships
10 berths
One choice per millisecond -gt Search in
3000 million years
28
Tackling Scalability

• Breakdown into Subproblems
• eg first 10 ships in first 5 berths,and next ten
  ships in next 10 berths
• gt Explore 1 / 10000000000000 of the search space
• Mathematical programming
• linear relaxation
• gt may yield 1/2 a ship in each of two berths!
• Meta-heuristics
• move from one solution to a neighbour
• gt may not satisfy hard constraints!
• CSOP
• prune by propagation
• gt may not prune enough

Research question How do we combine accuracy
with scalability
29
Three paradigms and Search

• Tree Search
• Search a tree starting at the root
• Mathematical programming
• CSOP
• Local Search
• Search a space of assignments by making changes
• Meta-heuristics

30
Illustrative Example
Solutions A0, B,C,D 1 -gt Profit10 A1,
B,C,D 0 -gt Profit15
31
Local search change one variables value
15
15
Key (a,b,c,d) means Aa, Bb, Cc, Dd
(0,1,1,0)
(0,1,0,1)
5
10
20
(1,1,0,1)
(0,0,1,0)
(0,1,1,1)
0
15
(1,1,1,1)
(0,0,1,1)
20
5
5
Starting from it takes 4 moves to reach
(0,0,0,1)
10
(1,0,1,1)
(1,1,1,0)
10
10
(1,0,0,1)
(1,0,1,0)
15
15
(1,0,0,0)
32
Tree Search prune infeasible branches
33
Comparing tree and local search
Local Search
Tree Search

• Focus on optimality
• The cost of each node is known precisely
• Explore complete assignments
• Cannot handle partial assignments

• Focus on feasibility
• Descendants of infeasible nodes are never
  explored
• Explore partial assignments
• Can only reach a node after reaching all its
  ancestors in the tree

34
Exploding some myths about search
partial assignments

• Tree search
• constructive (partial/total assignments)
• systematic or random
• complete or incomplete

• Local search
• move-based (only total assignments)
• random or systematic
• incomplete or complete

35
Constraint Programming

• Mathematical programming
• Meta-heuristics
• CPOC

Constraint Programming
Research Question Devise hybridisation forms
for combining the paradigms
36
Overview

• Introduction
• Three Optimisation Paradigms
• Mapping Conceptual Models to Design Models
• Conclusion

37
Architecture
PhD
Solver Platform
38
Conceptual to Design

• Conceptual Model
• Zinc
• Solver Choices
• Add constraint behaviour
• Search Choices
• Add search
• Design Model
• Mercury

Out of box solve
39
Constraint Behaviours
Propagation
Simplex
Constraint Solver Libraries
Constraint
Global
Repair
40
Global constraint cumulative
Task (var 0..100 start PosInt duration,
resource)
cumulative(list of TaskTasks, PosIntTotalResourc
e)
Meaning The tasks do not exceed the available
resource Implementation OR algorithm by Carlier
and Pinson
41
Cumulative global constraint
cumulative(task1,task2,task3, 1)
9
9
9
task1
task3
task2
start5..7 start10..30
start0..6
Detecting Inconsistency
18
task1 and task 3
Span18
Min start0
Max end 16
42
Cumulative global constraint propagation
cumulative(task1,task2,task3, 1)
9
9
Span18
task1
task2
Max end 19
Min Start 0
9
task3
start1..30
task3 cant fit between Min Start and Max
End Propagated constraint Task3.start gt 18
43
Cumulative Intuition
Tasks
Fit the tasks onto the resource
Resource
44
Conceptual to Design

• Conceptual Model
• Zinc
• Solver Choices
• Add constraint behaviour
• Search Choices
• Add search
• Design Model
• Mercury

1
2
Research question Devise control language and
mechanism for adding constraint behaviour and
search
45
Logical Transformations

• Conceptual Model
• Decompose
• Transform
• Tighten
• Link
• Conceptual Model

46
Decompose

• Constraints
• Variables
• Dynamic
• Overlapping

47
Decomposing the Perfect Rectangle Problem
Subproblem 1 schedule from left to right
15
1
4
32
7
8
18
33
9
18
14
15
10
10
9
7
8
4
1
14
32X33 Rectangle
9 Squares
48
Decomposing the Perfect Rectangle Problem
Subproblem 1 schedule from left to
right Subproblem 2 schedule from bottom to top
15
1
4
32
7
18
14
8
18
33
9
4
10
15
7
10
9
1
8
14
32X33 Rectangle
9 Squares
49
Combining subproblem solutions

• Sequential
• simplest hybridisation form
• Parallel
• optimistic but can be efficient if
  subproblems have few solutions
• Interleaved
• propagation
• cut generation
• Master/sub problem
• Column generation Benders decomposition
  Lagrangian relaxation

50
Transform

• Global constraints
• Dual forms
• Manageable constraints
• Separable subproblems

51
Transformed perfect rectangle problem
Idea Use decomposition into two scheduling
problems
Structured Data Task (var 0..100 start
PosInt duration, PosInt resource) list of
Task xtasks,ytasks (start_,durations,resour
ces) s
in 1,4,7,8,9,10,14,15,18 Constraints cumulati
ve(xtasks,33) cumulative(ytasks,32)
52
Tighten

• By programmer
• Experience
• Maths
• Automatically
• Propagation
• Intelligent methods

53
Tightened perfect rectangle problem
Redundant constraints Each row must be completely
covered onRow(Squares, int r) s.x lt r /\
r lt s.x s.size forall(p in 1..32) sum(s in
Squares) (s.size holds(onRow(s,p))) 33
54
Link

• Channeling constraints
• Communicating intermediate solutions

55
Conceptual Model Transformations
ConceptualModel
ConceptualModel
Decompose
Transform Add Inference
Specify Subproblem Interaction
Research question Devise transformation
primitives and language
56
Overview

• Introduction
• Three Optimisation Paradigms
• Mapping Conceptual Models to Design Models
• Conclusion

57
Research Aims

• Separate modelling from behaviour
• Support an extensive toolset
• Representations
• Algorithms
• Support experimentation
• Representations
• Algorithms
• Hybrids

58
Constraint Programming an environment for
problem solving
Model
Algorithm
Finite Domain Library
Repair Library
Interval Reasoning Library
Linear Programming Library
CPLEX
Xpress-MP
59
NICTA Constraint Programming Platform G12

• Richer Modelling
• Zinc
• Richer Mapping
• Cadmium
• Richer Solving
• Mercury
• Richer Environment
• G12

5 years 5M
60
Local potential applications of CP

• Internet Routing and Balancing
• Cisco
• Supply Chain
• Patrick Corp
• Hi Fert
• Coles Myers
• Transportation
• Fonterra
• IBM
• CTI
• NAB
• Scheduling
• Synctec
• Bovis Lend Lease
• Chartered semiconductor
• Health Systems