CSINFO 372: Explorations in Artificial Intelligence

1
CS-INFO 372Explorations in Artificial
Intelligence

• Prof. Carla P. Gomes
• gomes_at_cs.cornell.edu
• Module 2
• Examples of Different Modeling Formalisms
• http//www.cs.cornell.edu/courses/cs372/2008sp

2
Example of a reasoning formalismConstraint
Satisfaction Problems
3
EscherWaterfall, 1961
4
EscherBelvedere, May 1958
5
EscherAscending and Descending, 1960
6
How do we Interpret the Scenes in Eschers
Worlds?

• Analysis of Polyhedral Scenesorigins of
  Constraint Reasoning ?
  
• researchers in computer vision in the 60s-70s
  were
  
• interested in developing a procedure to assign
  3-
  
• dimensional interpretations to scenes

They identified Three types of edges Four
types of junctions

7
Edge Types

• Hidden if one of its planes cannot be seen
• represented with arrows
• ?
• Convex from the viewers perspective
• represented with
• Concave from the viewers perspective
• represented with
• -

Huffman-ClowesLabeling
8
Types of Junctions
Type of junction L Fork
T Arrow
9
Scene InterpretationConstraint Reasoning Problem

• Variables ? Edges
• Domains ? ,-,?,?
• Constraints
• 1- The different type junctions define
  constraints
  
• L, Fork, T, Arrow
• L (?, ?) , ( ?, ?), (, ?), (?,), (-, ?),
  (?,-)
  
• Fork (,,), (-,-,-), (?,?,-),
  (?,-,?),(-,?,?)

L(A,B) ? the pair of values assigned to variables
A,B
has to belong in the set L
Fork(A,B,C) ? the trio of values assigned to
variables A,B,C has to belong in the set Fork

10
Constraint Satisfaction Problem (CSP)

• T (?, ?, ?) , ( ?,?,?), (?,?,), (??,-)
• Arrow (?,?,), (,,-), (-,-,)

T(A,B,C) ? the trio of values assigned to
variables A,B,C
has to belong in the set T
Arrow(A,B,C) ? the trio of values assigned to
variables A,B,C has to belong in the set Arro
w
2- For each edge XY its reverse YX has a
compatible value
Edge ,), (-,-), (?,?),(?,?)
Edge(A,B) ? the pair of values assigned to
variables A,B
has to belong in the set Edge
11
CSP Model - Cube
How to label the cube?
12
Constraint Satisfaction Problem (CSP Model)

• Variables Edges AB, BA,AC,CA,AE,EA,CD,
• DC,BD,DB,DG,GD,GF,FG,EF,FE,AE,EA
• Domains ? ,-,?,?
• Constraints
• L(AC,CD) L(AE,EF) L(DG,GF)
• Arrow(AC,AE,AB) Arrow(EF,FG,BF)
  Arrow(CD,DG,DB)
  
• Fork(AB,BF,BD)
• Edge(AB,BA) Edge(AC,CA) Edge(AE,EA)
• Edge(EF,FE) Edge(BF,FB) Edge(FG,GF)
• Edge(CD,DC) Edge(BD,DB) Edge(DG,GD)

13
CSP Model

• Variables Edges AB, BA,AC,CA,AE,EA,CD,
• DC,BD,DB,DG,GD,GF,FG,EF,FE,AE,EA
• Domains ? ,-,?,?
• Constraints
• L(AC,CD) L(AE,EF) L(DG,GF)
• Arrow(AC,AE,AB) Arrow(EF,FG,BF)
  Arrow(CD,DG,DB)
  
• Fork(AB,BF,BD)
• Edge(AB,BA) Edge(AC,CA) Edge(AE,EA)
• Edge(EF,FE) Edge(BF,FB) Edge(FG,GF)
• Edge(CD,DC) Edge(BD,DB) Edge(DG,GD)

(upper right corner)
One (out of four) possible labelings
14
The Impossible Objects is Eschers Worlds
Penrose Penrose Stairs
Penrose Triangle
15
Impossible ObjectsNo labeling!
16
Other examples using a Constraint Satisfaction
formalism
17
Sudoku
Constraint Satisfaction Problem (CSP)
(but also Satisfiability and Integer Programming)
9 55 3x 10 52 possible completions
18

Latin Squares
Given an N X N matrix, and given N colors, a
Latin Square of order N is a a colored matrix,
such that -all cells are colored. - each c
olor occurs exactly once in each row.
- each color occurs exactly once in each
column.
Quasigroup or Latin Square (Order 4)
Constraint Satisfaction Problem (CSP)
and Satisfiability and Integer Programming
19

Latin Squares
Given an N X N matrix, and given N colors, a
Latin Square of order N is a a colored matrix,
such that -all cells are colored -a color
is not repeated in a row -a color is not repeat
ed in a column
Quasigroup or Latin Square (Order 4)
Constraint Satisfaction Problem (CSP)
and Satisfiability and Integer Programming
20
Latin Square Completion Problem
Given a partial assignment of colors (10 colors
in this case), can the partial latin square be
completed so we obtain a full Latin square?
Example
32 preassignment
10 68 possible completions
21
Fiber Optic Networks
Nodes connect point to point fiber optic links

• Wavelength
• Division
• Multiplexing (WDM)
• the most promising
• technology for the
• next generation of
• wide-area
• backbone networks.

22
Routing in Fiber Optic Networks
Input Ports
Output Ports
1
1
2
2
3
3
4
4
Routing Node
How can we achieve conflict-free routing in each
node of the network?

Dynamic wavelength routing is an NP-hard problem.
23
LSCP Application Example Routers in Fiber Optic
Networks
Dynamic wavelength routing in Fiber Optic
Networks can be directly mapped into the Latin S
quare Completion Problem.

24
Design of Statistical Experiments

• We have 5 treatments for growing beans. We want
  to know what treatments are effective in
  increasing yield, and by how much.
  
• The objective is to eliminate bias and distribute
  the treatments somewhat evenly over the test
  plot.
  
• Latin Square Analysis of Variance

A D E B C
C B A E D D
C B A E
E A C D B B
E D C A
() Already in use in this sub-plot
25
Spatially Balanced Latin Squares
Really hard to build balanced LSs
26
Timetabling Constraint Satisfaction Problem
(CSP) and Integer Programming
The problem of generating schedules with complex

constraints (in this case for sports teams).
28 28 3.3 x 1040 possibilities
27
Sports Scheduling

• Big Business!
• US National TV pays 500 million / year for
  baseball
  
• College basketball conferences get up to 30
  million
  
• Manchester United has (had) a market cap of 400
  million
  
• No rights holder wants to pay those sums and then
  get a bad schedule
  
• Difficult to automate
• Huge variety of problem types
• Small instances are difficult
• Strong break between easy/hard (for all
  algorithms)
  
• Significant theoretical background
• CP and IP differ in modeling
• CP has clean models with 1..n variables
• IP uses 0-1 variables reasonably naturally
• Practical interest in instances at the easy/hard
  interface

Source Mike Trick
28
Graph Coloring
nn possible colorings for n nodes
Coloring the nodes of the graph
Whats the minimum number of colors such that
any two nodes connected by an edge h
ave different colors?
29
Graph Coloring
CSP Variables ? Nodes Domains ? Colors Constr
aints ? Edges

Can we color a graph such that no two nodes conn
ected by an edge have the
same color?
Graph coloring formulations can be used to sol
ve different problems.
Constraint Satisfaction Problem (CSP)
and Satisfiability and Integer Programming
30
Scheduling of Final Exams

• How can the final exams at Cornell be scheduled
  so that no student has
  
• two exams at the same time? (Note not obvious
  this has anything to do
  
• with graphs or graph coloring.)

31
Scheduling of Final Exams
1
2
7
3
6
5
4
What are the constraints between courses?
Find a valid coloring
32
AI PLANNING
In AI, planning involves the generation of an
action plan (i.e. a sequence of actions) for an
agent, such as a robot or a software system or a
living artefact, that can alter
its surroundings.

• Planning implies the notion of synthesis
  synthesis of actions, to go from an initial state
  to a goal state.
  
• Examples
• plan to perform astronomical observations for the
  Hubble space telescope
  
• plan for a robot to assemble pieces in a factory

33
Planning Example Blocks world

• objects blocks and a table
• actions move blocks on one object to on
  another object
  
• goals configurations of blocks
• plan sequence of actions to achieve goals

B
A
D
C
T
Initial State
Goal State
34
Blocks world propositional and first order
logic representation

• Knowledge Base
• On(A,T)On(B,T)On(C,T)On(D,C)
• Block(A)Block(B)Block(C)Block(D) )Table(T)
• Clear(A)Clear(B)Clear(D)

D
A
B
C
T
Move(A,T,D)
KB On(A,D)On(B,T)On(C,T)On(D,C) Block(A)Bl
ock(B)Block(C)Block(D)Table(T)
Clear(A) Clear(B)
A
D
B
C
T
35
Another example of a reasoning formalismA
restricted form of Constraint Satisfaction
Satisfiability
36
Propositional Satisfiability problem

• Satifiability (SAT) Given a formula in
  propositional calculus, is there
  
• an assignment to its variables making it true?
• We consider clausal form, e.g.
• ( a OR NOT b OR NOT c ) AND ( b OR
  NOT c) AND ( a OR c)

possible assignments
SAT prototypical hard combinatorial search and
reasoning problem. Problem is NP-Complete. (Cook
1971) Surprising power of SAT for encoding comp
utational problems.
37
Significant progress in Satisfiability Methods
Software and hardware verification complete
methods are critical - e.g. for verifying the
correctness of chip design, using SAT encodings
Applications Hardware and Software Verificati
on
Planning, Protocol Design, etc.

Going from 50 variable, 200 constraints
to 1,000,000 variables and 5,000,000
constraints
in the last 10 years
Current methods can verify automatically the
correctness of 1/7 of a Pentium IV.
38
A real world example
39
Bounded Model Checking instance

i.e. ((not x1) or x7) and ((not x1) or x
6)
and etc.
40
10 pages later


(x177 or x169 or x161 or x153
or x17 or x9 or x1 or (not
x185)) clauses / constraints are getting more
interesting
41
4000 pages later

!!! a 59-cnf clause

42
Finally, 15,000 pages later
Note that
!!!
The Chaff SAT solver solves this instance in les
s than one minute.
43
Another example of a reasoning formalismInteger
Programming
44
Knapsack Problem (one resource)

• A hiker trying to fill her knapsack to maximum
  total value. Each item she considers taking with
  
  
• her has a certain value and a certain weight.
  Goal maximize the value of the contents of the
  
  
• knapsack considering the overall weight
  constraint.
  
• This problem is an abstraction with many
  practical applications
  
• Project selection and capital budgeting
  allocation problems
  
• Storing a warehouse to maximum value given the
  indivisibility of goods and space limitations
  
• Sub-problem of other problems e.g., generation
  of columns for a given model in the course of
  optimization cutting stock problem (beyond the
  scope of this course)

45
Capital Budgeting Example
Investment budget 14,000
maximize 16x1 22x2 12x3 8x4 11x5 19x6
subject to 5x1 7x2 4x3 3x4 4x5 6x6 ?
14
xj binary for j 1 to 6
46
Binary Optimization Applications in Regional
Planning
Zevi Azzaino Jon Conrad Carla Gomes
Stream Footage Phosphorous Pathogen Parcel Size

Parcel Value Budget Constraint
Town of Skaneateles -1834 parcels -12341 acres
52 land use class.
Riparian Buffer in the Skaneateles Lake Watershe
d

Objective Identify the best collection
of parcels to include in a riparian buffer subj
ect to a budget constraint
Contribution to a scenic landscape
or agricultural setting Historic significance
Budget 2 million Max of 25,000 grant per
barn Office of Parks, Recreation and Historic
Preservation
Important Natural Community Geological Importance

Aesthetic/Cultural Qualities Budget Constraint
2,345 barns registered in year 2000
464 barns in Finger Lakes Region only.

Unique Natural Areas in Tompkins County
Preservation in NY State
47
Southwestern Airways Crew Scheduling

• Southwestern Airways needs to assign crews to
  cover all its upcoming flights.
  
• Simple example ? assigning 3 crews based in San
  Francisco (SFO) to 11 flights.
  
• Question How should the 3 crews be assigned 3
  sequences of flights so that every one of the 11
  flights is covered?

48
Other Integer Programming problems
49
Southwestern Airways Flights
50
Data for the Southwestern Airways Problem
51
Algebraic Formulation
Let xj 1 if flight sequence (paring) j is
assigned to a crew 0 otherwise. (j 1, 2, ,
12).Minimize Cost 2x1 3x2 4x3 6x4 7x5
5x6 7x7 8x8 9x9 9x10 8x11 9x12 (in
thousands)subject to Flight 1 covered x1 x4
x7 x10 1 Flight 2 covered x2 x5 x8
x11 1 Flight 11 covered x6 x9 x10
x11 x12 1 Three Crews x1 x2 x3 x4
x5 x6 x7 x8 x9 x10 x11 x12
3 and xj are binary (j 1, 2, , 12).
pairings
52
Combinatorial Problems
53
Combinatorial Problems

• Many computational tasks, such as planning or
  scheduling, can in principle be reduced to an
  exploration of a large set of all possible
  scenarios.
• Try all possible schedules, try all possible
  plans, pick the best.
  

Problem combinatorial explosion!
54
Planning Complexity
Planning (single-agent) find the right
sequence of actions
HARD 10 actions, 10! 3 x 106 possible plans
100 ! 9.33262154 10157
Contingency planning (multi-agent)
actions may or may not produce the desired
effect!
4 outof 8
2 outof 9
1 outof 10
REALLY HARD 10 x 92 x 84 x 78 x x 2256
10224 possible contingency plans!
55
Nice Problems ?!

• Nice combinatorial problem (Shortest Path)
  exception to combinatorial explosition ?
  polynomial scaling ?!
  
• General formulation for special problems
• shortest paths
• transportation problem
• assignment problem
• plus more
• Important subproblem of many optimization
  problems, including multicommodity flows

56
(No Transcript)
57
But most interesting real-world problems are
NP-Complete and NP-Hard Problems
Planning and Scheduling And Supply Chain Manageme
nt
Data Analysis Data Mining
Protein Folding And Medical Applications
Capital Budgeting And Financial Appl.
Information Retrieval
Combinatorial Auctions
Software Hardware Verification
Many more applications!!!
58
Goals of INFO 372

• Introduce the students to a range of
  computational
  
• modeling approaches and solution strategies
  using
  
• examples from AI and Information Science.
• Formalisms
• Logical representations
• Constraint-based languages,
• Mathematical programming Linear and Integer
  programming
  
• Multi-agent formalisms (including adversarial
  games)
  
• Solution strategies
• Logical inference
• General complete backtrack search (e.g.,
  Iterative Deepening)
  
• Local search
• Dynamic Programming
• Game tree search (e.g., alpha-beta pruning)

59
Goals of INFO 372

• Special models
• Satisfiability (SAT) Maximum SAT Horn
• Constraint Satisfaction Binary Constraint
  Satisfaction
  
• Mixed Integer Programming, Linear Programming
  and
  
• Network Flow Models

Themes Expressiveness and efficiency tradeoffs
of the various representation formalisms
?Students learn about the tradeoffs in modeling
choices. Concrete examples to move from one rep
resentation modeling formalism to another
formalism