Chapter 3 Solving Problems By Searching and Constraint Satisfaction Problem

1
Chapter 3Solving Problems By Searching
andConstraint Satisfaction Problem
cs570 Artificial Intelligence

• 2000. 3. 20.
• ??? ???

2
Overview

• Solving Problem by Searching
• Problem solving agents
• Problem types
• Problem formulation
• Example problems
• Basic search algorithms
• Constraint Satisfaction Problem
• Introduction - Scene Labeling Problem
• Solving Techniques
• Applications

3
Problem-Solving Agents

• Problem-Solving Agents
• one kind of goal-based agent
• finding sequences of actions that lead to
  desirable states.
• Steps
• Goal Formulation
• limiting the objectives
• Problem Formulation
• deciding what actions and states to consider
• Search
• looking for the possible action sequence
• Execution

4
Formulating Problem - Example

• The eight possible states of the simplified
  vacuum world

1
2
3
4
5
6
7
8
5
Knowledge and Problem Types (1)
Formulating Problems

• Single-state problem
• accessible - the agents sensor knows
• which state it is in.
• deterministic - the agent knows
• what each of its actions does
• Action sequence can be completely planned.
• example Right, Suck

5
6
8
6
Knowledge and Problem Types (2)
Formulating Problems

• Multiple-state problem
• inaccessible
• limited access to the world state
• deterministic
• The agent must reason about sets of states that
  it mignt get to.
• example Right, Suck, Left, Suck

7
Knowledge and Problem Types (3)
Formulating Problems

• Contingency problem
• inaccessible
• non-deterministic
• sensing during the execution phase.
• most of the real, physical world problems
• Keep your eyes open while walking!
• The agent must calculate a whole tree of actions,
  rather than a single action sequence.

1
3
8
Knowledge and Problem Types (4)
Formulating Problems

• Exploration problem
• unknown state space (no map, no sensor)
• The agent must experiment, gradually discovering
  what its actions do and what sorts of states
  exist.

9
Well-defined Problems and Solutions
single-state problem
Formulating Problems

• Problem
• a collection of information that the agent will
  use to decide what to do
• the basic elements of a problem definition
• initial state
• operator (or successor function S)
• goal test
• path cost function

path
state space
10
Problem-Solving Agents - Example

• A simplified road map of Romania

Oradea
Neamt
Iasi
Zerind
Arad
Vaslui
Fagaras
Sibiu
Rimnicu Vilcea
Timisoara
Urziceni
Hirsova
Pitesti
Lugoj
Bucharest
Mehadia
Eforie
Dobreta
Giurgiu
Craiova
11
Problem-Solving Agents Example

• Situation
• On holiday in Romania currently in Arad.
• Flight leaves tomorrow from Bucharest.
• Formulate goal
• be in Bucharest
• Formulate problem
• initial state be in Arad
• state various cities
• operators driving between cities
• Find solution
• sequence of cities, e.g., Arad, Sibiu, Fagaras,
  Bucharest

Abstraction
12
Measuring Problem-Solving Performance
Formulating Problems

• Effectiveness of a search
• Does it find a solution at all?
• Is it a good solution (one with low path cost)?
• What is the search cost associated with the time
  and memory required to find a solution?
• total cost path cost search cost

13
Example Problems

• Toy Problems
• concise and exact description
• abstract version of real problem
• Real-World Problem
• no single agreed-upon description

14
The 8-puzzle
Toy Problems

• problem formulation
• State the location of each of the eight tiles
  in one of the nine squares
• Operators blank moves left, right, up, or down
• Goal test state matches the right figure
• Path cost each step costs 1, that is the length
  of the path

5 4
6 1 8
7 3 2
1 2 3
8 4
7 6 5
15
The 8-queens problem
Toy Problems

• problem formulation
• State any arrangement of 0 to 8 queens on board
• Operators add a queen to any square
• Goal test 8 queens on board, none attacked
• Path cost zero

16
Cryptarithmetic
Toy Problems

• problem formulation
• State a cryptarithmetic puzzle with some
  letters replaced by digits
• Operators replace all occurrences of a letter
  with a digit not already appearing in the puzzle
• Goal test puzzle contains only digits, and
  represents a correct sum
• Path cost zero

FORTY Solution 29786 F2, O9, R7, etc.
TEN 850 TEN 850 ------------ --
------- SIXTY 31486
17
The vaccum world
Toy Problems

• problem formulation
• State one of the eight states shown in Figure
• Operators move left, move right, suck
• Goal test no dirt left in any square
• Path cost each action costs 1

18
Missionaries and cannibals
Toy Problems

• problem formulation
• State 3 missionaries and 3 cannibals in the
  either side of the river
• Operators either 1 missionary, 1 cannibal, 2
  missionaries, 2 cannibals, or one of each across
  in the boat.
• Goal test 3 missionaries and 3 cannibals in the
  other side of the river
• Path cost the number of crossing

19
Example Real-World Problems

• Route finding
• Touring and travelling salesperson problems
• VLSI layout
• Robot navigation
• Assembly sequencing

20
Searching for Solutions

• Partial search tree for route finding from Arad
  to Bucharest.

goal test
Arad
(a) The initial state (search node)
Arad
generating a new state
(b) After expanding Arad
Sibiu
Timisoara
Zerind
choosing one option
Arad
(c) After expanding Sibiu
Sibiu
Timisoara
Zerind
Rimnicu Vilcea
Sibiu
Timisoara
Oradea
21
Search Strategies

• Criteria
• Completeness
• Time complexity
• Space complexity
• Optimality
• Classification
• Uninformed search ( blind search)
• have no information about the number of steps or
  the path cost from the current state to the goal
• Informed search ( heuristic search)
• have some information
• example Bucharest is southeast of Arad.

22
Breadth-first Search
Searching Strategies

• Properties
• Complete Yes (if b is finite)
• Time complexity 1bb2bd O(bd)
• Space complexity O(bd) (keeps every node in
  memory)
• Optimal Yes (if cost1 per step) not optimal in
  general
• where b is branching factor and
• d is the depth of the search tree

23
Uniform cost Search
Searching Strategies

• Expand least-cost unexpanded node
• the breadth-first search is just uniform cost
  search with g(n)DEPTH(n)

A
1
10
B
5
5
G
S
15
5
C
S
S
S
S
0
A
A
B
B
C
C
A
B
C
15
5
15
G
G
G
1
5
15
11
11
10
24
Uniform cost Search (2)
Searching Strategies

• Properties of Depth-first Search
• Complete Yes, if step cost gt e (epsilon)
• Time complexity of nodes with g lt cost of
  optimal solution, O(bd)
• Space complexity of nodes with g lt cost of
  optimal solution, O(bd)
• Optimal Yes, if step cost gt e (epsilon)

25
Depth-first Search
Searching Strategies
26
Depth-first Search (2)
Searching Strategies

• Properties
• Complete No
• fails in infinite-depth spaces, spaces with loops
• Modify to avoid repeated states along path
• gt complete in finite spaces
• Time complexity O(bm)
• where m is the maximum depth
• Space complexity O(bm)
• i.e., linear space
• Optimal No

27
Depth-limited Search
Searching Strategies

• Depth-first search with depth limit l
• Properties
• Complete Yes
• Time complexity O(bl)
• where l is the depth limit
• Space complexity O(bl)
• Optimal No

28
Iterative Deepening Search
Searching Strategies

• Limit0
• Limit1
• Limit2
• Limit3

29
Iterative Deepening Search (2)
Searching Strategies

• Properties
• Complete Yes
• Time complexity (d1)b0db(d-1)b21bd
  O(bl)
• Space complexity O(bd)
• Optimal Yes, if step cost 1
• Can be modified to explore uniform-cost tree

30
Bidirectional Search
Searching Strategies

• Simultaneously search both forward from the
  initial state and backward from the goal, and
  stop when the two searches meet in the middle.
• Operators are reversible.

31
Bidirectional Search (2)
Searching Strategies

• Properties
• Complete Yes
• Time complexity O(b(d/2))
• Space complexity O(b(d/2))
• Optimal Yes, if step cost 1
• Can be modified to explore uniform-cost tree

32
Comparison Search Strategies
Searching Strategies

• Evaluation of search strategies.
• b is the branching factor
• d is the depth of solution
• m is the maximum depth of the search tree
• l is the depth limit.

Criterion Breadth-First Uniform-Cost Depth-First Depth-Limited Iterative Deepening Bidirectional (if applicable)
Time bd bd bm bl bd b(d/2)
Space bd bd bm bl bd b(d/2)
Optimal? Yes Yes No No Yes Yes
Complete? Yes Yes No Yes, if lgtd Yes Yes
33
Avoiding Repeated States

• Do not return to the state you just came from.
• Do not create paths with cycles in them.
• Do not generate any state that was ever generated
  before.

34
Overview CSP

• Constraint Satisfaction Problem
• Introduction - Scene Labeling Problem
• Solving Techniques
• Consistency techniques
• Search
• Example
• Applications
• References

35
Constraint Satisfaction Problem

• CSP is defined as
• a set of variables X x1 , x2 , x3 xn
• for each variable, xi , a finite domain Di
• a set of constraints restricting the values that
  the variables can simultaneously take

36
The Solution of CSP

• An assignment of a value from its domain to every
  variable
• We may want to find
• All solutions
• Just one solution
• An optimal, or at least a good solution

37
Scene labeling problem

• Probably the first Constraint Satisfaction
  Problem, Walts
• Goal
• Recognize the object in a 3D scene by
  interpreting lines the the 2D drawings
• The lines or edges must be categorized into few
  types, convex(), concave(-), occluding edges(lt)

38
(No Transcript)
39
Logic programming

• Logic programming is just a particular kind of
  constraint programming
• Gallaire, Jaffar Lassez
• Constraint Logic Programming

40
Solving Technology

• Consistency Techniques
• Systematic Search
• Generate and Test (GT)
• Smart generator
• Generator is merged with tester

41
Consistency Techniques

• Removing inconsistent values from variable domain
• Graph representation - constraint graph
• Consistency Techniques are not complete

Consistent pairs of values
removed by AC
a b c
a b c
Vi
Vj
42
Systematic Search (1)

• Generator merged with tester
• Look back schema
• Backtracking, Backjumping, Backmarking,
  Backchecking
• Look ahead schema
• Forward checking (FC) , Partial Look Ahead (PLA)

43
Backtracking
Look Back Schema

• Incrementally extending a partial solution
• When conflict occurs, choose another value for
  inconsistent variable
• Drawbacks
• Thrashing
• Redundant work
• Late detection of collision

44
Example - Backtracking
Look Back Schema
Q
Q
Q
Q
Q
45
Backjumping
Look Back Schema

• Avoid trashing
• most recent conflicting variable instead of
  immediate past variable

46
Backchecking, Backmarking
Look Back Schema

• Avoid redundant work

47
Look Ahead Schema

• Prevent future conflict
• Examples of Look Ahead Schema
• Forward Checking
• When an assignment, X/a occurs, temporally delete
  all values from other variable domain
• Partial Look Ahead

48
Example - Look Ahead Schema
49
not yet instantiated variables
already instantiated variables
partial look ahead full look ahead
checked by backtracking
forward checking
50
Systematic Search (2)

• Smart generator
• Heuristic Method
• Hill-climbing
• Min-conflict (MC)
• Random walk
• Tabu search

51
Application

• Traditional Operational Research (OR)
• Planning
• Scheduling
• Optimization
• NP-Hard

52
References

• Stuart Russell, Peter Norvig Artificial
  Intelligence a Mordern Approach
• Walts, D.L. Understanding line drawings of
  scenes with shadow
• Gallaire H., Logic Programming Further
  Developments
• Jaffar, J. Lassez J.L. Constraint Logic
  Programming
• Tsang, E. Foundations of Constraint Satisfaction
• Montanary, U. Networks of constraints
  fundamental properties and application to picture
  processing
• Nadal, B. Tree Search and Arc Consistency in
  Constraint Satisfaction Algorithms
• Gaschnig, J. Performance Measurement and Analysis
  of Certain Search Algorithms
• Haralick, R.M., Elliot, G.L. Incresing tree
  search efficiency for constraint satisfaction
  problems