Single Agent Search With Inconsistent Heuristics

1
Single Agent Search With Inconsistent Heuristics

• presented by Zhifu Zhang

2
Outline

• Introduction
• Path-finding and Heuristics
• Consistent and Inconsistent Heuristics
• Inconsistent Heuristics and A
• Emergence of Inconsistent Heuristics
• Recent New Techniques BPMX and Delay
• Literature in Perspective
• As New Worst-case Result in A Class of
  Applications
• New Algorithm DP
• Comparing A and Its Variants with Inconsistent
  Heuristics
• Conclusion

3
Outline

• Introduction
• Path-finding and Heuristics
• Consistent and Inconsistent Heuristics
• Inconsistent Heuristics and A
• Emergence of Inconsistent Heuristics
• Recent New Techniques BPMX and Delay
• Literature in Perspective
• As New Worst-case Result in A Class of
  Applications
• New Algorithm DP
• Comparing A and Its Variants with Inconsistent
  Heuristics
• Conclusion

4
Path-finding Applications
5
Path-finding Applications
6
Path-finding Applications

• Planning
• Genome sequence alignment

7
Outline

• Introduction
• Path-finding and Heuristics
• Consistent and Inconsistent Heuristics
• Inconsistent Heuristics and A
• Emergence of Inconsistent Heuristics
• Recent New Techniques BPMX and Delay
• Literature in Perspective
• As New Worst-case Result in A Class of
  Applications
• New Algorithm DP
• Comparing A and Its Variants with Inconsistent
  Heuristics
• Conclusion

8
Dijkstras algorithm

• Maintains two lists OPEN CLOSED.
• Each node stores an estimate of the minimum path
  cost (g) from the start node, and the
  corresponding parent node.
• In each iteration, extract the node from OPEN
  with minimum g, and propagate its g value to the
  successors, trying to reduce their g estimate.
  Then put it into CLOSED.
• If a closed nodes g is reduced, move to OPEN.
• Iterate until the goal is found.

9
Heuristics

• A heuristic is a function that gives each node an
  estimate of how far away the goal is.
• An admissible heuristic is a heuristic that never
  over-estimates the cost.

10
A Algorithm

• Heuristic search algorithms uses heuristics to
  guide the search. Denoted h(n).
• The most famous heuristic algorithm is A.
• A is very similar to Dijkstras algorithm.
  Instead of extracting the node with minimum g
  from OPEN, it extracts the node with minimum f.
• f g h.

11
How Heuristics Guide the Search
12
Outline

• Introduction
• Path-finding and Heuristics
• Consistent and Inconsistent Heuristics
• Inconsistent Heuristics and A
• Emergence of Inconsistent Heuristics
• Recent New Techniques BPMX and Delay
• Literature in Perspective
• As New Worst-case Result in A Class of
  Applications
• New Algorithm DP
• Comparing A and Its Variants with Inconsistent
  Heuristics
• Conclusion

13
Consistent and Inconsistent Heuristics

• When using an admissible heuristic, A guarantees
  finding the minimum-cost path.
• Admissible heuristics can be divided into
  consistent heuristics and inconsistent heuristics.

14
Consistent Heuristics
n
m
Goal

• For two nodes m and n, denote the optimal
  distance from m to n as dist(m,n).
• If for any two nodes m and n,
• h(m) h(n) dist(m,n), then h is consistent.
• Otherwise h is inconsistent.
• Using a consistent heuristic, A expands each
  node at most once.

15
Outline

• Introduction
• Path-finding and Heuristics
• Consistent and Inconsistent Heuristics
• Inconsistent Heuristics and A
• Emergence of Inconsistent Heuristics
• Recent New Techniques BPMX and Delay
• Literature in Perspective
• As New Worst-case Result in A Class of
  Applications
• New Algorithm DP
• Comparing A and Its Variants with Inconsistent
  Heuristics
• Conclusion

16
Inconsistent Heuristics and A

• Martelli has constructed a graph family in which
  the number of node expansions by A is
  exponential of the number of distinct nodes
  expanded by A, which is also the graph size.

17
An Example
13
n4
6
1
4
7
10
1
0
0
3
1
goal
n3
n2
n1
3
18
An Example
13
g0 f13
n4
6
1
4
g1 f8
7
10
1
0
0
3
1
goal
n3
n2
n1
g4 f7
g6 f6
3
19
An Example
13
g0 f13
n4
6
1
4
g1 f8
7
10
1
0
0
3
1
goal
n3
n2
n1
g4 f7
g6 f6
g16 f16
3
20
An Example
13
g0 f13
n4
6
1
4
g1 f8
7
10
1
0
0
3
1
goal
n3
n2
n1
g4 f7
g5 f5
g16 f16
3
21
An Example
13
g0 f13
n4
6
1
4
g1 f8
7
10
1
0
0
3
1
goal
n3
n2
n1
g4 f7
g5 f5
g15 f15
3
22
An Example
13
g0 f13
n4
6
1
4
g1 f8
7
10
1
0
0
3
1
goal
n3
n2
n1
g2 f5
g4 f4
g15 f15
3
23
An Example
13
g0 f13
n4
6
1
4
g1 f8
7
10
1
0
0
3
1
goal
n3
n2
n1
g2 f5
g4 f4
g14 f14
3
24
Improvements to the Worst-Case Result

• Algorithms B and B are variants of A, which has
  O(N2) worst-case node expansions, where N is the
  number of distinct nodes expanded by A.

25
Algorithm B

• Maintains a global variable F, which is the
  maximum f of the expanded nodes.
• If there are nodes with f lt F in OPEN, then
  select among them the one with minimum g
  otherwise select the one with minimum f.

26
Algorithm B

• Based on B.
• Uses two pathmax rules to update the heuristic
  values during the search.
• If h(u) e(u,vi) gt h(vi) , then
• h(vi) ? h(u) e(u,vi)
• If h(u) lt mini(h(vi) e(u,vi)), then
• h(u) ? mini(h(vi) e(u, vi))

27
Pathmax Rules
u
u
v3
v1
v2
v1
Goal
Goal
28
Consequence

• Due to the worst-case complexity, inconsistent
  heuristics are avoided when using A-like
  algorithm.
• It is also believed that regular method will not
  produce inconsistent heuristics.

29
Outline

• Introduction
• Path-finding and Heuristics
• Consistent and Inconsistent Heuristics
• Inconsistent Heuristics and A
• Emergence of Inconsistent Heuristics
• Recent New Techniques BPMX and Delay
• Literature in Perspective
• As New Worst-case Result in A Class of
  Applications
• New Algorithm DP
• Comparing A and Its Variants with Inconsistent
  Heuristics
• Conclusion

30
Pattern Databases

• A pattern database (PDB) is a lookup table
  pre-computed by completely solving an abstraction
  of the original problem.
• State-of-the-art results have been reported in
  using PDBs to solve Rubik's Cube problem , the
  4-peg Towers of Hanoi problem and the sliding
  tile puzzles

31
PDBs and Inconsistent Heuristics

• Researchers summarized several ways where
  inconsistent heuristics may be produced
• Random selection of heuristics.
• Dual heuristics.
• Operators reversible
• Costs symmetric
• Object and location have 1-to-1 mapping
• Compressed PDBs.

32
Outline

• Introduction
• Path-finding and Heuristics
• Consistent and Inconsistent Heuristics
• Inconsistent Heuristics and A
• Emergence of Inconsistent Heuristics
• Recent New Techniques BPMX and Delay
• Literature in Perspective
• As New Worst-case Result in A Class of
  Applications
• New Algorithm DP
• Comparing A and Its Variants with Inconsistent
  Heuristics
• Conclusion

33
Recent New Techniques BPMX and Delay

• BPMX
• In undirected graphs, the role of parent and
  child can be interchanged, thus rule (a) can be
  used in both directions. Short for
  bi-directional pathmax
• Has been applied to IDA and got good results. In
  IDA, only rule (a) and its reverse is used.
• Delay
• Recently invented by Dr. Sturtevant.
• Aimed at limiting the re-expansions at most k
  re-expansions per new expansion.
• Proved to have worst-case O(N1.5) when ksqrt(N).

34
Outline

• Introduction
• Path-finding and Heuristics
• Consistent and Inconsistent Heuristics
• Inconsistent Heuristics and A
• Emergence of Inconsistent Heuristics
• Recent New Techniques BPMX and Delay
• Literature in Perspective
• As New Worst-case Result in A Class of
  Applications
• New Algorithm DP
• Comparing A and Its Variants with Inconsistent
  Heuristics
• Conclusion

35
The Pathmax Rules

• The rules stated in Meros paper
• If h(vi) lt h(u) e(u,vi), set
• h(vi) h(u) e(u,vi)
• If h(u) lt mini(h(u) e(u,vi)), set
• h(u) mini(h(u) e(u,vi))
• The second rule should be
• If h(u) lt mini(h(vi) e(u,vi)), set
• h(u) mini(h(vi) e(u,vi))

36
Interaction of the Two Rules
u
10
Rule (a) effective Rule (b) will be ineffictive
3
1
5
2
10
7
8
v3
v1
v2
37
Interaction of the Two Rules
u
10 11
Rule (b) effective Rule (a) will be ineffective
3
1
8
2
10
12
v3
v1
v2
38
Optimality of B compared to B

• Algorithms A, B and B requires that ties in f
  are broken arbitrary, but in favour of goal
  nodes.
• Mero claimed that For every graph, algorithm B
  expands each node at most as many times as
  algorithm B, if both algorithms resolve ties in
  the same way.
• Common tie-breaking rules given two nodes with
  equal f, decide using their g h.

39
A Simple Counter Example
7
n3
3
1
n1
Goal
0
3
n2
5
1
40
Execution of B
7
F7
n3
g0
3
1
n1
Goal
0
3
n2
5
1
g1 f4
g3 f3
41
Execution of B
7
F7
n3
g0
3
1
n1
Goal
0
3
n2
5
1
g1 f4
g2 f2
42
Execution of B
7
F7
n3
g0
3
1
n1
Goal
0
3
n2
5
1
g1 f4
g2 f2
g7 f7
43
Execution of B (favours larger g)
7
F7
n3
g0
3
1
n1
Goal
0 4
3 6
n2
5
1
g1 f7
g3 f7
44
Execution of B (favours larger g)
7
F7
n3
g0
3
1
n1
Goal
4 5
6
n2
5
1
g1 f7
g3 f8
g8 f8
45
Execution of B (favours larger g)
7
F7
n3
g0
3
1
n1
Goal
5
6
n2
5
1
g1 f7
g2 f7
g8 f8
46
Execution of B (favours larger g)
7
F7
n3
g0
3
1
n1
Goal
5
6
n2
5
1
g1 f7
g2 f7
g7 f7
47
Outline

• Introduction
• Path-finding and Heuristics
• Consistent and Inconsistent Heuristics
• Inconsistent Heuristics and A
• Emergence of Inconsistent Heuristics
• Recent New Techniques BPMX and Delay
• Literature in Perspective
• As New Worst-case Result in A Class of
  Applications
• New Algorithm DP
• Comparing A and Its Variants with Inconsistent
  Heuristics
• Conclusion

48
Some Definitions

• If the possible edge costs are a set of rational
  numbers, then the decrement of a nodes g value
  cannot be arbitrarily small.
• Define y being a factor of x if x my,
  where m is an integer.
• Define ? as the greatest common factor of the
  edge costs.
• Then any decrement of g must be an integer
  multiple of ?.

49
Lower Bound of the Max Heuristic Value

• If there are f(N) (gtO(N)) total node expansions,
  then there must be a node with at least (f(N) -
  N) / N re-expansions. (Pigeon-Hole)

50
Lower Bound of the Max Heuristic Value
with high h value
S
B
K
with many re-expansions
L2
L1

• K is first expanded via path L1, last expanded
  via L2.
• L2 is shorter than L1.
• B has max f on L2.

51
Lower Bound of the Max Heuristic Value
with high h value
S
B
K
with many re-expansions
L2
L1

• f(B) g(B) h(B) (4.1)
• f(B) fL(K) (4.2)
• fL(K) gL(K) h(K) (4.3)
• g(B) lt g(K) (4.4)
• gL(K) g(K) ? (f(N) - N) / N (4.5)

52
Lower Bound of the Max Heuristic Value

• f(B) g(B) h(B) (4.1)
• f(B) fL(K) (4.2)
• fL(K) gL(K) h(K) (4.3)
• g(B) lt g(K) (4.4)
• gL(K) g(K) ? (f(N) - N) / N (4.5)
• Thus,
• h(B) gL(K) h(K) g(B), by 4.1, 4.2, 4.3
  (4.6)
• gt gL(K) h(K) g(K), by 4.4
  (4.7)
• gt gL(K) g(K) , since h(K) 0
  (4.8)
• ? (f(N) - N) / N, by 4.5
  (4.9)

53
Lower Bound of the Optimal Solution

• The optimal solution cost is f(B).
• f(B) gt ? (f(N) - N) / N

54
Upper Bound of the Total Node Expansions

• If the edge costs are a fixed set of rational
  constants that do not change with the problem
  size, then the total number of node expansions is
  at most O(N2/?). Since
• g(goal) O(N)
• g(goal) O(? f(N) / N)
• Thus, f(N) O(N2 / ?)

55
Outline

• Introduction
• Path-finding and Heuristics
• Consistent and Inconsistent Heuristics
• Inconsistent Heuristics and A
• Emergence of Inconsistent Heuristics
• Recent New Techniques BPMX and Delay
• Literature in Perspective
• As New Worst-case Result in A Class of
  Applications
• New Algorithm DP
• Comparing A and Its Variants with Inconsistent
  Heuristics
• Conclusion

56
Motivation

• Numerous re-expansion makes inconsistent
  heuristics un-attractive.
• Re-expansions are caused by re-openings.
• A re-opening is a closed nodes g value being
  reduced by its neighbour.

57
Forward vs Backward g Propagation
58
Backward g Propagation
parent
g10 6
1
1
2
g5
g12 8
g11 7
v3
v1
v2
59
DP Algorithm

• Procedure DP(G,e,h,s,Goal)
• g(s) ? 0 CLOSED ? Ø OPEN ? s
• While (OPEN ? Ø) do
• u ? EXTRACT-MIN-F(OPEN)
• CLOSED ? CLOSED ? u
• If (u ? Goal) then
• return BUILD-PATH(G,u)
• For each node v ? Neighborsu do
• If (v ? (CLOSED ? OPEN) and g(u) gt g(v) e(v,u)
  ) then
• g(u) ? g(v) e(v,u) f(u) ? g(u) h(u)
  parent(u) ? v
• For each node v ? Neighborsu do
• If ( v ?(CLOSED ? OPEN) ) then
• g(v) ? g(u) e(u,v) f(v) ? g(v) h(v)
  parent(v) ? u OPEN ? OPEN ? v
• Else If ( g(v) gt g(u) e(u,v) ) then
• g(v) ? g(u) e(u,v) f(v) ? g(v) h(v) parent
  (v) ? u
• If (v ? CLOSED) then
• CLOSED ? CLOSED v OPEN ? OPEN ? V

60
An Example Problem
(A)
2
1
1
(C)
Goal
Start
0
0
g1 f3
0
1
0
1.5
1
(B)
g1 f1
61
An Example Problem
(A)
2
1
1
(C)
Goal
Start
0
0
g1 f3
0
1
g2.5 f2.5
0
1.5
1
(B)
g1 f1
62
An Example Problem
(A)
2
1
1
(C)
Goal
Start
0
0
g1 f3
0
1
g2 f2
g3 f3
0
1.5
1
(B)
g1 f1
63
Properties of DP

• There solution path may contain open nodes.
• If the heuristic is consistent, identical to A.
• Guarantees finding optimal path with an
  admissible heuristic. (admissibility)
• Proof is identical to that for A.

64
Outline

• Introduction
• Path-finding and Heuristics
• Consistent and Inconsistent Heuristics
• Inconsistent Heuristics and A
• Emergence of Inconsistent Heuristics
• Recent New Techniques BPMX and Delay
• Literature in Perspective
• As New Worst-case Result in A Class of
  Applications
• New Algorithm DP
• Comparing A and Its Variants with Inconsistent
  Heuristics
• Conclusion

65
Path-finding on Maps

• Path-finding on maps
• 8 directions of movement
• Octile distance sqrt(2)min(dx,dy) dx dy
• Use 116 512x512 maps
• Problem instances are grouped by optimal solution
  length, each group (bucket) has about 1160
  problems.
• We select 10 points on the map. Compute single
  source shortest path for each point.
• Random heuristic is equal in 75 , higher in 25
  of the nodes compared to octile.

66
Node Expansions
67
Time
68
Categorized Node Expansions (random g)
69
Categorized Node Expansions (smaller g)
70
Categorized Node Expansions (larger g)
71
(No Transcript)
72
Topspin

• Topspin domain
• Use (9,4) (14,4) Topspin, 1000 problems each.
• For each (T,K) puzzle, a ?T/2?-piece pattern
  database is built.
• Uses the normal lookup and dual lookup.
• Dual lookup is lower in 17, equal in 5, higher
  in 78 of the nodes, compared to normal lookup.

73
The dual lookup
74
Node Expansions
75
Time
76
Categorized Node Expansions (random g)
Algorithm C uses results on (13,4) topspin
77
Categorized Node Expansions (smaller g)
Due to lack of memory, these are the results on
(13,4) topspin
78
Categorized Node Expansions (larger g)
79
Conclusion

• Inconsistent heuristics can be used in practice.
• We made a few clarifications to the research
  literature.
• As worst-case result with inconsistent
  heuristics in many applications is lowered.
• A new variant of A called DP is introduced,
  which performs good in experiments on the maps.
• We presented the first comparison of the variants
  of A for inconsistent heuristics. Techniques
  like Delay, DP and BPMX appear to be promising in
  practice, and BPMX performs best in the
  experiments.
• When the branching factor is huge, BPMX may have
  the risk of dramatically increasing the
  re-expansions.