From AND/OR Search to AND/OR BDDs

1
From AND/OR Search to AND/OR BDDs

• Rina Dechter
• Information and Computer Science, UC-Irvine, and
• Radcliffe Institue of Advanced Study, Cambridge

Joint work with Robert Mateescu and John Cobb
2
Combining Decision Diagrams
g CG DH
f AE BF
OBDD f g
3
Combining AND/OR BDDs
g CG DH
f AE BF
AOBDD f g
4
OBDDs vs AOBDD
f g
AOBDD
OBDD
5
Outline

• Background in Graphical models
• AND/OR search trees and Graphs
• Minimal AND/OR graphs
• From AND/OR graphs to AOMDDs
• Compilation of AOMDDs
• AOMDDs and earlier BDDs

6
Constraint Networks
A

• Example map coloring
• Variables - countries (A,B,C,etc.)
• Values - colors (red, green, blue)
• Constraints

Constraint graph
A
E
D
F
B
G
C
Semantics set of all solutions Primary task
find a solution
7
Propositional Satisfiability
? (C), (A v B v C), (A v B v E), (B v C v
D).
8
Graphical models

• A graphical model (X,D,C)
• X X1,Xn variables
• D D1, Dn domains
• C F1,,Ft functions(constraints, CPTS, cnfs)

• Examples
• Constraint networks
• Belief networks
• Cost networks
• Markov random fields
• Influence diagrams

• All these tasks are NP-hard
• ? identify special cases
• ? approximate

9
Tree-solving is easy
CSP consistency (projection-join)
Belief updating (sum-prod)
CSP (sum-prod)
MPE (max-prod)
Trees are processed in linear time and memory
10
Transforming into a Tree

• By Inference (thinking)
• Transform into a single, equivalent tree of
  sub-problems
• By Conditioning (guessing)
• Transform into many tree-like sub-problems.

11
Inference and Treewidth
ABC
DGF
G
D
A
B
BDEF
F
C
EFH
E
M
K
H
FHK
L
J
Inference algorithm Time exp(tree-width) Space
exp(tree-width)
HJ
KLM
treewidth 4 - 1 3 treewidth (maximum
cluster size) - 1
12
Conditioning and Cycle cutset
Cycle cutset A,B,C
13
Search over the Cutset

• Inference may require too much memory
• Condition (guessing) on some of the variables

14
Search over the Cutset (cont)

• Inference may require too much memory
• Condition on some of the variables

15
Inference vs. Conditioning

• By Inference (thinking)

Exponential in treewidth Time and memory
Variable-elimination Directional
resolution Join-tree clustering

• By Conditioning (guessing)

Exponential in cycle-cutset Time-wise, linear
memory
Backtracking Branch and bound Depth-first search
16
Outline

• Background in Graphical models
• AND/OR search trees and Graphs
• Minimal AND/OR graphs
• From AND/OR graphs to AOMDDs
• Compilation of AOMDDs
• AOMDDs and earlier BDDs

17
Classic OR Search Space
Ordering A B E C D F
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
18
AND/OR Search Space
A
F
B
C
D
E
Primal graph
DFS tree
19
AND/OR vs. OR
OR
A
AND
0
1
AND/OR
OR
B
B
AND
0
1
0
1
OR
E
C
E
C
E
C
E
C
AND
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
OR
D
F
D
F
D
F
D
F
D
F
D
F
D
F
D
F
AND
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
AND/OR size exp(4), OR size exp(6)
0
OR
1
1
1
0
1
20
AND/OR vs. ORwith Constraints
No-goods (A1,B1) (B0,C0)
AND
0
1
AND/OR
OR
B
B
AND
0
1
0
1
OR
E
C
E
C
E
C
E
C
AND
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
OR
D
F
D
F
D
F
D
F
D
F
D
F
D
F
D
F
AND
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
OR
21
AND/OR vs. ORwith Constraints
No-goods (A1,B1) (B0,C0
OR
A
AND
0
1
AND/OR
OR
B
B
AND
0
1
0
OR
E
C
E
C
E
C
AND
1
0
1
0
1
0
1
1
0
1
OR
D
F
D
F
D
F
D
F
AND
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
A
A
A
0
1
0
1
0
1
OR
B
B
B
0
1
0
0
1
0
0
1
0
E
E
E
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
C
C
C
1
1
0
1
0
1
1
1
1
1
0
1
0
1
1
1
1
1
0
1
0
1
1
1
D
D
D
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
F
F
F
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
22
DFS algorithm (CSP example)
A
0
B
4
0
OR node Marginalization operator (summation)
E
C
2
2
AND node Combination operator (product)
0
0
1
0
1
2
1
1
D
F
D
F
2
1
2
0
0
1
0
1
0
1
0
1
1
1
1
0
1
1
0
0
Value of node number of solutions below it
23
Pseudo-Trees (Freuder 85, Bayardo 95, Bodlaender
and Gilbert, 91)
1
4
6
1
2
3
2
7
5
3
m lt w log n
(a) Graph
4
7
1
1
2
7
5
3
5
3
5
2
7
4
6
6
4
6
(b) DFS tree depth3
(c) pseudo- tree depth2
(d) Chain depth6
24
Tasks and value of nodes

• V( n) is the value of the tree T(n) for the task
• Consistency v(n) is 0 if T(n) inconsistent, 1
  othewise.
• Counting v(n) is number of solutions in T(n)
• Optimization v(n) is the optimal solution in
  T(n)
• Belief updating v(n), probability of evidence in
  T(n).
• Partition function v(n) is the total
  probability in T(n).
• Goal compute the value of the root node
  recursively using dfs search of the AND/OR tree.
• AND/OR search tree and algorithms are
• (Freuder Quinn85, Collin, Dechter
  Katz91, Bayardo Miranker95,Darwiche 2001,
  Bacchus et. Al, 2003)
• Space O(n)
• Time O(exp(m)), where m is the depth of the
  pseudo-tree
• Time O(exp(w log n))
• BFS is time and space O(exp(w log n)

25
From AND/OR Tree
A
J
A
F
B
B
C
K
C
E
D
E
D
F
G
J
G
H
OR
A
H
K
AND
0
1
OR
B
B
AND
0
1
0
1
OR
E
C
E
C
E
C
E
C
AND
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
OR
D
F
D
F
D
F
D
F
D
F
D
F
D
F
D
F
AND
OR
AND
OR
AND
26
To an AND/ORGraph
A
J
A
F
B
B
C
K
C
E
D
E
D
F
G
J
G
H
OR
A
H
K
AND
0
1
OR
B
B
AND
0
1
0
1
OR
E
C
E
C
E
C
E
C
AND
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
OR
D
F
D
F
D
F
D
F
D
F
D
F
D
F
D
F
AND
0
1
0
1
OR
G
G
J
J
AND
0
1
0
1
0
1
0
1
OR
H
H
H
H
K
K
K
K
AND
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
27
AND/OR Search Graphs

• Any two nodes that root identical
  subtrees/subgraphs (are unifiable) can be merged
• Minimal AND/OR search graph of R relative to
  tree T is the closure under merge of the AND/OR
  search tree of R relative to T, where
  inconsistent subtrees are pruned.
• Canonicity The minimal AND/OR graph relative to
  T is unique for all constraints equivalent to R.

28
Context based caching

• Caching is possible when context is the same
• context parent-separator set in induced
  pseudo-graph current variable ancestors
  connected to subtree below

context(B) A, B context(C)
A,B,C context(D) D context(F) F
29
Caching
A
J
A
F
B
B
C
K
C
E
D
E
D
F
G
J
G
H
context(B) A, B context(C)
A,B,C context(D) D context(F) F
OR
A
H
K
AND
0
1
OR
B
B
AND
0
1
0
1
OR
E
C
E
C
E
C
E
C
AND
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
OR
D
F
D
F
D
F
D
F
D
F
D
F
D
F
D
F
AND
OR
AND
OR
AND
30
Caching
A
J
A
F
B
B
C
K
C
E
D
E
D
F
context(D)D context(F)F
G
J
G
H
OR
A
H
K
AND
0
1
OR
B
B
AND
0
1
0
1
OR
E
C
E
C
E
C
E
C
AND
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
OR
D
F
D
F
D
F
D
F
D
F
D
F
D
F
D
F
AND
0
1
0
1
OR
G
G
J
J
AND
0
1
0
1
0
1
0
1
OR
H
H
H
H
K
K
K
K
AND
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
31
All Four Search Spaces
Full OR search tree 126 nodes
Context minimal OR search graph 28 nodes
Context minimal AND/OR search graph 18 AND nodes
Full AND/OR search tree 54 AND nodes
32
Properties of minimal AND/OR graphs

• Complexity
• Minimal AND/OR R relative to pseudo-tree T is
  O(exp(w)) where w is the tree-width of R along
  T.
• Minimal OR search graph is O(exp(pw)) where pw
  is path-width
• w pw, pw wlog n
• Canonicity
• The minimal AND/OR search graph is unique
  (canonical) for all equivalent formulas (Boolean
  or Constraints), consistent with its pseudo tree.

33
Searching AND/OR Graphs

• AO(j) searches depth-first, cache i-context
• j the max size of a cache table (i.e. number of
  variables in a context)

i0
iw
j
Space O(exp w) Time O(exp w)
Space O(n) Time O(exp(w log n))
Space O(exp(j) ) Time O(exp(m_jj )
34
Context-Based Caching
context(A) A context(B) B,A context(C)
C,B context(D) D context(E)
E,A context(F) F
A
B
Primal graph
E
C
D
F
Cache Table (C)
B C Value
0 0 5
0 1 2
1 0 2
1 1 0
Space O(exp(2))
35
Outline

• Background in Graphical models
• AND/OR search trees and Graphs
• Minimal AND/OR graphs
• From AND/OR search graphs to AOMDDs
• Compilation of AOMDDs
• AOMDDs and earlier BDDs

36
OR Search Graphs vs OBDDs
A B C f(ABC)
0 0 0 0
0 0 1 0
0 1 0 0
0 1 1 1
1 0 0 0
1 0 1 1
1 1 0 0
1 1 1 1
A
B
C
redundant
Minimal AND/OR search graph
37
AND/OR Search Graphs AOBDDs
F(A,B,C,D) (0,1,1,1), (1,0,1,0), (1,1,1,0)
AOBDD(F)
B
redundancy
B
1
1
D
C
D
D
C
1
1
0
D
1
1
0
38
AND/OR Search Graphs AOBDDs
F(A,B,C,D) (0,1,1,1), (1,0,1,0), (1,1,1,0)
AOBDD(F)
B
redundancy
B
1
1
D
C
D
D
C
1
1
0
D
1
1
0
39
AOBDD Conventions
Point dead-ends to terminal 0 Point goods to
terminal 1
40
AOBDD Conventions
Introduce Meta-nodes
41
Combining AOBDD (apply)
A B C f(ABC)
0 0 0 0
0 0 1 0
0 1 0 0
0 1 1 1
1 0 0 0
1 0 1 1
1 1 0 0
1 1 1 1
A B D g(ABC)
0 0 0 0
0 0 1 0
0 1 0 0
0 1 1 1
1 0 0 1
1 0 1 0
1 1 0 1
1 1 1 0
A
A
A
B
B
B
C
D
D
C


42
Example
(fh) (a!h) (abg) (fg) (bf) (ae)
(ce) (cd) (bc)
m7
43
Example (continued)
44
AOBDD vs. OBDD
A
B
B
C
C
C
C
D
D
D
D
D
D
E
E
E
F
F
F
F
G
G
G
G
G
A
H
H
B
C
1
0
D
AOBDD 18 nonterminals 47 arcs
OBDD 27 nonterminals 54 arcs
E
F
G
H
45
Complexity

• Complexity of apply Complexity of apply is
  bounded quadratically by the product of input
  AOBDDs restricted to each branch in the output
  pseudo-tree.
• Complexity of VE-AOBDD is exponential in the
  tree-width along the pseudo-tree.

46
AOMDDs and tree-BDDs

• Tree-BDDs (McMillan1994) are
• AND/OR BDDS are

47
Related work

• Related work in Search
• Backjumping learning (Freuder 1988,Dechter
  1990,Bayardo and Mirankar 1996)
• Recursive-conditioning (Darwiche 2001)
• Value elimination (Bacchus et. Al. 2003)
• Search over tree-decompositions (Trrioux et. Al.
  2002)
• Related to compilation schemes
• Minimal AND/OR related to tree-OBDDs (McMillan
  94),
• d-DNNF (Darwiche et. Al. 2002)
• Case-factor diagrams (Mcallester, Collins,
  Pereira, 2005)
• Tutorial references
• Dechter, Constraint processing, Morgan Kauffman,
  2003
• Dechter, Tractable Structures of Constraint
  Satisfaction problems,
• In Handbook of constraint solving, forthcoming.

48
Conclusion

• AND/OR search should always be used.
• AND/OR BDDs are superior to OBDDs.
• A search algorithm with good and no-good learning
  generates an OBDD or an AND/OR Bdds.
• Dynamic variable ordering can be incorporated
• With caching, AND/OR search is similar to
  inference (variable-elimination)
• The real tradeoff should be rephrased
• Time vs space rather than search vs inference, or
  search vs model-checking.
• Search methods are more sensitive to this
  tradeoff.