Quantified formulas

1
Quantified formulas
Decision procedures An algorithmic point of
view Daniel Kroening and Ofer Strichman
2
Why do we need quantifiers ?

• As always more modeling power
• Examples of quantifiers usage
• Everyone in the room has a friend
• There is a person in the room that all of his
  cars are red
• There is not more than one person in the room
  that earns more than 1M

3
Quantifiers in Math

• For any integer x there is a smaller integer y
• 8x2Z 9y2Z. y lt x X
• Reverse claim There exists an integer y such
  that any integer x is greater than y
• 9y2Z 8x2Z. y lt x
• (Bertrands postulate) For any natural number
  greater than 1 there is a prime number p such
  that n lt p lt 2n
• 8n2 N. 9p2 N. n gt1 ! (isprime(p) Æ n lt p lt
  2n)

4
Actually

• Satisfiability of ?(x1,?,xn) does there exist
  an interpretation of x1,?,xn that satisfies ? ?
• Validity of ?(x1,?,xn) does it hold that all
  interpretation of x1,?,xn satisfy ??
• Conclusion what we did so far (satisfiability,
  validity) is non-alternating quantification.

5
Example Quantified Propositional Logic

• Better known as Quantified Boolean Formulas (QBF)
• formula var formula formula Ç formula
  ( formula ) T F 8 var. (formula) 9
  var. (formula)
• 8x. (x Ç 9y. (y ! x))
• 8x. (9y. ((x Ç y) Æ (x Çy)) Æ 9y. ((y Ç x) Æ
  (x Ç y)))

X
X
Binding scope of y
6
Prenex Normal-Form (PNF)

• Formulas in PNF look like this
• ? QnVn.? .Q1V1. Quantifier-free
  formula
• where Qi 2 8,9 and Vi is a variable.
• Every quantified formula can be transformed to
  PNF while preserving validity. How ?

prefix
7
Prenex Normal Form (PNF)

• Eliminate ! and (transform to Ç Æ )
• Push negations inside using 8 x. ? 9 x.
  ? 9 x. ? 8 x. ?
• If there are name conflicts across scopes, solve
  with renaming.
• Move quantifiers out by using recursively rules
  such as
• Q1 x. ?1(x) Æ Q2 y. ?2(y) Q1 x. Q2 y. (?1(x) Æ
  ?2(y)) Qi28,9
• Q1 x. ?1(x) Ç Q2 y. ?2(y) Q1 x. Q2 y. (?1(x) Ç
  ?2(y)) Qi28,9
• ?1 Æ 9 x. ?2(x) 9 x. (?1 Æ ?2(x))
  where x does not appear in ?1
• ?1 Æ 8 x. ?2(x) 8 x. (?1 Æ ?2(x))
  where x does not appear in ?1
• 8 x. ?1(x) Æ 8 x. ?2(x) 8 x. (?1(x) Æ ?2(x))
• 9 x. ?1(x) Ç 9 x. ?2(x) 9 x. (?1(x) Ç ?2(x))

8
Prenex Normal Form (PNF) example

• 9x. (9y. ((y ! x) Æ ( x Ç y)) Æ 8y. ((y Æ x)
  Ç ( x Æ y)))
• 1,2. Eliminate !, push negations inside
• 8x. (9y. ((y Ç x) Æ ( x Ç y)) Æ 9y. ((y Ç x)
  Æ (x Ç y)))
• 3. Renaming
• 8x. (9y1. ((y1 Ç x) Æ ( x Ç y1)) Æ 9y2. ((y2 Ç
  x) Æ (x Ç y2)))
• 4. Move quantifiers to front
• 8x. 9y1. 9y2. (x Ç y1) Æ ( x Ç y1) Æ (y2 Ç
  x) Æ (x Ç y2)

9
Why eliminating 9x. ÆiLi is enough

• A procedure for eliminating an existential
  quantifier applied to a conjunction of literals
  is enough, because
• Given a formula ?, write it in DNF.
• Use the fact that
• Eliminate universal quantifiers using the
  fact 8x. ? 9x. ?

10
Quantifier Elimination

• Examples first, generalization later.
• Example 1 Quantified Boolean Formulas (QBF)
• Example 2 Quantified Linear Arithmetic (QLA)

11
Example 1 QBF

• Examples of Quantified Boolean Formula
• ? ?u ?e.(uÇ e)(uÇ e)
• ? ?e4e5 ?u1u2u3 ?e1e2e3. f(e1,e2,e3,e4,e5,u1,u2
  ,u3)
• QBF Problem is ? valid?
• P-Space Complete, theoretically harder than
  NP-Complete problems such as SAT.

12
Motivations

• QBF has practical applications
• AI Planning
• Sequential circuit verification

13
Solving QBF with projection 9

• Eliminate 9x. by projecting x on variables in
  higher quantification levels (their scope
  includes xs scope).
• In Propositional Logic projection can be done
  with Resolution.
• Resolution example

14
Solving QBF with projection 8

• Transform 8 to 9 via (8x. ?) (9x. ?)
• CNF is easier than general formulas
• 8u1u2 9e1 8u3(u1Çe1)(u1Çe1)(u2Çu3Çe1)
• 8u1u2 9e19u3 ((u1Çe1)(u1Çe1)(u2Çu3Çe1))
• 8u1u2 9e19u3 ((u1Æ e1)Ç(u1Æ e1)Ç (u2Æu3Æe1))
• 8u1u2 9e1 ((u1Æ e1)Ç(u1Æ e1)Ç (u2Æ(9u3.
  u3)Æe1))
• 8u1u2 9e1 ((u1Æ e1)Ç(u1Æ e1)Ç (u2Æe1))
• 8u1u2 9e1 (u1Çe1)(u1Çe1)(u2Çe1)

Suffix is DNF
Replace with true
Back to CNF
Shortcut for CNF formulas simply erase
universally quantified variables!
15
Resolution Based QBF Algorithm

• 8u1u29e18u39e3e2(u1Çe1)(u1Çe2Çe3)(u2Çu3Çe1)(e
  1Çe2)(e1Çe3)
• 8u1u29e18u39e3 (u1Çe1)(u1Çe3Çe1)(u2Çu3Çe1)(e1Ç
  e3)
• 8u1u29e18u3 (u1Çe1)(u1Çe1)(u2Çu3Çe1)
• 8u1u29e1(u1Çe1)(u1Çe1)(u2Çe1)
• 8u1u2(u1Ç u2)
• FALSE

16
Example 2 Quantified Linear Arithmetic

• formula predicate formula Ç formula
  formula (formula) 8 var. formula 9 var.
  formula
• predicate ?i ai xi c
• 8x.9y.9z. (y1 x Æ z1 y Æ 2x1 z)

17
Solving QLA with projection

• Eliminate 9x. by projecting x.
• In Linear Arithmetic over R projection can be
  done with Fourier-Motzkin elimination.
• Fourier-Motzkin method to eliminate a variable
  xn- for each pair of constraints
  ?i1..n-1aixi lt xn lt ?i1..n-1aixi
• add a constraint ?i1..n-1aixi lt
  ?i1..n-1aixi
• - in the end remove all constraints involving
  xn.

18
Solving QLA with projection

• Fourier Motzkin example.
• Eliminate y

2y 2z 4
y 3z 3 Æ
x 1 y Æ
x 1 3z 3
x 1 z 2 Æ
19
Quantifier elimination - example

• 8x.9y.9z. (y1 x Æ z1 y Æ 2x1 z)
• 8x.9y. (y1 x Æ 2x1 y-1 )
• 8x. (2x2 x-1) // transform to 9
• 9x. (2x2 x-1)
• 9x. x gt -3
• true
• false

20
Quantifier elimination by projection summary

• Given a PNF formula f QnVn?Q1V1 ?
• For i 1 .. n
• if Qi 9 then ? project(?,Vi)
• else ? project(?,Vi)
• Return ?

21
More about QBF

• Example of using QBF (the diameter problem)
• A search-based procedure for QBF.

Acknowledgement QBF slides borrowed from S. Malik
22
The State Space Diameter Problem
diameter 3
Start from the initial states, the minimum number
of steps needed to visit every reachable state
23
Why is the Diameter Problem important?

• Bounded model checking (BMC) search for a bad
  state up to k steps from an initial step.
• BMC can be formulated as SAT. Increasing k makes
  is harder.
• Q how deep should we go ?
• A as deep as the diameter
• The diameter can be found by solving a QBF problem

24
Why is the Diameter Problem important?

• Bounded model checking (BMC)
• Circuit state space diameter completes BMC
• Can be formulated as QBF instances
• Provides insights to sequential verification
  problems in general

S0 the set of initial states
Does property P hold for the system?
S2
S1
S0
S0
S2
S1
S3
S3
useful for falsification, but incomplete for
verification
25
Circuit Constructed for the Diameter Problem
The idea prove that for every state reachable in
k1 steps, there exists inputs that drive the
model to this state earlier.
26
Some Terminology for the Formulations
Variables V Circuit consistency condition C(V)
27
Some Terminology for the Formulations
Variables V Circuit consistency condition C(V)
28
QBF Formulation
C(V)
C(V)
Other V variables
V inputs
V variables, incl. inputs
29
Another way to project Boolean variables

• Shannon expansion9x. ? ?x0 Ç ?x1 8x. ?
  ?x0 Æ ?x1 // can be derived from 8x.?
  9x.?
• The same applies for all finite-range variables.
• Applying 9x.?, where ? in CNF resolution
• But ? does not need to be in CNF, and there is
  no need to transform the formula to DNF.

30
Projection for non-CNF formulas example

• 9y 8z 9x. (y Ç (x Æ z))
• 9y 8z. (y Ç (x Æ z))x0 Ç (y Ç (x Æ z))x1
• 9y 8z. ((y) Ç (y Ç z))
• 9y 9z. (y Æ z)
• 9y. ((y Æ z)z0 Ç (y Æ z)z1)
• 9y. (y)
• True

31
Search Based QBF Algorithms

• Work by gradually assigning variables
• A partial assignment ?

KGS98 M. Cadoli, A. Giovanardi, M. Schaerf. An
Algorithm to Evaluate Quantified Boolean
Formulae. In Proc. of 16th National Conference on
Artificial Intelligence (AAAI-98)
32
Search Based QBF Algorithms

• Work by gradually assigning variables
• A partial assignment ?
• Undetermined
• Continue search

KGS98 M. Cadoli, A. Giovanardi, M. Schaerf. An
Algorithm to Evaluate Quantified Boolean
Formulae. In Proc. of 16th National Conference on
Artificial Intelligence (AAAI-98)
33
Search Based QBF Algorithms

• Work by gradually assigning variables
• A partial assignment ?
• Undetermined
• Conflict
• Backtrack
• Record the reason

KGS98 M. Cadoli, A. Giovanardi, M. Schaerf. An
Algorithm to Evaluate Quantified Boolean
Formulae. In Proc. of 16th National Conference on
Artificial Intelligence (AAAI-98)
34
Search Based QBF Algorithms

• Work by gradually assigning variables
• A partial assignment ?
• Undetermined
• Conflict
• Satisfied
• Backtrack
• Determine the covered satisfying space

KGS98 M. Cadoli, A. Giovanardi, M. Schaerf. An
Algorithm to Evaluate Quantified Boolean
Formulae. In Proc. of 16th National Conference on
Artificial Intelligence (AAAI-98)
35
Search Based QBF Algorithms

• Work by gradually assigning variables
• A partial assignment ?
• Undetermined
• Conflict
• Satisfied
• The majority of QBF solvers are search based, the
  DPLL algorithm is an example of this

36
Basic DPLL Flow for QBF
?e?u (e Ç u)(e Ç u)
Unknown
True (1)
False(0)
37
Basic DPLL Flow for QBF
?e?u (e Ç u)(e Ç u)
e 0
Unknown
True (1)
False(0)
38
Basic DPLL Flow for QBF
Existential quantification
?e?u (e Ç u)(e Ç u)
Universal quantification
e 0
Satisfying Node
Unknown
True (1)
u 1
False(0)
39
Basic DPLL Flow for QBF
?e?u (e Ç u)(e Ç u)
e 0
Backtrack
Unknown
True (1)
u 1
False(0)
40
Basic DPLL Flow for QBF
?e?u (e Ç u)(e Ç u)
e 0
Unknown
True (1)
u 1
u 0
False(0)
41
Basic DPLL Flow for QBF
?e?u (e Ç u)(e Ç u)
e 0
Unknown
True (1)
u 1
u 0
False(0)
42
Basic DPLL Flow for QBF
?e?u (e Ç u)(e Ç u)
e 1
e 0
Unknown
True (1)
u 1
u 0
False(0)
43
Basic DPLL Flow for QBF
?e?u (e Ç u)(e Ç u)
e 1
e 0
Unknown
True (1)
u 1
u 1
u 0
False(0)
44
Basic DPLL Flow for QBF
?e?u (e Ç u)(e Ç u)
e 1
e 0
Conflicting Node
Unknown
True (1)
u 1
u 1
u 0
False(0)
45
Basic DPLL Flow for QBF
?e?u (e Ç u)(e Ç u)
e 1
e 0
Unknown
True (1)
u 1
u 1
u 0
False(0)
46
Basic DPLL Flow for QBF
False
?e?u (e Ç u)(e Ç u)
e 1
e 0
Unknown
True (1)
u 1
u 1
u 0
False(0)
47
Basic DPLL Flow for QBF
?u?e (u Ç e)(u Ç e)
Unknown
True (1)
False(0)
48
Basic DPLL Flow for QBF
?u?e (u Ç e)(u Ç e)
u 1
Unknown
True (1)
False(0)
49
Basic DPLL Flow for QBF
?u?e (u Ç e)(u Ç e)
u 1
Unknown
True (1)
e 1
False(0)
50
Basic DPLL Flow for QBF
?u?e (u Ç e)(u Ç e)
u 1
Unknown
True (1)
e 1
e 0
False(0)
51
Basic DPLL Flow for QBF
?u?e (u Ç e)(u Ç e)
u 1
Unknown
True (1)
e 1
e 0
False(0)
52
Basic DPLL Flow for QBF
?u?e (u Ç e)(u Ç e)
u 1
u 0
Unknown
True (1)
e 1
e 1
e 0
False(0)
53
Basic DPLL Flow for QBF
True
?u?e (u Ç e)(u Ç e)
u 1
u 0
Unknown
True (1)
e 1
e 1
e 0
False(0)
54
What we saw is DPLL - QBF

• while (true)
• decide_next_branch() //choose a branch variable
• while(true)
• status deduce() //unit propagation
• if (status CONFLICT)
• blevel analyze_conflict() //find out the
  reason for conflict
• if (blevel lt 0) return UNSATISFIABLE
• else backtrack(blevel)
• else if (status SATISFIABLE)
• blevel analyze_SAT() //find out the reason
  for satisfaction
• if (blevel lt 0) return SATISFIABLE
• else backtrack(blevel)
• else break

55
Naïve DPLL Based Approach

• Works on a CNF database
• Learning and non-chronological backtracking is
  much harder requires a change!
• Modern QBF solvers do not work with CNF, rather
  with a combination of CNF with Cubes. This lets
  them apply learning efficiently.

56
ACNF

• Definition Augmented CNF (ACNF)
• ? C1 Æ C2Æ Æ CmÇ S1 Ç S2 Ç Ç Sk
• Where Cis are clauses, and Sj s are cubes.
• Each Sj is contained in the clause term C1 C2Cm.
  
• i.e.?i?1,2k, Si ? C1 Æ C2 Æ Æ Cm
• In ACNF, cubes are redundant
• Example
• (aÇbÇc)Æ(aÇbÇc) Æ(aÇbÇc) Æ(aÇbÇc) Ç
• (aÆbÆc) Ç (aÆ bÆ c)

57
Solving QBF of ACNF formulas

• Cubes are necessary for saving information on the
  (universal) space already covered.
• We will see a special case 2QBF
• 2QBF QBF with one quantifier alternation
• very useful!

58
A special case 2QBF

• DPLL search based, utilize a standard SAT solver
• Algorithm I Assign universal variables first
• Algorithm II no restriction in decision order
  w.r.t. variable quantification order
• Resolution based
• No simplification, just Q-resolution
• With complete two-level minimization (using
  Logic-Minimization) at each resolution step

59
Coverage Cubes and Blocking Clauses
(u1 Ç u2 Ç e1) Æ
(u3 Ç e2) Æ
(u1Ç e1Ç e2)
(u1Ç u2Ç e2) Æ
satisfying assignment u11, u20, u3X, e10,
e20
coverage cube for the universal Boolean space
u1 ? u2
select a set of literals that satisfy all
clauses
blocking clause u1 Ç u2 prevents revisiting
the already searched space
satisfying cube u11, u20, e20
60
Example for Algorithm I
8u1u29e1e2. (u1Ç e1)Æ(u1Ç e1)Æ(u1Ç u2Çe2)
Æ(u2Çe1Ç e2) Æ (e1Çe2)
universal assignment u10, u20
SAT assignment u10, u20, e11, e21
satisfying cube (cover set) u10, e11, e21
coverage cube u10
universal assignment u11, u20
SAT assignment u11, u20, e10, e20
satisfying cube (cover set) u11, e10, e20
coverage cube u11
no more universal assignment left, instance is
true
61
Algorithm I
SAT assignment
universal assignment
1
5
2
4
coverage cube
universal space (u1,u2,,um)
existential space (e1,e2,,en)
62
Example for Algorithm II
8u1u29e1e2. (u1Ç e1)Æ(u1Ç e1)Æ(u1Ç u2Çe2)
Æ(u2Çe1Ç e2) Æ (e1Çe2)
SAT assignment u10, e11, e21, u20
blocking clause (u1 Ç e1 Ç e2)
coverage cube u10
SAT assignment u11, e10, e20 , u20
blocking clause (u1 Ç e1 Ç e2)
coverage cube u11
The entire universal space is covered, instance
is true
63
Algorithm II
SAT assignment, no need to respect quantification
order to get that
coverage cube
3
1
universal variable space (u1,u2,,um)
all variable space (u1,u2,,um,e1,e2,,en)
64
Resolution Based Algorithm

• First resolve out existential variables
• After resolving out all existential variables
• An empty clause (a clause with no literal or
  consisting only of universal variables) ? false
• An empty set of clauses ? true
• Has the memory blowup problem
• Alleviate by simplifying the propositional part
  after each resolution step

65
Example for Resolution Based Algorithm
8u1u29e1e2. (u1Ç e1)Æ(u1Ç e1)Æ(u1Ç u2Çe2)
Æ(u2Çe1Ç e2) Æ (e1Çe2)
resolve out e1
8u1u29e2. (u1Ç u2 Ç e2)Æ(u1Ç e2)Æ(u1Ç u2Çe2)
simplify
8u1u29e2. (u1Ç u2 Ç e2)Æ(u1Ç e2)
resolve out e2
empty set of clause instance is true
66
Experimental Results (Malik et al.)
Res w/o simp
Res w/ simp
Quaffle
Alg. II
Alg. I
of clauses (100 vars, 5 lits/clause)
0.01(100)
0.25(100)
100 (100 instances)
gt400(0)
16.22(98)
0(100)
0.36(10)
169.09(6)
200 (10 instances)
gt400(0)
gt400(0)
160(6)
Res w/o simp
Res w/ simp
Depth
Quaffle
Alg. II
Alg. I
Circuit
gt400
2.19
s1488
0.27
0.26
0.15
1
gt400
25.04
18.23
294.27
0.80
3
gt400
208.6
gt400
gt400
2.5
5
gt400
4.29
s1423
0.22
gt400
0.15
1
gt400
gt400
gt400
gt400
2.11
15
gt400
gt400
gt400
gt400
298.86
34
The tests were done on an Intel Pentium III 933
MHz machine with 1GB of RAM running linux.
Improves on previous diameter lower bound of 26