Analyzing classic SDG for MAXCSP

1
Analyzing classic SDGfor MAXCSP

• Karl Lieberherr
• Northeastern University

2
Algorithm Design

• Algorithmic techniques Greedy, Divide and
  Conquer, Local Search, Exhaustive search,
  Randomization
• Telling the machine about algorithms DemeterF
  and DemFGen
• Our application domain Artificial Market of
  Derivatives success in the market relies on
  algorithmic knowledge

3
Randomized Algorithms(Chapter 13.
Kleinberg/Tardos)

• Yield correct answer with high probability
• Internal randomization
• Conceptually much simpler
• Needs very little probability theory

4
Random Variables and Their Expectations

• Expected size of satisfied constraints over
  random choices made by algorithm.
• Random variable f sample space -gt natural
  number
• PrXj Xs probability of taking a given value
  random variable
• Example Formula with n variables expected
  number of satisfied constraints under random
  assignment (n,p)
• Each variable is set to 1 with prob. p.

5
Expectation of random variable X

• average value assumed by X
• EX sum from j0 to inf jPrXj
• Waiting for a first success p -gt 1/p
• There must be at least one assignment that
  satisfies at least as much as EX

6
Linearity of expectation

• Given two random variables X and Y defined over
  the same probability space, we can define XY to
  be the random variable equal to X(w)Y(w) on a
  sample point w. For any X and Y, we have
• EXY EX EY

7
Application MAX 3-SAT

• derivative d ((3,0) (3,1) (3,2) (3,3), p, s)
• derivative d ((127) (191) (239) (254), p, s)
• negatives first
• n variables
• each is set to 0 or 1, with probability ½
• what is the expected number of clauses satisfied
  by such random assignment

8
Application MAX 3-SAT

• Z a random variable equal to the number of
  satisfied clauses.
• Decompose Z into a sum of random variables that
  each take the value 0 or 1.
• Zi 1 if clause Ci is satisfied, 0 otherwise.
• Z Z1Z2 Zk.
• EZi7/8
• Linearity of expectation EZ (7/8) k

9
Application 22

• Generalize Is ½ best (fair coin)? Use
  probability b instead. Think of a bent coin.
• Decompose Z into a sum of random variables that
  each take the value 1 or 0 with probability b.
• Zi 1 if clause Ci is satisfied, 0 otherwise.
• Z Z1Z2 Zk.
• EZi3b(1-b)2
• Linearity of expectation EZ 3b(1-b)2 k
• b ½ 3/8 375/1000
• b 4/10 54/125 448/1000 gt 3/8 !!
• what is best b?

10
Reminder of calculus

• chain rule (g o f)(x) g(f(x))f(x)
• product rule h(x) f(x)g(x)
  h(x)f(x)g(x)f(x)g(x)
• quadratic polynomials ax2bxc
• x(-b-sqrt(b2-4ac))/2a

11
From Boolean Relation to Random Variable to
Polynomial

• Examples
• 22 gtgtgt 3b(1-b)2
• 127 gtgtgt 1-b3
• 254 gtgtgt 1-(1-b)3
• 191 gtgtgt 1-b(1-b)2
• 170 gtgtgt b // clause A
• 119 gtgtgt 1-b2 // clause !A or !B
• Choose b to maximize satisfaction ratio.
• 22 b1/3

12
Checking polynomials

• 000 1 1
• 001 2 1
• 010 4 1
• 011 8 1
• 100 16 1
• 101 32 1
• 110 64 1
• 111 128 0
• 127 (3,0)

polynomial pr(127,b) 1-b3
13
Checking polynomials
dont care
y x

• 000 1 1
• 001 2 1
• 010 4 1
• 011 8 0
• 100 16 1
• 101 32 1
• 110 64 1
• 111 128 0
• 119 !x or !y

polynomial pr(119,b) 1-b2
14
Checking numbers
dont care
x

• 000 1 0
• 001 2 1
• 010 4 0
• 011 8 1
• 100 16 0
• 101 32 1
• 110 64 0
• 111 128 1
• 170 unit clause x

polynomial pr(127,b) b
15
Checking numbers

• 000 1 0
• 001 2 0
• 010 4 0
• 011 8 1
• 100 16 0
• 101 32 0
• 110 64 0
• 111 128 1
• 136 (2 important variables)
• function a and b

polynomial pr(136,b) b2
16
Checking numbers

• 000 1 0
• 001 2 1
• 010 4 1
• 011 8 0
• 100 16 1
• 101 32 0
• 110 64 0
• 111 128 0
• 22 (1 in 3)

polynomial pr(22,b) 3 b(1-b)2
17
Linear combinations

• 170 gtgtgt b
• 119 gtgtgt 1-b2
• t1b t2(1-b2), t1 t2 1
• PrClause is satgt
• max 0ltblt1 min(b,1-b2)
• Set b1-b2 b ½ (sqrt(5)-1)h 0.618

18
Finding the best bias

• max 0ltblt1 min(b,1-b2)
• The minimum is maximum when b1-b2
• Theorem
• min0ltt1lt1 max0ltblt1 t1b (1-t1)(1-b2)
• max 0ltblt1 min(b,1-b2)
• (does this hold in general?)

19
(No Transcript)
20
Noise elimination

• Probabilistic method eliminates noise
• Predicates determine optimal bias!

21
What does this mean?

• In any raw material for ((170,119),p,s) we can
  satisfy the fraction h 0.618 by flipping a bent
  coin.
• Questions that remain
• can we find such an assignment fast enough?
• is there a raw material where we can only satisfy
  about the fraction h?

22
Also works with weighted clauses

• Consider random variable Zj 1 if clause Cj is
  satisfied, 0 otherwise.
• W sum j1 to k wj Zj
• E(W) sum j1 to k wj E(Zj)
• For (170,119)
• E(W) min(b,1-b2) sum j1 to k wj
• min(b,1-b2) TotalWeight

23
More questions

• Given a predicate (R1,R2,R3), is it a matter of
  finding
• min(pr(R1,b), pr(R2,b), pr(R3,b))?
• pr(R,b) is the probability that the relation is
  satisfied if the variables are set with bias b.
• Is it enough to consider only pairs and take the
  minimum over all pairs?
• min(pr(R1,b), pr(R2,b)) min(pr(R1,b), pr(R3,b))
  min(pr(R2,b), pr(R3,b))

24
Are there formulas where the biased coin result
is optimal?

• Yes formulas where it does not matter which
  subset of the k variables are set to true for any
  k. All those binomial(n.k) assignments must give
  the same value.
• Compute kmax and set kmax/n bmax
• How do those formulas look like?

25
General Form (2 relations)

• t1pr(r1,b)t2pr(r2,b)
• t1t21
• Given a raw material, compute the best b to
  finish the raw material.
• To construct a raw material, find the worst
  (t1,t2). Use all combinations to generate
  constraints for a given relation. Formula becomes
  symmetric.

26
General Form (3 relations)

• fsat(t,b)t1pr(r1,b)t2pr(r2,b)t3pr(r3,b)
• t1t2t31
• Given t1, t2 and t3, compute the best b.
• For raw material construction find the worst t1,
  t2, t3.
• min t max b fsat(t,b)

27
Golden nugget of insight

• May simplify your program significantly.
• Are at a conceptual level.

28
Application domain knowledge

• dozens of relations in predicate
• reduce to two for raw material construction!!

29
Controlling complexity of raw materials

• Set of relations R(1,1), R(pos,length) pos of
  length literals are positive, length gt2.
• RR R(1,1), R(pos,length), length gt2
• e.g., RR (R(1,1),R(0,2),R(1,2),R(2,2), R(0,3),
  R(2,3)
• Lemma The worst raw material in this class only
  needs to use RRsimpleR(1,1), R(0,2).

30
Proof

• Let s be an arbitrary RR-formula. We simplify it
  to a cnf T(S) using
• In clauses containing positive literals, drop all
  but one.
• In clauses containing only negative literals, but
  more than one, drop all except two.
• T(s) is in RRsimple and to each assignment I1 of
  T(s) corresponds an assignment I of s which
  satisfies at least as many clauses as I1 in T(s).

31
Example of reduction
A B C A B !A !B !C A C B !C !A
!B !C !D

• A B C
• A B
• !A !B !C
• A C
• B !C
• !A !B !C !D

3A 2!A !B 1 B
A1,B0,C0 all sat
A1,B0,C0 5/6 sat
32
Switch to secret version

• derivative d (pred, price, s)
• if instances(pred) in P, price 1
• d(secret MAX 3-SAT (3,3) (3,2) (3,1) (3,0), 7/8,
  s) Hastad
• d(classic MAX 3-SAT (3,3) (3,2) (3,1) (3,0),
  7/8, s)
• pred instances of Independent Set that come
  from Interval Scheduling

33
Algorithm Design

• Algorithmic techniques Greedy, Divide and
  Conquer, Local Search, Exhaustive search,
  Randomization
• Telling the machine about algorithmic techniques
  DemeterF and DemFGen
• Our application domain Artificial Market of
  Derivatives success in the market relies on
  algorithmic knowledge.

34
Derandomization

• E is expectation, F is a weighted CSP formula,
  Z(F,b) random variable for weighted satisfied
  constraints. F is a CSP formula, Fx0 Fx1 are
  reduced formulas.
• E(Z(F,b))(1-b)E(Z(Fx0,b))bE(Z(Fx1,b))
• max(E(Z(Fx0,b)), E(Z(Fx1,b))gt E(Z(F,b))
• Greedy algorithm take better of the two choices.
  Greedy algorithm stays ahead of the average.

35
How to prove?

• E(Z,F)(1-b)E(Z,Fx0)bE(Z,Fx1)