Constructive Algorithms for Discrepancy Minimization

1
Constructive Algorithms for Discrepancy
Minimization

• Nikhil Bansal (IBM)

2
Discrepancy What is it?

• Study of gaps in approximating the continuous by
  the discrete.
• Problem How uniformly can you distribute points
  in a grid.
• Uniform For every axis-parallel
  rectangle R
• ( points in R) - (Area of R)
  should be low.

Discrepancy Max over rectangles R ( points
in R) (Area of R)
n1/2
n1/2
3
Distributing points in a grid

• Problem How uniformly can you distribute points
  in a grid.
• Uniform For every axis-parallel
  rectangle R
• ( points in R) - (Area of R)
  should be low.

n 64 points
Van der Corput Set
Uniform
Random
n1/2 discrepancy
n1/2 (loglog n)1/2
O(log n) discrepancy!
4
Discrepancy Example 2

• Input n points placed arbitrarily in a grid.
• Color them red/blue such that each rectangle is
  colored as evenly as possible
• Discrepancy max over rect. R ( red in R -
  blue in R )

Continuous Color each element 1/2 red and 1/2
blue (0 discrepancy) Discrete Random has about
O(n1/2 log1/2 n) Can achieve O(log2.5 n)
5
Applications
CS Computational Geometry, Comb. Optimization,
Monte-Carlo simulation, Machine learning,
Complexity, Pseudo-Randomness, Math Dynamical
Systems, Combinatorics, Mathematical Finance,
Number Theory, Ramsey Theory, Algebra, Measure
Theory,
6
Combinatorial Discrepancy

• Universe U 1,,n
• Subsets S1,S2,,Sm
• Color elements red/blue so each
• set is colored as evenly as possible.
• Find ? n ! -1,1 to
• Minimize ?(S)1 maxS ?i 2 S ?(i)
• For simplicity consider mn henceforth.

7
Best Known Algorithm

• Random Color each element i independently as
• x(i) 1 or -1 with probability ½ each.
• Thm Discrepancy O (n log n)1/2
• Pf For each set, expect O(n1/2) discrepancy
• Standard tail bounds Pr ?i 2 S x(i) c
  n1/2 ¼ e-c2
• Union bound Choose c ¼ (log n)1/2
• Analysis tight Random actually incurs ?((n log
  n)1/2).

8
Better Colorings Exist!

• Spencer 85 (Six standard deviations suffice)
• Always exists coloring with discrepancy
  6n1/2
• (In general for arbitrary m, discrepancy
  O(n1/2 log(m/n)1/2)
• Tight For mn, cannot beat 0.5 n1/2
  (Hadamard Matrix, orthogonal sets)
• Inherently non-constructive proof
• (pigeonhole principle on exponentially large
  universe)
• Challenge Can we find it algorithmically ?
• Certain algorithms do not work Spencer
• Conjecture Alon-Spencer May not be possible.

9
Beck Fiala Thm

• U 1,,n Sets S1,S2,,Sm
• Suppose each element lies in at most t sets (t
  ltlt n).
• Beck Fiala 81 Discrepancy 2t -1.
• (elegant linear algebraic argument, algorithmic
  result)
• Beck Fiala Conjecture O(t1/2) discrepancy
  possible
• Other results O( t1/2 log t log n ) Beck
• O( t1/2 log n )
  Srinivasan
• O( t1/2 log1/2 n )
  Banaszczyk

Non-constructive
10
Approximating Discrepancy

• Question If a set system has low discrepancy
  (say ltlt n1/2)
• Can we find a good discrepancy coloring ?
• Charikar, Newman, Nikolov 11
• Even 0 vs. O (n1/2) is NP-Hard
• (Matousek) What if system has low Hereditary
  discrepancy?
• herdisc (U,S) maxU ½ U disc
  (U, SU)
• Robust measure of discrepancy (often same as
  discrepancy)
• Widely used TU set systems, Geomety,

11
Our Results

• Thm 1 Can get Spencers bound constructively.
• That is, O(n1/2) discrepancy for mn sets.
• Thm 2 If each element lies in at most t sets,
  get bound of O(t1/2 log n) constructively
  (Srinivasans bound)
• Thm 3 For any set system, can find
• Discrepancy O(log (mn)) Hereditary discrepancy.

Other Problems Constructive bounds (matching
current best) k-permutation problem Spencer,
Srinivasan,Tetali Geometric problems ,
12
Relaxations LPs and SDPs

• Not clear how to use.
• Linear Program is useless. Can color each element
  ½ red and ½ blue. Discrepancy of each set 0!
• SDPs (LP on vi vj, cannot control dimension
  of vs)
• ?i 2 S vi 2 n 8 S
• vi2 1
• Intended solution vi (1,0,,0) or
  (-1,0,,0).
• Trivially feasible vi ei (all vis
  orthogonal)

Yet, SDPs will be a major tool.
13
Punch line

• SDP very helpful if tighter bounds needed for
  some sets.
• ?i 2 S vi 2 2 n
• ?i 2 S vi2 n/log n
• vi2 1
• Not apriori clear why one can do this.
• Entropy Method.
• Algorithm will construct coloring over time and
• use several SDPs in the process.

Tighter bound for S
14
Talk Outline

• Introduction
• The Method
• Low Hereditary discrepancy -gt Good coloring
• Additional Ideas
• Spencers O(n1/2) bound

15
Our Approach
16
Algorithm (at high level)
Each dimension An Element Each vertex A
Coloring
Cube -1,1n
Algorithm Sticky random walk Each
step generated by rounding a suitable SDP
Move in various dimensions correlated, e.g. ?t1
?t2 ¼ 0
Analysis Few steps to reach a vertex (walk has
high variance) Disc(Si) does a
random walk (with low variance)
17
An SDP

• Hereditary disc. ? ) the following SDP is
  feasible

SDP Low discrepancy ?i 2 Sj vi 2
?2
vi2 1
Obtain vi 2 Rn
Rounding Pick random Gaussian g
(g1,g2,,gn) each coordinate gi is iid
N(0,1) For each i, consider ?i g vi
18
Properties of Rounding

• Lemma If g 2 Rn is random Gaussian. For any v 2
  Rn,
• g v is distributed as N(0, v2)
• Pf N(0,a2) N(0,b2) N(0,a2b2)
  g v ?i v(i) gi N(0, ?i v(i)2)

Recall ?i g vi

• Each ?i N(0,1)
• For each set S,
• ?i 2 S ?i g (?i2 S vi) N(0, ?2)
• (std deviation ?)

SDP vi2 1 ?i2 S vi2 ?2
?s mimics a low discrepancy coloring (but is not
-1,1)
19
Algorithm Overview

• Construct coloring iteratively.
• Initially Start with coloring x0 (0,0,0, ,0)
  at t 0.
• At Time t Update coloring as xt xt-1 ?
  (?t1,,?tn)
• (? tiny 1/n suffices)

xt(i) ? (?1i ?2i ?ti) Color of
element i Does random walk over time with step
size ¼ ? N(0,1)

x(i)
Fixed if reaches -1 or 1.
Set S xt(S) ?i 2 S xt(i) does a random
walk w/ step ? N(0, ?2)
20
Analysis

• Consider time T O(1/?2)
• Claim 1 With prob. ½, at least n/2 elements
  reach -1 or 1.
• Pf Each element doing random walk with size ¼
  ?.
• Recall Random walk with step 1, is ¼
  O(t1/2) away in t steps.
• A Trouble Various element updates are correlated
• Consider basic walk x(t1) x(t) 1 with
  prob ½
• Define Energy ?(t) x(t)2
• E?(t1) ½ (x(t)1)2 ½ (x(t)-1)2 x(t)2
  1 ?(t)1
• Expected energy n at t n.
• Claim 2 Each set has O(?) discrepancy in
  expectation.
• Pf For each S, xt(S) doing random walk with step
  size ¼ ? ?

21
Analysis

• Consider time T O(1/?2)
• Claim 1 With prob. ½, at least n/2 variables
  reach -1 or 1.
• ) Everything colored in O(log n) rounds.
• Claim 2 Each set has O(?) discrepancy in
  expectation per round.
• ) Expected discrepancy of a set at end O(?
  log n)
• Thm Obtain a coloring with discrepancy O(? log
  (mn))
• Pf By Chernoff, Prob. that disc(S) gt 2
  Expectation O(? log m)
• 
  O(? log (mn))
• is tiny (poly(1/m)).

22
Recap

• At each step of walk, formulate SDP on unfixed
  variables.
• Use some (existential) property to argue SDP
  is feasible
• Rounding SDP solution -gt Step of walk
• Properties of walk
• High Variance -gt Quick convergence
• Low variance for discrepancy on sets -gt Low
  discrepancy

23
Refinements

• Spencers six std deviations result
• Goal Obtain O(n1/2) discrepancy for any
  set system on m O(n) sets.
• Random coloring has n1/2 (log n)1/2
  discrepancy
• Previous approach seems useless
• Expected discrepancy for a set O(n1/2),
  
• but some random walks will deviate by up
  to (log n)1/2 factor

Need an additional idea to prevent this.
24
Spencers O(n1/2) result

• Partial Coloring Lemma For any system with m
  sets, there exists a coloring on n/2 elements
  with discrepancy O(n1/2 log1/2 (2m/n))
• For mn, disc O(n1/2)
• Algorithm for total coloring
• Repeatedly apply partial coloring lemma
• Total discrepancy
• O( n1/2 log1/2 2 ) Phase 1
• O( (n/2)1/2 log1/2 4 ) Phase 2
• O((n/4)1/2 log1/2 8 ) Phase 3
• O(n1/2)

25
Proving Partial Coloring Lemma

• Beautiful Counting argument (entropy method
  pigeonhole)
• Idea Too many colorings (2n), but few
  discrepancy profiles
• Key Lemma There exist k24n/5 colorings X1,,Xk
  such that
• every two Xi, Xj are similar for every set
  S1,,Sn.
• Some X1,X2 differ on n/2 positions
• Consider X (X1 X2)/2
• Pf X(S) (X1(S) X2(S))/2 2 -10 n1/2 ,
  10 n1/2

26
A useful generalization

• There exists a partial coloring with non-uniform
  discrepancy bound ?S for set S
• Even if ?S ?( n1/2) in some average sense

27
An SDP

• Suppose there exists partial coloring X
• 1. On n/2 elements
• 2. Each set S has X(S) ?S

SDP Low discrepancy ?i 2 Sj vi 2
?S2 Many colors ?i vi2 n/2
vi2 1
Pick random Gaussian g (g1,g2,,gn) each
coordinate gi is iid N(0,1) For each i,
consider ?i g vi
Obtain vi 2 Rn
28
Algorithm

• Initially write SDP with ?S c n1/2
• Each set S does random walk and expects to reach
  
• discrepancy of O(DS) O(n1/2)
• Some sets will become problematic.
• Reduce their ?S on the fly.
• Not many problematic sets, and entropy penalty
  low.

Danger 3
Danger 1
Danger 2

35n1/2
0
30n1/2
20n1/2
29
Concluding Remarks

• Construct coloring over time by solving sequence
  of SDPs (guided by existence results)
• Works quite generally
• Can be derandomized Bansal-Spencer
• (use entropy method itself for derandomizing
  usual tech.)
• E.g. Deterministic six standard deviations can be
  viewed as a way to derandomize something stronger
  than Chernoff bounds.

30
Thank You!
31
(No Transcript)
32
(No Transcript)
33
Rest of the talk

• How to generate ?i with required properties.
• How to update ?S over time.
• Show n1/2 (log log log n)1/2 bound.

34
Why so few algorithms?

• Often algorithms rely on continuous relaxations.
• Linear Program is useless. Can color each element
  ½ red and ½ blue.
• Improved results of Spencer, Beck, Srinivasan,
  based on clever counting (entropy method).
• Pigeonhole Principle on exponentially large
  systems (seems inherently non-constructive)

35
Partial Coloring Lemma

• Suppose we have discrepancy bound ?S for set S.
• Consider 2n possible colorings
• Signature of a coloring X (b(S1), b(S2),,
  b(Sm))
• Want partial coloring with signature (0,0,0,,0)

36
Progress Condition

• Energy increases at each step
• E(t) \sum_i x_i(t)2
• Initially energy 0, can be at most n.
• Expected value of E(t) E(t-1) \sum_i
  \gamma_i(t)2
• Markovs inequality.

37
Missing Steps

1. How to generate the \eta_i
2. How to update \Delta_S over time

38
Partial Coloring

• If exist two colorings X1,X2
• 1. Same signature (b1,b2,,bm)
• 2. Differ in at least n/2 positions.
• Consider X (X1 X2)/2
• -1 or 1 on at least n/2 positions, i.e. partial
  coloring
• Has signature (0,0,0,,0)
• X(S) (X1(S) X2(S)) / 2, so X(S) ?S
  for all S.
• Can show that there are 24n/5 colorings with
  same signature.
• So, some two will differ on gt n/2 positions.
  (Pigeon Hole)

X1 (1,-1, 1 , , 1,-1,-1) X2 (-1,-1,-1, ,
1,1, 1)
39
(No Transcript)
40
Spencers O(n1/2) result

• Partial Coloring Lemma For any system with m
  sets,
• there exists a coloring on n/2 elements
  with discrepancy O(n1/2 log1/2 (2m/n))
• For mn, disc O(n1/2)
• Algorithm for total coloring
• Repeatedly apply partial coloring lemma
• Total discrepancy
• O( n1/2 log1/2 2 ) Phase 1
• O( (n/2)1/2 log1/2 4 ) Phase 2
• O((n/4)1/2 log1/2 8 ) Phase 3
• O(n1/2)

Let us prove the lemma for m n
41
Proving Partial Coloring Lemma
-10 n1/2
-30 n1/2
10 n1/2
30 n1/2
0
2
1
-1
-2

• Pf Associate with coloring X, signature
  (b1,b2,,bn)
• (bi bucket in which X(Si) lies )
• Wish to show There exist 24n/5 colorings with
  same signature
• Choose X randomly Induces distribution ? on
  signatures.
• Entropy (?) n/5 implies some signature has
  prob. 2-n/5.
• Entropy (? ) ?i Entropy( bi)
  Subadditivity of Entropy
• bi 0 w.p. ¼ 1- 2 e-50,
• 1 w.p. ¼ e-50
• 2 w.p. ¼ e-450
• .

42
A useful generalization

• Partial coloring with non-uniform discrepancy ?S
  for set S

Suffices to have ?s Ent (bs) n/5 Or, if ?S
?s n1/2 , then ?s g(?s) n/5 g(?)
¼ e-?2/2 ? gt 1
¼ ln(1/?) ? lt 1
43
Recap

• Partial Coloring ?S ¼ 10 n1/2 gives low
  entropy
• ) 24n/5 colorings exist with same
  signature.
• ) some X1,X2 with large hamming
  distance.
• (X1 X2) /2 gives the desired
  partial coloring.
• Trouble 24n/5/2n is an exponentially small
  fraction.

Only if we could find the partial coloring
efficiently