Combinatorial Compressed Sensing: Fast algorithms with Recovery Guarantees

1
Combinatorial Compressed Sensing Fast
algorithms with Recovery Guarantees

• Mark Iwen

2
Outline

• Group Testing
• Group Testing to Compressive Sensing
• Fast Guaranteed Recovery
• Applications

3
Group Testing

• Find a small number of defective/interesting
  items from a large set.
• Syphilis Testing D43
• Industrial experiment design SG59
• Test large groups of objects
• Tests must be sensitive to defectives isolated
  with (many) other non-defective items

4
Group Testing Continued

Algorithm

• Boolean t x N measurement matrix M
• Boolean signal s in 0,1N containing d ones
• All arithmetic Boolean ( OR, AND)
• Identify the location of d ones using
• y Ms
• measurements.
• How small can we make t and still recover s?

5
Good Group Testing Matrices

• d-disjunct The component-wise OR of any d
  columns of M do not contain any other column of
  M.
• Equivalent To
• (d1)-strongly selective Let X be a subset of
  (d1) of Ms columns, and choose any x in X.
  There is a row in M with a 1 in column x and 0s
  in all of the X-x columns.

6
Example Group Testing Matrix

• The 3 x 8 Bit Test Matrix
• 1-disjunct and 2-strongly selective
• Simple identification

7
Constructions

• Explicit
• Number Theoretic M05
• Binary Superimposed Codes PR08
• Number of rows t O(d2 log N)
• Randomized Non-adaptive binary M GS06
• w.h.p. per signal (adaptively gives uniformity)
• Number of rows t O(d polylog(N))

8
Outline

• Group Testing
• Group Testing to Compressive Sensing
• Fast Guaranteed Recovery
• Applications

9
Group Testing to CS

• Compressive Sensing with Binary Measurements
  Noisy Group Testing
• We allow various degrees of defectiveness to
  exist in the population and arithmetic no longer
  Boolean
• Finding a sparse representation for s in RN is
  equivalent to finding the most defective members
  of the population and their defectiveness
  quantities

10
Fast Group Testing in CS

• Randomized Constructions
• Near optimal d-term sparse approximation with
• O(d log3 N)-measurements (w.h.p. per signal)
  CM05
• Near optimal d-term sparse approximation with
• O(d logO(1) N)-measurements (uniform) GSTV06
• Both sublinear in reconstruction runtimes

11
Fast Group Testing in CS Contd

• Explicit Deterministic Algorithms
• (c,p)-compressible signal s in RN s(ib) c
  b-p
• Using (d-1)-disjunct matrices recover
  near-optimal d-term approximation with
  O(d4p2/(p-1)2 log4 N) measurements CM06
• Improved O(d2p/(p-1) log4 N) measurements I08
• Both sublinear in reconstruction runtimes
• BUT Sampling and Runtime depend on p!

12
Outline

• Group Testing
• Group Testing to Compressive Sensing
• Fast Guaranteed Recovery
• Applications

13
Sublinear-time Recovery, Guaranteed

• Best d-sparse approximation to s in RN
• Theorem BGIKS08 Fix ? in (0,1). Given Ms a
  d-sparse approximation, a, with
• s - a 1 (1 ?) s - aopt 1
• can be recovered in time
• polynomial(d, 1/?, 2(log log N)O(1) ).

14
Sublinear-time Recovery, Guaranteed

• Theorem BGIKS08 Fix ? in (0,1). Given Ms a
  d-sparse approximation, a, with
• s - a 1 (1 ?) s - aopt 1
• can be recovered in time
• polynomial(d, 1/?, 2(log log N)O(1) ).
• M is the adjacency matrix of an unbalanced
  expander graph (binary entries).
• Sampling and runtime independent of p.

14
Uniformly Bounded Explicit
15
A Simplified Result with Guarantees

• Best d-sparse approximation to s in RN
• Theorem Fix ? in (0,1). Given Ms a d-sparse
  approximation, a, with
• s - a 1 (1 ?) s - aopt 1
• can be recovered in time
• (d/?)2 polylog(N).

15
Uniformly Bounded Explicit
16
A Simplified Result with Guarantees

• Theorem Fix ? in (0,1). Given Ms a d-sparse
  approximation, a, with
• s - a 1 (1 ?) s - aopt 1
• can be recovered in time
• (d/?)2 polylog(N).
• M is number theoretic with binary entries
• Sampling and runtime independent of p.

16
Uniformly Bounded Explicit
17
Runtime Vs Measurements

• Slower reconstruction allows fewer measurements
• If LP decoding is used t O(d log(N/d)) binary
  linear measurements suffice BGIKS08
• Explicit M have t O(d 2(log log N)O(1)) rows
• Provides near-optimal Noisy Group Testing
• Bernoulli M may sometimes be acceptable

17
Compressive Sensing
18
Outline

• Group Testing
• Group Testing to Compressive Sensing
• Fast Guaranteed Recovery
• Applications

18
Uniformly Bounded Explicit
19
Applications

• Traditional group testing applications can
  benefit from generalized CS-related approaches
• Several CS-related approaches use constrained
  measurement types (e.g. Fourier)
• High throughput drug screening
• Want to find active compounds.
• Design combinations of compounds and test for
  desired results (i.e., use Boolean M rows!)
• Streaming algorithms (trend statistics, )
• Streamed information (e.g., Walmart sales)
• Linear measurements quickly updated yn1 yn
  M(?s)

20
Function Learning

• We can sample an unknown function
• f RN ? R
• at will.
• Each function evaluation is costly (requires many
  experiments, simulations, etc.)
• We want to know which variables are most
  important (i.e., have the largest partial
  derivatives) near an interesting point x0 in RN

21
Examples

• f RN ? R
• Which 5 design parameters (tire specs, body shape
  specs, engine specs, etc.) most effect the fuel
  efficiency of my new car design?
• Which 5 species should I protect to best preserve
  the health of ecosystem X?
• Which 5 code parameter/option changes best
  improve the runtime of task Y?

22
Point Evaluation Design

• Group testing (or CS) matrices give us good local
  point evaluations for our function
• Let Mj be a t x N group testing matrix row. We
  have
• The domain of the function dictates applicability
  of M

23
THANK YOU!!!!

• QUESTIONS?

24
References

• D43 R.Dorfman. The detection of defective
  members of large populations. Ann. Math. Stat.,
  14436-440, 1943.
• SG59 M. Sobel and P. A. Groll. Group testing
  to eleminate efficiently all defectives in a
  binomial sample. Bell System Technical Journal,
  281179-1252, 1959.
• M05 S. Muthukrishnan. Data Streams
  Algorithms and Applications. Foundations and
  Trends in Theoretical Computer Science, 1, 2005.
• PR08 Ely Porat and Amir Rothschild. Explicit
  Non-adaptive Combinatorial Group Testing Schemes.
  ICALP, 748-759, 2008.
• GS06 A. C. Gilbert and M. J. Strauss. Group
  Testing in Statistical Signal Recovery.
  Preprint, 2006.
• CM06 G. Cormode and S. Muthukrishnan.
  Combinatorial Algorithms for Compressed Sensing.
  DIMACS Technical Report, 2006.
• I08 M. A. Iwen. A Deterministic Sublinear
  Time Sparse Fourier Algorithm via Non-adaptive
  Compressed Sensing Methods. SODA, 2008.
• GSTV06 A. Gilbert, M. Strauss, J. Tropp, and
  R. Vershynin. Algorithmic linear dimension
  reduction in the l1 norm for sparse vectors.
  Submitted, 2006.
• BGIKS08 R. Berinde, A. C. Gilbert, P. Indyk,
  H. Karloff, and M. J. Strauss. Combining
  geometry and combinatorics a unified approach
  to sparse signal recovery. Preprint, 2008.