Complexity%20Theory%20Lecture%201

1
Complexity TheoryLecture 1

• Lecturer Moni Naor

2
Computational Complexity Theory

• Study the resources needed to solve computational
  problems
• Computer time
• Computer memory
• Communication
• Parallelism
• Randomness
• Identify problems that are infeasible to compute
  by any reasonable machine
• Taxonomy classify problems into classes with
  similar properties wrt the resource requirements
• Help find the most efficient algorithm for a
  problem

3
Goal of Complexity Theory

• Given a computational problem determine the
  best method of solving it
• For a given computational setting determine
  which computation tasks can be executed
• State of the art very far from these goals
• Cannot provide unconditional interesting lower
  bounds for almost any interesting problem
• But

4
Complexity, Algorithms and Cryptography
Complexity
Cryptography
Algorithms

• Complexity Theory has developed a taxonomy
  allowing to determine for many problems

5
Brief History of Complexity Theory

• Logical foundations
• 1930s Computability Theory
• Kurt Gödel
• Alan Turing
• Alonzo Church
• 1940s 1950s work on circuit complexity
  (Shannon)
• 1960s first papers dealing explicitly with
  complexity issues
• Definitions of polynomial time (Cobham, Edmonds,
  Peterson)
• Time hierarchies, Speedup Theorems, Abstract
  Complexity Measures
• Early 1970s formulation of the PNP? question,
  NP-Completeness an industry (Cook, Karp, Levin)
• Since then an explosion

Developments in the Soviet Union

• 1956 Gödels letter to von Neumann with the PNP
  question

6
What will we learn in this course

• Time and space hierarchies
• Non-determinism and NP-Completeness
• Relativization (oracles)
• Alternation
• Space Complexity
• Randomness
• Interactive Proofs
• PCP
• Complexity of counting
• Concrete Complexity models
• Communication Complexity
• Pebbling
• Branching Programs
• Circuit Complexity

P, RP, BPP BQP NP, co-NP, PH, P, PSPACE, IP,
AM, PCP NC, AC, L, NL
7
Course Information

• Complexity Theory  - Winter 2004/5
• Instructor Moni Naor
• Grader Asaf Nussbaum
• When     Thursdays, 1400--1700 (3 points)
• Where    Ziskind 1
• Course web page www.wisdom.weizmann.ac.il/naor/C
  OURSE/complexity.html
• Prerequisites familiarity with algorithms, data
  structures, probability theory, and linear
  algebra, at an undergraduate level a basic
  course in computability is assumed.
• Requirements
• Homework There will be around ten homework
  assignments and a final test.
• Homework assignments should be turned in on time
  (usually two weeks after they are given)!
• Try and do as many problems from each set.
• You may discuss the problems with other students,
  but the write-up MUST be individual.
• You may look up any material, but should cite any
  external source.
• Exam The exam will be in class.

8
Bibliography

• No textbook for the course. A lot of background
  and relevant material is available in 
• Christos Papadimitriou , Computational Complexity
  , Cambridge, 1994
• Michael Sipser, Introduction to the Theory of
  Computation, 1997
• Garey and  Johnson. Computers and Intractability
  A Guide to the Theory of NP-Completeness. New
  York W. H. Freeman, 1979.
•      Two online  courses close in nature to our
  course
• Lecturers Steven Rudich and Avrim Blum,
  University CMU
• Lecturer Luca Trevisan, Computational
  Complexity, University UC Berkeley.

9
Things you should already know

• Turing Machine
• One-tape vs. Multi-tape
• Defined by (?,Q,?)
• ? finite alphabet including blank symbol
• State space including qstart, qaccept and
  qreject
• transition function ? ? X Q ? ? X Q X
  left,right
• A configuration is represented as a string of
  symbols from ? Q
• ?1 ?2 q ?m
• Multi-tape machine has
• Read-only input tape
• Write-only output tape
• Several working tapes

10
Things you should already know

• Recognition or decision of language L by a TM M
• if M accepts every string in L and rejects all
  strings not in L
• A language L is recursive or decidable if there
  is a TM M which recognizes it
• Acceptance of language L by a TM M
• if M accepts every string in L and does not
  accept strings not in L
• does not necessarily halt on all inputs
• A language L is recursive enumerable (r.e.) if
  there is a TM M which accepts it

11
Turing Machines are general

• Truing Machines can simulate many other models
  and machines
• RAM Random Access Machines
• Java..
• Circuits if the circuit itself is uniform
• Various logical systems
• Turing machines the language
• Lu ltM,xgtM accepts x
• is r.e. there is a (universal) TM accepting it
• Simulation are pretty efficient
• Counterexample quantum

12
Time and Space

• Let fN? N by any function.
• Time L 2 TIME(f(n)) if there is a (multi-tape)
  TM which recognizes L and uses no more than
  f(x) steps on input x
• Space L 2 SPACE(f(n)) if there is a (multi-tape)
  TM which recognizes L and uses no more than
  f(x) cells of its work tapes during the
  computation on input x
• Cell is used if it scanned by the tape head
• Time and space can be defined for function
  computation as well
• No point in accuracy in the time function
• Theorem (Linear Speedup) for any L 2 TIME(f(n))
  and ?gt0 there is an equivalent TM M requiring
  only ?f(n)n steps
• Conclusion lets talk about O(f(n))

13
Proper complexity functions

• f(n) is a proper complexity function if
• f(n) f(n-1)
• There is a TM which on input x outputs a string
  of length exactly f(x) while running it time
  O(xf(x)) and space O(f(x))
• Exercise show that if f(n) and g(n) are proper
  complexity functions, then so are
• f(n)g(n)
• f(n)g(n)
• Clocked Simulation Theorem for every proper
  complexity function f
• There exists a TM Mf running in time f3(Mx)
  that on input (M,x) accepts iff M(x) accepts
  within f(x) steps
• Can improve f3(n) to f(n) log f(n)

14
Hierarchy Theorems and Diagonalization

• Diagonalization proof technique invented by
  Cantor to prove the uncountability of the real
  numbers
• Theorem (Time Hierarchy) for any proper
  complexity functions f(n) and g(n) f(n)
  log(f(n))
• TIME(f(n)) is properly contained in TIME(g(2n))
• Proof Recall Mf. Consider
• L ltMMf(MM) rejects
• We have L2TIME(g(2n)) but L is not in TIME(f(n))
  
• Corollary TIME(nk) is properly contained in
  TIME(nk1)

15
Communication Complexity
x2X
y2Y
Let fX x Y? Z Input is split between two
participants Want to compute outcome zf(x,y)
while exchanging as few bits as possible
16
A protocol is defined by the communication tree
z5
z0 z1 z2 z3 z4 z5 z6 z7 ...
17
A Protocol

• A protocol P over domain X x Y with range Z is a
  binary tree where
• Each internal node v is labeled with either
• avX? 0,1 or
• bvY? 0,1
• Each leaf is labeled with an element z 2 Z
• The value of protocol P on input (x,y) is the
  label of the leaf reached by starting from the
  root and walking down the tree.
• At each internal node labeled av walk
• left if av(x)0
• right if av(x)1
• At each internal node labeled bv walk
• left if bv(y)0
• right if bv(y)1
• The cost of protocol P on input (x,y) is the
  length of the path taken on input (x,y)
• The cost of protocol P is the maximum path length

18
Motivation for studying communication complexity

• Originally for studying VLSI questions
• Connection with Turing Machines
• Data structures and the cell probe model
• Boolean circuit depth

19
Communication Complexity of a function

• For a function fX x Y? Z the (deterministic)
  communication complexity of f (D(f)) is the
  minimum cost of protocol P over all protocols
  that compute f
• Observation For any function fX x Y? Z
• D(f) log X log Z
• Example
• let x,y µ 1,,n and let f(x,y)maxx y
• Then D(f) 2 log n

20
Median

• let x,y µ 1,,n and let MED(x,y) be the median
  of the multiset x y
• If the size is even then element ranked x y/2
• Claim D(MED) is O(log2 n)
• protocol idea do a binary search on the value,
  each party reporting how many are above te
  current guess
• Homework D(MED) is O(log n)
• protocol idea each party proposes a candidate
• See which one is larger - no need to repeat bits

21
Combinatorial Rectangles

• A combinatorial rectangle in X x Y is a subset R
  µ X x Y such that R A x B for some A µ X and B
  µ Y
• Proposition R µ X x Y is a combinatorial
  rectangle iff (x1,y1) 2 R and (x2,y2) 2 R implies
  that (x1,y2) 2 R
• For Protocol P and node v let Rv be the set of
  inputs (x,y) reaching v
• Claim For any protocol P and node v the set Rv
  is a combinatorial rectangle
• Claim For any given the transcript of an
  exchange between Alice an Bob possible (but not
  x and y) possible to determine zf(x,y)

22
Fooling Sets

• For fX x Y? Z a subset R µ X x Y is
  f-monochromatic if f is fixed on R
• Observation any protocol P induces a partition
  of X x Y into f-monochromatic rectangles. The
  number of rectangles is the number of leaves in P
• A set Sµ X x Y is a fooling set for f if there
  exists a z 2 Z where
• For every (x,y) 2 S, f(x,y)z
• For every distinct (x1,y1), (x2,y2) 2 S either
• f(x1,y2)?z or
• f(x2,y1)?z
• Property no two elements of a fooling set S can
  be in the same monochromatic rectangle
• Lemma if f has a fooling set of size t, then
  D(f) log2 t

23
Applications

• Equality Alice and Bob each hold x,y 2 0,1n
• want to decide whether xy or not.
• Fooling set for Equality
• S(w,w)w 2 0,1n
• Conclusion D(Equality) n
• Disjointness let x,y µ 1,,n and let
• DISJ(x,y)1 if x ? y 1 and
• DISJ(x,y)0 otherwise
• Fooling set for Disjointness
• S(A,comp(A))A µ 1,,n
• Conclusion D(DISJ) n

24
Applications to Time-Space lower bounds in Turing
Machines

• Let PalindromesxxR x 2 ?
• Palindromes 2 TIME(n)
• Palindromes 2 SPACE(log n)
• But what about simultaneous linear time and log
  space?
• Lemma let f 0,1n x 0,1n ? 0,1 be a
  function with communication complexity D(f) . For
  any multi-tape TM running in time T(n) and space
  S(n) accepting all string
• x0ny yxn, f(x,y)1
• and rejecting all strings
• x0ny yxn, f(x,y)0
• We have D(f) O(T(n) S(n)/n)
• Proof joint simulation of the TM by Alice and
  Bob.
• To get time-space lower bound on Palindromes use
  Equality
• Consider x0nxR x 2 ?, xn

25
Rank lower bound

• For fX x Y? 0,1 let Mf tbe the X x Y
  matrix where entry f(x,y) has value f(x,y). The
  rank of f, rank(f), is the rank of Mf over the
  reals.
• Theorem For any fX x Y? 0,1
• D(f) log2 rank(f)
• Proof For any protocol P Mf ?l leaf Ml
• Ml is the matrix corresponding to the rectangle
  of leaf l
• Examples
• Equality MEquality is the identity matrix rank
  is 2n

26
Inner Product

• Let x,y 2 0,1n
• IP(x,y) ?i1n xi yi mod 2
• What is rank(MIP)?
• Let NMIP MIP. Then entry (x,y) in N is
• ?z 2 0,1n ltx,zgt lty,zgt
• which is zs where ltx,zgtlty,zgt1
• 2n-2 if x?y
• 2n-1 if xy
• 0 if x or y is 0
• Hence rank(N) 2n-1. Since
• rank(MIP MIP) minrank(MIP,rank(MIP
• rank(MIP 2n-1 and D(IP) n

27
Non-determinism and covers

• Can a protocol tree be very unbalanced?
• Technique for balancing given a protocol with t
  leaves there is a protocol with communication
  complexity O(log t)
• Is the monotone rectangles lower bound tight?
• Consider instead of
• partition into monochromatic rectangles
• a cover by monochromatic rectangles
• The rectangles are not necessary disjoint
• For z 2 Z a z-cover handles only the inputs (x,y)
  where f(z,y)z
• This corresponds to non-deterministic complexity
• A non-deterministic protocol for verifying that
  f(x,y)z
• Alice Guess a rectangle R intersecting row x,
  send name to Bob
• Bob verify that R intersects column y tell
  Alice
• Accept only if Bob approve
• Complexity log of z-rectangles