NPComplete

1
NP-Complete
What is an algorithm What is a proof What is a
hard problem NP-Complete problems -- practical
problems that are strongly suspected to be
computationally hard.
2
Computability theory When can this be computed?
Can this be computed at all? Complexity theory
How cheaply can this be computed? How hard is
this to compute? Inconsistencies 1900 Georg
Cantor, invented set theory.
3
August, 1900, The Second International Congress
of Mathematicians in Paris. Hilbert, formalist
school of mathematic, manipulated symbols by
fixed rules without thinking of any possible
meaning the symbols could have. Strict formalism,
by not letting semantic interpretations influence
how or when rules are applied, supposedly
guarantees that whenever we prove something we
are sure it is correct.
4
Formal system
A formal system is a finite set of assumed
truths, called axioms, and a finite set of rule,
called inference rules. We can use the rules on
the axioms to infer new results called theorems.
A proof of a theorem is a demonstration of a
sequence of inferred theorems starting only with
the axioms and using only the inference rules and
ending with the theorem.
5
One of Hilberts problem was to establish that
arithmetic was consistent, meaning that, under
the accepted rules of arithmetic, it is not
possible to generate a contradiction. 1931 Kurt
Godel (24), showed that if any system as
"powerful" as arithmetic is consistent, then it
cannot be complete -- meaning that some true
statements are unprovable in the system!
6
Today we don't know whether arithmetic is
consistent all we can say is that if it is, then
there are true things we cannot prove in it.
This is similar to the uncertainty principle in
quantum mechanics, Heisenberg 1927. Hilbert's
continuum problem Cantor, there is more than
one infinity. Using the proof called
diagonalization, Cantor showed that there were
more real numbers than there were integers. But
no one knew if there were an infinity between the
number of integers and the number of reals.
7
r0 0.d00d01d02. . . r1 0.d10d11d12. . . r2
0.d20d21d22. . . . . . rn 0.dn0dn1dn2. . .dnn .
. . diagonal d00d11d22dnn. . . change each digit
into the "opposite". This will be a new number
not in the original listing. N lt R Are
there any thing between N lt S lt R ?
8
1938 Godel showed that it couldn't be proved
false from Cantor's axioms of set theory. 1963
Cohen showed that it couldn't be proved true from
Cantor's axioms. The continuum hypothesis is
independent of the usual axioms of set theory.
If set theory is consistent to begin with, then
we can assume the continuum hypothesis true or
false and still have a consistent set theory!
9
The formalist school was resisted by a group of
mathematicians called constructivists
(intuitionists) who wanted to toss infinite sets
and rebuild mathematics from the bottom up
starting from the integers and applying only
finite operations to them.
10
Model of computation
Another Hilbert's problem was to find a "finitary
way" to solve any diophantine equation. A
diophantine equation is a polynomial equation in
n unknowns that can have only integer solutions.
Hilbert wanted an "algorithm" (not yet invented
the name) 1970 Matijasevic (22) showed, using
fibonacci numbers, that this problem is
computationally unsolvable.
11

• Turing machine, Alan Turing (24) 1936
• Herbrand functions, Jacques Herbrand
• Godel functions, Kurt Godel
• Church calculus, Alonzo Church
• Kleen functions, Stephen Kleen
• Post systems, Emil Post
• Markov algorithms, Andrei Markov
• same computational ability

12
Hilbert's problem the decision problem -- asked
whether there is an algorithm deciding the truth
of falsify of any mathematical statement.
Church and Turing, 1936, no such algorithm
exists. The result is provable from Godel's work
on incompleteness in 1931. What it meant for
something to be computable.
13

• Traveling salesman problem
• Knapsack problem
• Graph coloring problem
• Are some problems harder than the other?
• Which ones?
• How much harder are they?
• 1965 Edmonds, Cobham, definition of a feasible
  problem A problem is feasible if it has a
  solution whose cost is at most polynomial.

14
Why polynomial

• Polynomials are closed under composition and
  addition
• All sequential digital computers are polynomially
  related
• In general, a polynomial algorithm will do a
  feasible amount of work.

15
1947 Dantzig, algorithm for optimize linear
programs, simplex. Simplex worst cost is
exponential in the number of variables. 1979
Hacijan (kah-chi-yan) proved that an algorithm,
ellipsoid algorithm, is polynomial.
(multidimensional binary search). 1984 Karmakar,
projective scaling, improved ellipsoid to run
well in practice. Karmakar's algorithm is
polynomial. Although Simplex is exponential, the
linear programming problem is polynomial.
16
NP-completeness
polynomial time algorithm O( nk )
tractable superpolynomial time algorithm
intractable
17
Class of problems NP-complete, status unknown No
polynomial time algorithm has yet been discovered
for any NP-complete problem. P ! NP question
(posed 1971) Believe NP-complete problems are
intractable
18
If any single NP-complete problem can be solved
in polynomial time, then every NP-complete
problem has a polynomial time algorithm. problem
-gt NP-complete -gt approximate algorithm
19
Define complexity P (formal language) complexity
NP decision problem where solutions can be
verified in polynomial time. (more work on this
slide)
20
Abstract problems
Binary relations of a set of problem instances to
a set of solutions. Theory of NP-completeness
restricts to decision problem yes/no
answer. Example Shortest Path Give a graph G
(V,E) and u, v in V, non negative integer k, does
a path exist in G between u and v whose length is
at least k?
21
Cast optimization problems to decision problems
by imposing a bound on the value to be optimized.
22
Encoding
An encoding of a set S of abstract object is a
mapping e from S to the set of binary string. A
problem whose instance set is the set of binary
strings is called a concrete problem. An
algorithm "solves" a concrete problem in O(T(n))
if when it is provided a problem instance i of
length n i the algorithm can produce the
solution in at most O(T(n)).
23
Complexity class P the set of concrete decision
problem that are solvable in polynomial
time. Encoding map abstract problem Q to concrete
problem. If solution to an abstract problem i in
I is Q(i) in 0,1, then the solution to the
concrete problem instance e(i) in 0,1 is also
Q(i).
24
We would like the definition to be independent of
any particular encoding. An algorithm with the
only input k is running in time Zeta(k) if k is
unary ( a string of 1's ) its running time is
O(n) if k is binary n ceil( lg k) its running
time is Zeta(2n)
25
A function f 0,1 -gt 0,1 is polynomial
time computable if there exists a polynomial time
algorithm A that, given input x in 0,1,
produces as output f(x). For some set I of
problem instances, two encoding e1 and e2 are
polynomially related if there exist two
polynomial time computable functions f12 and f21
such that for any i in I, we have f12(e1(i))
e2(i) and f21(e2(i)) e1(i).
26
That is, encoding e2(i) can be computed from the
encoding e1(i) by a polynomial time algorithm,
and vice versa. Lemma 36.1 Let Q be an abstract
decision problem on an instance set I, and let e1
and e2 be polynomially related encodings on I.
Then, e1(Q) is-in P if and only if e2(Q) is-in
P.
27
Standard encoding
we shall generally assume that problem instances
are encoded in any reasonable, concise fashion.
Encoding of an integer is polynomially related
to its binary representation. Encoding of a
finite set is polynomially related to its
encoding as a list of elements.
28
Formal language framework
An algorithm A accepts a string x in 0,1 if,
given input x, the algorithm outputs A(x) 1.
The language accepted by an algorithm A is the
set L x in 0,1 A(x) 1 . An algorithm
rejects a string x if A(x) 0.
29
A language L is decided by an algorithm A if
every binary string is either accepted or
rejected by the algorithm. A language L is
accepted in polynomial time by an algorithm A if
for any length-n string x in L, the algorithm
accepts x in time O(nk) for some constant k. A
language L is decided in polynomial time by an
algorithm A if for any length-n x in 0,1, the
algorithm decides x in time O(nk) for some
constant k.
30
Complexity class P
P L subset 0,1 there exists an algorithm
A that decides L in polynomial time. Theorem
36.2 P L L is accepted by a polynomial time
algorithm.
31
Polynomial time verification
Hamiltonian cycles Does a graph G have a
Hamiltonian cycle? HAM-CYCLE ltGgt G is
hamiltonian grah How an algorithm decide the
language HAM-CYCLE?
32
Given a problem instance ltGgt, lists all
permutations of the vertices of G and then checks
each permutation. With "reasonable" encoding of a
graph as its adjacency matrix, the number m of
vertices in the graph is Ohmega(sqrt(n)), n
ltGgt is the length of the encoding of G. The
running time is Ohmega(m!) Ohmega(sqrt(n)!)
Ohmega( 2 sqrt n).
33
Another easier problem Given a graph G, claims
to be hamiltonian with the prove as a list of
vertices. It is easy to verify the proof by
checking if the given list of vertices is a
permutation of the vertices of V and whether each
of the consecutive edges along the cycle actually
exists in the graph. O(n2). Thus a proof that
a hamiltonian cycle exists in a graph can be
verified in polynomial time.
34
Define a verification algorithm , A , a
two-argument algorithm, an input string x and a
binary string y called a certificate. A verifies
an input x if there exists a certificate y such
that A(x,y) 1. The language verified by a
verification algorithm A is L x in 0,1
there exists y in 0,1 such that A(x,y) 1 .
35
An algorithm A verifies a language L if for any
string x in L, there is a certificate y that A
can use to prove that x is-in L.