CSE 3813 Introduction to Formal Languages and Automata

1
CSE 3813Introduction to Formal Languages and
Automata

• Chapter 14
• An Introduction to Computational Complexity
• These class notes are based on material from our
  textbook, An Introduction to Formal Languages and
  Automata, 4th ed., by Peter Linz, published by
  Jones and Bartlett Publishers, Inc., Sudbury, MA,
  2006. They are intended for classroom use only
  and are not a substitute for reading the
  textbook.

2
Additional notes on complexity

• (Acknowledgement These notes are derived from
  material developed by Dr. Susan Bridges and Dr.
  Lois Boggess, and from the course textbook for CS
  8833.)

3
Algorithmics and Computational Complexity

• Algorithmics is the study of the performance
  characteristics of specific algorithms.
  Computational Complexity (also called
  Computability Theory) looks at the
  characteristics of problems, rather than
  algorithms.

4
Solvable and unsolvable

• We make a distinction between solvable and
  unsolvable problems. Within the solvable
  problems, we make a three-way distinction. So
  the subdivision looks like this

5
Solvable and unsolvable

• Unsolvable problems
• Example The halting problem
• Solvable problems
• Provably intractable problems (also
  called infeasible problems)
• Probably intractable problems
• Tractable problems (feasible problems)

6
Solvable and unsolvable

• Example of a provably intractable (infeasible)
  problem
• Generalized chess - Chess is played on a board of
  size 8 x 8 generalized chess is played on a
  board of size N x N, with the set of pieces and
  the set of moves adjusted as needed. It has been
  proven to have an exponential lower bound.

7
Does P NP?

• There is a question that has been unanswered in
  spite of a lot of work in computational theory
  Does P NP?
• Some related questions
• What are the meanings of the terms, P, NP,
  NP-complete, and NP-hard?
• Why do we care whether P NP?
• What is the best guess as to whether P NP?
• What are the consequences, if that best guess is
  right?

8
Polynomial time algorithms

• What is a polynomial time algorithm?
• Traditional answer A polynomial time algorithm
  is one that can be executed by a standard
  (deterministic) Turing machine in polynomial
  time, proportional to the size of the input.
• Easier to understand answer A polynomial time
  algorithm is one in which the worst case running
  time is O(nk) where n is the input size and k is
  a constant.

9
Polynomial time algorithms

• Can all problems be solved in polynomial time, if
  we just make k large enough?
• Answer No.
• It can be proved that some problems that we can
  describe cant be solved at all.
• Some problems can be proven to require at least
  exponential time.
• There is an important class of problems that
  probably require exponential time.

10
Polynomial time algorithms

• Are algorithms that are polynomial time
  considered efficient?
• Answer Algorithms are considered efficient if
  there is a polynomial p(n) such that the
  algorithm can solve any instance of the problem
  of size n in O(p(n)) time. Of course, in
  reality, the efficiency of a polynomial-time
  algorithm also depends on

11
Polynomial time algorithms

• the constant of proportionality that is being
  ignored in the big-O formalism. If very large,
  the algorithm still isnt efficient, for any
  practical purposes
• the size of the exponent of the polynomial. An
  algorithm whose exponent is 50 wouldnt be
  efficient in practice
• the size of n itself. Very, very large inputs
  make even n2 algorithms unattractive.

12
Whats so great about polynomial time?

• Whats so great about polynomial time, then?
• A problem that is ?(n50) wouldnt really be
  efficient, but polynomial time problems with that
  high an exponent are rare. Usually the exponents
  are small.

13
Whats so great about polynomial time?

• For most reasonable models of computation, a
  problem that can be solved in polynomial time in
  one model can be solved in polynomial time in
  another.
• So, for example, a problem that is polynomial in
  time on a serial random access machine is
  typically also polynomial on a Turing machine as
  well as on a parallel computing system, and vice
  versa.

14
Whats so great about polynomial time?

• The class of polynomial time problems has some
  nice closure properties polynomial-time
  algorithms are closed under addition (f(n)
  g(n)), multiplication (f(n)g(n)), and
  composition (g(f(n)).

15
Decision problems

• What are decision problems?
• In the theory of NP-Completeness, we concentrate
  on one particular type of problem decision
  problems. We think of each instance of a given
  problem as mapping onto a solution space that
  consists of yes, no or 0,1.
• This may seem like a severe restriction on the
  type of problems that we will consider, but it is
  not. For example, optimization problems are
  fairly easy to restate as decision problems. And
  there exists a proof that if the decision problem
  is easy, then the optimization problem is easy.

16
Optimization problems

• Example optimization problem - A shortest path
  problem
• Given a graph G ltV, Egt, two vertices, u, v ? V,
  and a non-negative integer k, does a path exist
  in G between u and v whose length is k or less?
• Let i (G, u, v, k) be an instance of the
  problem. Then the shortest path question with
  a yes/no answer becomes Path(i) 1 or Path(i)
  0, where 1 yes and 0 no.

17
Optimization problems

• To change an optimization problem into a decision
  problem, we can usually just impose a bound on
  the value to be optimized.
• Examples from path problems include minimization
  problems (Is the length of the shortest path at
  most k?) and maximization problems (Is the length
  of the longest path at least k?).

18
Optimization problems

• Usually, if the optimization problem can be
  solved quickly, the decision problem can also be
  solved quickly.
• Conversely, evidence that the decision problem is
  hard normally is evidence that the optimization
  problem is also hard.

19
Abstract problems

• What are abstract problems?
• An abstract problem Q is a binary relation on a
  set I of problem instances and a set S of problem
  solutions.

20
Encoding

• What is an encoding?
• An encoding of a set S of abstract objects is a
  mapping e from S to the set of binary strings.

21
Concrete problems

• What is a concrete problem?
• A concrete problem is a problem whose instance
  is the set of binary strings.

22
What is P?

• The complexity class P is the set of concrete
  decision problems that are solvable in polynomial
  time.
• Alternately, P is the class of languages that are
  accepted in polynomial time by a standard Turing
  machine.

23
What is P?

• (The limitation to concrete, rather than
  abstract, decision problems is not a big
  limitation, because abstract decision problems
  are mapped into concrete problems by an encoding.
  Care must be taken for this step, however,
  because a bad encoding can make a problem much
  more difficult. We will ignore the difference
  between abstract and concrete, and assume that
  any encodings required were implemented
  reasonably well.)

24
The class NP

• What is a verification algorithm?
• A verification algorithm is a two-argument
  algorithm A, in which one argument is an input
  string x and the other is a binary string y
  called a certificate.
• The verification algorithm A verifies an input
  string x if there exists a certificate y such
  that A(x, y) 1.
• The algorithm A verifies a language L if, for any
  string x ? L, there is a certificate y that A can
  use to prove that x ? L.
• For example, in the Hamiltonian cycle problem,
  the certificate is a list of vertices that
  comprise the Hamiltonian cycle.

25
The class NP

• The complexity class NP is the class of languages
  that can be verified by a polynomial time
  algorithm.
• Alternatively, a language L belongs to NP iff
  there exists a two-input polynomial-time
  algorithm A and a constant c such that L is a set
  of binary strings for which there exists a
  certificate y with y O(xc) such that A(x,
  y) 1.

26
The class NP

• What does the term NP mean?
• NP stands for Nondeterministic Polynomial. That
  is, a nondeterministic Turing machine can solve
  it in polynomial time, but no known deterministic
  Turing machine can.

27
The class NP

• When does a problem belong to the class NP?
• A problem belongs to the class NP under the
  following conditions
• Given that an oracle can guess an answer to the
  problem (provide a certificate),
• then we can find an algorithm that can verify
  that the answer is correct in polynomial time.

28
The class NP

• Note that the problem is NP. It is the verifying
  algorithm that is P.
• Note also that the class of problems solvable in
  polynomial time (P) is a subset of NP by
  definition.

29
Example of an NP problem

• Consider the question of whether a given graph
  has a Hamiltonian cycle (a simple cycle that
  visits every vertex in the graph). This question
  is NP as we show on the next slide

30
Example of an NP problem

• Given a certificate (a list of vertices that
  purports to be a Hamiltonian cycle in the graph)
  
• check to see if the first and last vertex on the
  list are the same
• check that every vertex in the graph is on the
  list
• check that every vertex in the list is connected
  by an edge of the graph to the next vertex on the
  list.
• check to see that no vertex other than the
  first/last vertex is repeated.
• Is this doable in polynomial time? Yes!

31
NP-Complete problems

• What is the class of NP-Complete problems?
• In some sense, the NP-complete problems are the
  hardest problems in the class NP.
• No polynomial time algorithm is known to solve
  any of them.
• Every NP-Complete problem is reducible to every
  other NP-complete problem in polynomial time.

32
NP-Complete problems

• What do we mean by reducible, in the previous
  sentence?
• A decision problem Q is polynomial-time reducible
  to decision problem Q' if a polynomial time
  algorithm can be developed which changes each
  instance of problem Q into an instance of problem
  Q' such that if the answer for an instance of Q'
  is yes, the answer to the corresponding instance
  of Q is yes. Then we write

33
NP-Complete problems

• Lemma 1
• If Q1 and Q2 are problems such that
• then Q2 ? P implies Q1 ? P.
• Proof
• Suppose A1 is a polynomial time algorithm that
  reduces instances of Q1 to instances of Q2 and
  that A2 is a polynomial time algorithm for
  problem Q2. Then the composition of A1 and A2 is
  a polynomial time algorithm for Q1.

34
NP-Complete problems

• Definition of NP-Complete
• A problem Q is NP-Complete if
• Q is an element of the class NP
• for every Q' in NP

35
NP-Hard problems

• Definition of NP-Hard
• A problem Q is NP-Hard if
• for every Q' in NP

36
NP-Complete problems

• Theorem
• If any NP-complete problem is polynomial-time
  solvable, then P NP.
• If any problem in NP is not polynomial-time
  solvable, then all NP-complete problems are not
  polynomial-time solvable.

37
P NP???

• Proof of the first statement
• Let Q be NP-complete and polynomial. Then for
  all Q' in NP,
• and by a previous Lemma it follows that Q' is in
  P.

38
P NP???

• Proof of the second statement
• Let Q be NP but not polynomial. By definition of
  NP-complete, for all Q' in the
  set of NP-complete problems. It follows that all
  Q' ? P.
• (Suppose not. Then there exists some Q' in P,
  but all elements of NPC, including
  Q', so Q is in P by a previous Lemma, which
  contradicts our initial assumption that Q was not
  in P.)

39
Conclusion

• It can be proved that some problems that we can
  describe cant be solved at all. These are
  Unsolvable problems.
• Example The halting problem.

40
Conclusion

• Of the problems that can be solved
• some problems can be proven to require at least
  exponential time. These are infeasible or
  provably intractable problems.
• There is an important class of problems that
  probably require exponential time. These are
  probably intractable or NP problems.
• The problems that we prefer to work with require
  only polynomial time. These are feasible,
  tractable, or P problems.