Great Ideas in Computer Science

1
Great Ideas in Computer Science

• Hard Problemsand CS Theory
• COMP 41 May 1

2
Some Facts

• All cows are purple.
• The King of Spain is a cow.
• Is there a possible world in which these
  statements can be true?

3
Satisfiability

• A set of statements is satisfiable if they are
  true in some possible world (not necessarily are
  own).
• This set of statements is satisfiable
• All cows are purple.
• The King of Spain is a cow.

4
Satisfiability

• This set of statements is not satisfiable
• All cows are purple.
• The King of Spain is a cow.
• The King of Spain is green.
• This is an example of a paradox.

5
Satisfiability is HARD

• Identifying a paradox is an example of a really
  hard problem
• Since all possible subsets need to be checked,
  the difficulty of the problem increases
  dramatically as the size of the set increases
• For even 100 statements, the problem is, for all
  practical purposes, insolvable

6
Richard Karp and Steven Cook

• In 1972, Richard Karp identified 21 really hard
  problems, including
• Satisfiability
• Traveling Salesman
• Hamiltonian Circuit (a cyclic path through a
  graph that visits each node exactly once)
• Steven Cook wrote a similar paper in 1971.
• This set of problems is described as NP-Complete

7
NP-Complete Problems

• Other Np-Complete problems
• Finding a path through a labyrinth
• Decoding Ciphers
• Constructing crossword puzzles
• Subset sum problem (is the sum of any subset of a
  set of numbers zero)
• The surprising thing these can all be reduced to
  the same problem!

8
Reduction

• All NP-Complete problems are the same problem
  meansIf we can find an efficient solution to
  any of them, well have an efficient solution to
  all of them
• This is the essence of reduction, the ability to
  transform one problem into another

9
Why is Hard Hard?

• The difficulty in NP-Complete problems is that
  the only way we know to solve them is to try
  every possibility.
• It takes a very long time to try every
  possibility,but only a very short time to verify
  the correctness of one possibility

10
How Hard is Hard?

• Subset sum problem for a set of threeA,B,C
• We have to checkA, B, C, A,B, A,C,
  B,C, A,B,C
• 7 possibilities

11
How Hard is Hard?

• Subset sum problem for a set of fourA,B,C,D
• We have to checkA, B, C, D A,B,
  A,C, B,C, A,D, B,D, C,D A,B,C,
  A,C,D, A,B,D, B,C,D, A,B,C,D
• 15 possibilities adding one element makes the
  problem more than twice as hard!

12
How Hard is Hard?

• Subset sum problem for a set of n numbers, there
  are 2n-1 subsets to check

13
How Hard is 1030?

• Suppose we had a computer that could 109 subsets
  per second (a gigahertz) (about 100 times faster
  than todays PCs)
• It would take 1030/109 1021 seconds to solve a
  set of 100 numbers 1.26765e021
  seconds 2.11275e019 minutes 3.52125e017
  hours 1.46719e016 days 4.01969e013
  years 4.01969e011 centuries 4.01969e010
  millenium

14
Hardness grows fast!

• The same hypothetical computer could solve set
  of 30 in a second set of 40 in 18 minutes set
  of 50 in 13 days set of 60 in 36 years set of
  70 in 374 centuries

15
Theoretical Problem Space
All problems
NP problems
NPcomplete
Pproblems
16
Problems Outside of NP

• These are solutions that are hard to solve and
  hard to verify
• For all practical purposes, these arent
  interesting from the perspective of computer
  science theory

17
Polynomial (P) Problems

• If you have a jigsaw puzzle with n pieces, the
  solution time is proportional to n2
• This is an example of a polynomial time problem
• These are considered easy problems

18
Nondeterministic Polynomial (NP)

• NP problems are hard to solve but easy to verify
• A nondeterministic computer can be thought of as
  an infinite number of parallel computers
• Each one is assigned one of the possible
  solutions to check
• Since checking a solution is easy (polynomial),
  an ND computer can easily solve an NP problem

19
Nondeterminism isnt Real

• Unfortunately, ND computers arent real
• But they do illustrate that NP problems are easy
  to verify
• An ND computer can also be thought of as an
  Oracle that magically guesses the correct
  solution and is only required to verify its
  correctness

20
Mazes are NP Problems

• Consider a maze
• If you have to check every possible path, you
  have an NP problem.
• If you can magically make the correct choice at
  every intersection, you can simply walk through
  to verify the solution.

21
NP-Complete

• The NP-Complete problems are the hardest NP
  problems
• No has found an easy algorithm to solve the
  NP-complete problems,
• but no one has proven that an easy algorithm does
  not exist!
• It is conceivable that all NP problems have an
  easy (polynomial) solution

22
The Challenge

• This one of the most significant open challenges
  in computer science
• Demonstrate a polynomial time solution to the
  NP-complete problems, orprove that one
  cannot exist.

23
Problems not Computers

• All this theory, in fact all CS theory, is about
  the problems and solutions, not about particular
  computers (real or imagined)
• The theory is as universal as mathematics and
  applies to any possible computer