Algorithms

1
Algorithms
Tradeoff Execution speed vs. solution quality
Short sweet
Quick dirty
Slowly but surely
Too little, too late
2
Computational Complexity
Problem Avoid getting trapped in local minima
Global optimum
3
Approximation Algorithms

• Idea Some intractable problems can be
  efficiently approximated within close to optimal!
• Fast
• Simple heuristics (e.g., greed)
• Provably-good approximations
• Slower
• Branch-and-bound approaches
• Integer Linear Programming relaxation

4
Approximation Algorithms

• Wishful
• Simulated annealing
• Genetic algorithms

5
Minimum Vertex Cover
Minumum vertex cover problem Given a graph, find
a minimum set of vertices such that each edge is
incident to at least one vertex of these
vertices. Example
Input graph
Heuristic solution
Optimal solution

• Applications bioinformtics, communications,
• civil engineering, electrical engineering, etc.
• One of Karps original NP-complete problems

6
Minimum Vertex Cover Examples
7
Approximate Vertex Cover
Theorem The minimum vertex cover problem is
NP-complete (even in planar graphs of max degree
3). Theorem The minimum vertex cover problem
can be solved exactly within exponential time
nO(1)2O(n). Theorem The minimum vertex cover
problem can not be approximated within 1.36OPT
unless PNP. Theorem The minimum vertex cover
problem can be approximated (in linear time)
within 2OPT. Idea pick an edge, add its
endpoints, and repeat.
8
Approximate Vertex Cover

• Algorithm Linear time 2OPT approximation for
  the minimum vertex cover problem
• Pick random edge (x,y)
• Add x,y to the heuristic solution
• Eliminate x and y from graph
• Repeat until graph is empty

Best approximation bound known for VC!
Idea one of x,y must be in any optimal
solution. Þ Heuristic solution is no worse than
2OPT.
9
Maximum Cut
Maximum cut problem Given a graph, find a
partition of the vertices maximizing the of
crossing edges. Example
cut size 2
cut size 4
cut size 5
Input graph
Heuristic solution
Optimal solution

• Applications VLSI circuit design, statistical
  physics,
• communication networks.
• One of Karps original NP-complete problems.

10
Maximum Cut
Theorem Karp, 1972 The minimum vertex cover
problem is NP-complete. Theorem The maximum
cut problem can be solved in polynomial time
for planar graphs. Theorem The maximum cut
problem can not be approximated within
17/16OPT unless PNP. Theorem The maximum cut
problem can be approximated in polynomial time
within 2OPT. Theorem The maximum cut problem
can be approximated in polynomial time within
1.14OPT.
1.0625OPT
11
Maximum Cut

• Algorithm 2OPT approximation for maximum cut
• Start with an arbitrary node partition
• If moving an arbitrary node across the partition
• will improve the cut, then do so
• Repeat until no further improvement is possible

cut size 2
cut size 3
cut size 5
Input graph
Heuristic solution
Optimal solution
Idea final cut must contain at least half of all
edges. Þ Heuristic solution is no worse than
2OPT.
12
Approximate Traveling Salesperson
Traveling salesperson problem given a pointset,
find shortest tour that visits every point
exactly once.

• 2OPT metric TSP heuristic
• Compute MST
• T Traverse MST
• S shortcut tour
• Output S

Analysis
S lt T

MST lt OPT TSP
2
2
triangle inequality!
TSP minus an edge is a spanning tree
T covers minimum spanning tree twice
13
Non-Approximability

• NP transformations typically do not preserve the
• approximability of the problem!
• Some NP-complete problems can be approximated
• arbitrarily close to optimal in polynomial time.
• Theorem Geometric TSP can be approximated in
• polynomial time within (1e)OPT for any egt0.
• Other NP-complete problems can not be
  approximated
• within any constant in polynomial time (unless
  PNP).
• Theorem General TSP can not be approximated
• efficiently within KOPT for any Kgt0 (unless
  PNP).

14
Graph Isomorphism
Definition two graphs G1(V1,E1) and G2(V2,E2)
are isomorphic iff bijection ƒV1V2 such
that "vi,vjÎV1 (vi,vj)ÎE1 Û
(ƒ(vi),ƒ(vj))ÎE2 Isomorphism º edge-preserving
vertex permutation Problem are two given graphs
isomorphic?


Note Graph isomorphism ÎNP, but not known to be
in P
15
Graph Isomorphism






16
Zero-Knowledge Proofs
Idea proving graph isomorphism without
disclosing it! Premise Everyone knows G1 and G2
but not must remain secret!

Create random G G1 Note is
() Broadcast G Verifier asks for or
Broadcast or Verifier checks GG1 or
GG2 Repeat k times Þ Probability of cheating 2-k


Approximating a proof!
17
Zero-Knowledge Proofs
Idea prove graph 3-colorable without disclosing
how! Premise Everyone knows G1 but not its
3-coloring ? which must remain secret!
?'
Create random G2 G1 Note 3-coloring ?'(G2)
is (?(G1)) Broadcast G2 Verifier asks for
or ?' Broadcast or ?' Verifier checks G1G2
or ?'(G2) Repeat k times Þ Probability of
cheating 2-k

?
Interactive proof!
18
Zero-Knowledge Caveats

• Requires a good random number generator
• Should not use the same graph twice
• Graphs must be large and complex enough

• Applications
• Identification friend-or-foe (IFF)
• Cryptography
• Business transactions

19
Zero-Knowledge Proofs
Idea prove that a Boolean formula P is
satisfiable without disclosing a satisfying
assignment! Premise Everyone knows P but not its
secret satisfying assignment V !
P (xyz)(x'y'z)(x'yz')
ƒ
Convert P into a graph 3-colorability
instance G ƒ(P) Publically broadcast ƒ and
G Use zero-knowledge protocol to show that G
is 3-colorable Þ P is satisfiable iff G is
3-colorable Þ P is satisfiable with probability
1-2-k
G
Interactive proof!
20
Interactive Proof Systems

• Prover has unbounded power and may be malicious
• Verifier is honest and has limited power
• Completeness If a statement is true, an honest
  verifier
• will be convinced (with high prob) by an honest
  prover.
• Soundness If a statement is false, even an
  omnipotent
• malicious prover can not convince an honest
  verifier that
• the statement is true (except with a very low
  probability).
• The induced complexity class depends on the
  verifiers
• abilities and computational resources
• Theorem For a deterministic P-time verifier,
  class is NP.
• Def For a probabilistic P-time verifier, induced
  class is IP.
• Theorem Shamir, 1992 IP PSPACE

21
Concepts, Techniques, Idea Proofs

1. 2-SAT
2. 2-Way automata
3. 3-colorability
4. 3-SAT
5. Abstract complexity
6. Acceptance
7. Ada Lovelace
8. Algebraic numbers
9. Algorithms
10. Algorithms as strings
11. Alice in Wonderland
12. Alphabets
13. Alternation
14. Ambiguity
15. Ambiguous grammars
16. Analog computing
17. Anisohedral tilings
18. Aperiodic tilings
19. Approximate min cut

1. Approximate vertex cover
2. Approximations
3. Artificial intelligence
4. Asimovs laws of robotics
5. Asymptotics
6. Automatic theorem proving
7. Autonomous vehicles
8. Axiom of choice
9. Axiomatic method
10. Axiomatic system
11. Babbages analytical engine
12. Babbages difference engine
13. Bin packing
14. Binary vs. unary
15. Bletchley Park
16. Bloom axioms
17. Boolean algebra
18. Boolean functions
19. Bridges of Konigsberg

1. Busy beaver problem
2. C programs
3. Canonical order
4. Cantor dust
5. Cantor set
6. Cantors paradox
7. CAPCHA
8. Cardinality arguments
9. Cartesian coordinates
10. Cellular automata
11. Chaos
12. Chatterbots
13. Chess-playing programs
14. Chinese room
15. Chomsky hierarchy
16. Chomsky normal form
17. Chomskyan linguistics
18. Christofides heuristic
19. Church-Turing thesis

22
Concepts, Techniques, Ideas Proofs

1. Clique problem
2. Cloaking devices
3. Closure properties
4. Cogito ergo sum
5. Colorings
6. Commutativity
7. Complementation
8. Completeness
9. Complexity classes
10. Complexity gaps
11. Complexity Zoo
12. Compositions
13. Compound pendulums
14. Compressibility
15. Computable functions
16. Computable numbers
17. Computation and physics
18. Computation models
19. Computational complexity

1. Computational universality
2. Computer viruses
3. Concatenation
4. Co-NP
5. Consciousness and sentience
6. Consistency of axioms
7. Constructions
8. Context free grammars
9. Context free languages
10. Context sensitive grammars
11. Context sensitive languages
12. Continuity
13. Continuum hypothesis
14. Contradiction
15. Contrapositive
16. Cooks theorem
17. Countability
18. Counter automata
19. Counter example

1. Crossing sequences
2. Cross-product construction
3. Cryptography
4. DARPA Grand Challenge
5. DARPA Math Challenges
6. De Morgans law
7. Decidability
8. Deciders vs. recognizers
9. Decimal number system
10. Decision vs. optimization
11. Dedekind cut
12. Denseness of hierarchies
13. Derivations
14. Descriptive complexity
15. Diagonalization
16. Digital circuits
17. Diophantine equations
18. Disorder
19. DNA computing

23
Concepts, Techniques, Ideas Proofs

1. Dovetailing
2. DSPACE
3. DTIME
4. EDVAC
5. Elegance in proof
6. Encodings
7. Enigma cipher
8. Entropy
9. Enumeration
10. Epsilon transitions
11. Equivalence relation
12. Euclids Elements
13. Euclids axioms
14. Euclidean geometry
15. Eulers formula
16. Eulers identity
17. Eulerian tour
18. Existence proofs
19. Exoskeletons

1. Exponentiation
2. EXPSPACE
3. EXPSPACE
4. EXPSPACE complete
5. EXPTIME
6. EXPTIME complete
7. Extended Chomsky hierarchy
8. Fermats last theorem
9. Fibonacci numbers
10. Final states
11. Finite automata
12. Finite automata minimization
13. Fixed-point theorem
14. Formal languages
15. Formalizations
16. Four color problem
17. Fractal art
18. Fractals
19. Functional programming

1. Fundamental thm of Arith.
2. Gadget-based proofs
3. Game of life
4. Game theory
5. Game trees
6. Gap theorems
7. Garey Johnson
8. General grammars
9. Generalized colorability
10. Generalized finite automata
11. Generalized numbers
12. Generalized venn diagrams
13. Generative grammars
14. Genetic algorithms
15. Geometric / picture proofs
16. Godel numbering
17. Godels theorem
18. Goldbachs conjecture
19. Golden ratio

24
Concepts, Techniques, Ideas Proofs

1. Grammars
2. Grammars as computers
3. Graph cliques
4. Graph colorability
5. Graph isomorphism
6. Graph theory
7. Graphs
8. Graphs as relations
9. Gravitational systems
10. Greibach normal form
11. Grey goo
12. Guess-and-verify
13. Halting problem
14. Hamiltonian cycle
15. Hardness
16. Heuristics
17. Hierarchy theorems
18. Hilberts 23 problems
19. Hilberts program

1. Historical perspectives
2. Household robots
3. Hung state
4. Hydraulic computers
5. Hyper computation
6. Hyperbolic geometry
7. Hypernumbers
8. Identities
9. Immermans Theorem
10. Incompleteness
11. Incompressibility
12. Independence of axioms
13. Independent set problem
14. Induction its drawbacks
15. Infinite hotels applications
16. Infinite loops
17. Infinity hierarchy
18. Information theory
19. Inherent ambiguity

1. Intelligence and mind
2. Interactive proofs
3. Intractability
4. Irrational numbers
5. JFLAP
6. Karps paper
7. Kissing number
8. Kleene closure
9. Knapsack problem
10. Lambda calculus
11. Language equivalence
12. Law of accelerating returns
13. Law of the excluded middle
14. Lego computers
15. Lexicographic order
16. Linear-bounded automata
17. Local minima
18. LOGSPACE
19. Low-deg graph colorability

25
Concepts, Techniques, Ideas Proofs

1. Machine equivalence
2. Mandelbrot set
3. Manhattan project
4. Many-one reduction
5. Matiyasevichs theorem
6. Mechanical calculator
7. Mechanical computers
8. Memes
9. Mental poker
10. Meta-mathematics
11. Millennium Prize
12. Minimal grammars
13. Minimum cut
14. Modeling
15. Multiple heads
16. Multiple tapes
17. Mu-recursive functions
18. MAD policy
19. Nanotechnology

1. Navier-Stokes equations
2. Neural networks
3. Newtonian mechanics
4. NLOGSPACE
5. Non-approximability
6. Non-closures
7. Non-determinism
8. Non-Euclidean geometry
9. Non-existence proofs
10. NP
11. NP completeness
12. NP-hard
13. NSPACE
14. NTIME
15. Occams razor
16. Octonions
17. One-to-one correspondence
18. Open problems
19. Oracles

1. P vs. NP
2. Parallel postulate
3. Parallel simulation
4. Dovetailing simulation
5. Parallelism
6. Parity
7. Parsing
8. Partition problem
9. Paths in graphs
10. Peano arithmetic
11. Penrose tilings
12. Physics analogies
13. Pi formulas
14. Pigeon-hole principle
15. Pilotless planes
16. Pinwheel tilings
17. Planar graph colorability
18. Planarity testing
19. Polyas How to Solve It

26
Concepts, Techniques, Ideas Proofs

1. Polynomial hierarchy
2. Polynomial-time
3. P-time reductions
4. Positional system
5. Power sets
6. Powerset construction
7. Predicate calculus
8. Predicate logic
9. Prime numbers
10. Principia Mathematica
11. Probabilistic TMs
12. Proof theory
13. Propositional logic
14. PSPACE
15. PSPACE completeness
16. Public-key cryptography
17. Pumping theorems
18. Pushdown automata
19. Puzzle solvers

1. Quantifiers
2. Quantum computing
3. Quantum mechanics
4. Quaternions
5. Queue automata
6. Quine
7. Ramanujan identities
8. Ramsey theory
9. Randomness
10. Rational numbers
11. Real numbers
12. Reality surpassing Sci-Fi
13. Recognition and enumeration
14. Recursion theorem
15. Recursive function theory
16. Recursive functions
17. Reducibilities
18. Reductions
19. Regular expressions

1. Rejection
2. Relations
3. Relativity theory
4. Relativization
5. Resource-bounded comput.
6. Respect for the definitions
7. Reusability of space
8. Reversal
9. Reverse Turing test
10. Rices Theorem
11. Riemann hypothesis
12. Riemanns zeta function
13. Robots in fiction
14. Robustness of P and NP
15. Russells paradox
16. Satisfiability
17. Savitchs theorem
18. Schmitt-Conway biprism
19. Scientific method

27
Concepts, Techniques, Ideas Proofs

1. Self compilation
2. Self reproduction
3. Set cover problem
4. Set difference
5. Set identities
6. Set theory
7. Shannon limit
8. Sieve of Eratosthenes
9. Simulated annealing
10. Simulation
11. Skepticism
12. Soundness
13. Space filling polyhedra
14. Space hierarchy
15. Spanning trees
16. Speedup theorems
17. Sphere packing
18. Spherical geometry
19. Standard model

1. Steiner tree
2. Stirlings formula
3. Stored progam
4. String theory
5. Strings
6. Strong AI hypothesis
7. Superposition
8. Super-states
9. Surcomplex numbers
10. Surreal numbers
11. Symbolic logic
12. Symmetric closure
13. Symmetric venn diagrams
14. Technological singularity
15. Theory-reality chasms
16. Thermodynamics
17. Time hierarchy
18. Time/space tradeoff
19. Tinker Toy computers

1. Tradeoffs
2. Transcendental numbers
3. Transfinite arithmetic
4. Transformations
5. Transition function
6. Transitive closure
7. Transitivity
8. Traveling salesperson
9. Triangle inequality
10. Turbulance
11. Turing complete
12. Turing degrees
13. Turing jump
14. Turing machines
15. Turing recognizable
16. Turing reduction
17. Turing test
18. Two-way automata
19. Type errors

28
Concepts, Techniques, Ideas Proofs

• Uncomputable functions
• Uncomputable numbers
• Uncountability
• Undecidability
• Universal Turing machine
• Venn diagrams
• Vertex cover
• Von Neumann architecture
• Von Neumann bottleneck
• Wang tiles cubes
• Zero-knowledge protocols
• .
• .
• .
• .

Make everything as simple as possible, but not
simpler. - Albert Einstein (1879-1955)