Title: Algorithms
1Algorithms
Tradeoff Execution speed vs. solution quality
Short sweet
Quick dirty
Slowly but surely
Too little, too late
2Computational Complexity
Problem Avoid getting trapped in local minima
Global optimum
3Approximation Algorithms
- Idea Some intractable problems can be
efficiently approximated within close to optimal! - Fast
- Simple heuristics (e.g., greed)
- Provably-good approximations
- Slower
- Branch-and-bound approaches
- Integer Linear Programming relaxation
4Approximation Algorithms
- Wishful
- Simulated annealing
- Genetic algorithms
5Minimum Vertex Cover
Minumum vertex cover problem Given a graph, find
a minimum set of vertices such that each edge is
incident to at least one vertex of these
vertices. Example
Input graph
Heuristic solution
Optimal solution
- Applications bioinformtics, communications,
- civil engineering, electrical engineering, etc.
- One of Karps original NP-complete problems
6Minimum Vertex Cover Examples
7Approximate Vertex Cover
Theorem The minimum vertex cover problem is
NP-complete (even in planar graphs of max degree
3). Theorem The minimum vertex cover problem
can be solved exactly within exponential time
nO(1)2O(n). Theorem The minimum vertex cover
problem can not be approximated within 1.36OPT
unless PNP. Theorem The minimum vertex cover
problem can be approximated (in linear time)
within 2OPT. Idea pick an edge, add its
endpoints, and repeat.
8Approximate Vertex Cover
- Algorithm Linear time 2OPT approximation for
the minimum vertex cover problem - Pick random edge (x,y)
- Add x,y to the heuristic solution
- Eliminate x and y from graph
- Repeat until graph is empty
Best approximation bound known for VC!
Idea one of x,y must be in any optimal
solution. Þ Heuristic solution is no worse than
2OPT.
9Maximum Cut
Maximum cut problem Given a graph, find a
partition of the vertices maximizing the of
crossing edges. Example
cut size 2
cut size 4
cut size 5
Input graph
Heuristic solution
Optimal solution
- Applications VLSI circuit design, statistical
physics, - communication networks.
- One of Karps original NP-complete problems.
10Maximum Cut
Theorem Karp, 1972 The minimum vertex cover
problem is NP-complete. Theorem The maximum
cut problem can be solved in polynomial time
for planar graphs. Theorem The maximum cut
problem can not be approximated within
17/16OPT unless PNP. Theorem The maximum cut
problem can be approximated in polynomial time
within 2OPT. Theorem The maximum cut problem
can be approximated in polynomial time within
1.14OPT.
1.0625OPT
11Maximum Cut
- Algorithm 2OPT approximation for maximum cut
- Start with an arbitrary node partition
- If moving an arbitrary node across the partition
- will improve the cut, then do so
- Repeat until no further improvement is possible
cut size 2
cut size 3
cut size 5
Input graph
Heuristic solution
Optimal solution
Idea final cut must contain at least half of all
edges. Þ Heuristic solution is no worse than
2OPT.
12Approximate Traveling Salesperson
Traveling salesperson problem given a pointset,
find shortest tour that visits every point
exactly once.
- 2OPT metric TSP heuristic
- Compute MST
- T Traverse MST
- S shortcut tour
- Output S
Analysis
S lt T
MST lt OPT TSP
2
2
triangle inequality!
TSP minus an edge is a spanning tree
T covers minimum spanning tree twice
13Non-Approximability
- NP transformations typically do not preserve the
- approximability of the problem!
- Some NP-complete problems can be approximated
- arbitrarily close to optimal in polynomial time.
- Theorem Geometric TSP can be approximated in
- polynomial time within (1e)OPT for any egt0.
- Other NP-complete problems can not be
approximated - within any constant in polynomial time (unless
PNP). - Theorem General TSP can not be approximated
- efficiently within KOPT for any Kgt0 (unless
PNP).
14Graph Isomorphism
Definition two graphs G1(V1,E1) and G2(V2,E2)
are isomorphic iff bijection ƒV1V2 such
that "vi,vjÎV1 (vi,vj)ÎE1 Û
(ƒ(vi),ƒ(vj))ÎE2 Isomorphism º edge-preserving
vertex permutation Problem are two given graphs
isomorphic?
Note Graph isomorphism ÎNP, but not known to be
in P
15Graph Isomorphism
16Zero-Knowledge Proofs
Idea proving graph isomorphism without
disclosing it! Premise Everyone knows G1 and G2
but not must remain secret!
Create random G G1 Note is
() Broadcast G Verifier asks for or
Broadcast or Verifier checks GG1 or
GG2 Repeat k times Þ Probability of cheating 2-k
Approximating a proof!
17Zero-Knowledge Proofs
Idea prove graph 3-colorable without disclosing
how! Premise Everyone knows G1 but not its
3-coloring ? which must remain secret!
?'
Create random G2 G1 Note 3-coloring ?'(G2)
is (?(G1)) Broadcast G2 Verifier asks for
or ?' Broadcast or ?' Verifier checks G1G2
or ?'(G2) Repeat k times Þ Probability of
cheating 2-k
?
Interactive proof!
18Zero-Knowledge Caveats
- Requires a good random number generator
- Should not use the same graph twice
- Graphs must be large and complex enough
- Applications
- Identification friend-or-foe (IFF)
- Cryptography
- Business transactions
19Zero-Knowledge Proofs
Idea prove that a Boolean formula P is
satisfiable without disclosing a satisfying
assignment! Premise Everyone knows P but not its
secret satisfying assignment V !
P (xyz)(x'y'z)(x'yz')
ƒ
Convert P into a graph 3-colorability
instance G ƒ(P) Publically broadcast ƒ and
G Use zero-knowledge protocol to show that G
is 3-colorable Þ P is satisfiable iff G is
3-colorable Þ P is satisfiable with probability
1-2-k
G
Interactive proof!
20Interactive Proof Systems
- Prover has unbounded power and may be malicious
- Verifier is honest and has limited power
- Completeness If a statement is true, an honest
verifier - will be convinced (with high prob) by an honest
prover. - Soundness If a statement is false, even an
omnipotent - malicious prover can not convince an honest
verifier that - the statement is true (except with a very low
probability). - The induced complexity class depends on the
verifiers - abilities and computational resources
- Theorem For a deterministic P-time verifier,
class is NP. - Def For a probabilistic P-time verifier, induced
class is IP. - Theorem Shamir, 1992 IP PSPACE
21Concepts, Techniques, Idea Proofs
- 2-SAT
- 2-Way automata
- 3-colorability
- 3-SAT
- Abstract complexity
- Acceptance
- Ada Lovelace
- Algebraic numbers
- Algorithms
- Algorithms as strings
- Alice in Wonderland
- Alphabets
- Alternation
- Ambiguity
- Ambiguous grammars
- Analog computing
- Anisohedral tilings
- Aperiodic tilings
- Approximate min cut
- Approximate vertex cover
- Approximations
- Artificial intelligence
- Asimovs laws of robotics
- Asymptotics
- Automatic theorem proving
- Autonomous vehicles
- Axiom of choice
- Axiomatic method
- Axiomatic system
- Babbages analytical engine
- Babbages difference engine
- Bin packing
- Binary vs. unary
- Bletchley Park
- Bloom axioms
- Boolean algebra
- Boolean functions
- Bridges of Konigsberg
- Busy beaver problem
- C programs
- Canonical order
- Cantor dust
- Cantor set
- Cantors paradox
- CAPCHA
- Cardinality arguments
- Cartesian coordinates
- Cellular automata
- Chaos
- Chatterbots
- Chess-playing programs
- Chinese room
- Chomsky hierarchy
- Chomsky normal form
- Chomskyan linguistics
- Christofides heuristic
- Church-Turing thesis
22Concepts, Techniques, Ideas Proofs
- Clique problem
- Cloaking devices
- Closure properties
- Cogito ergo sum
- Colorings
- Commutativity
- Complementation
- Completeness
- Complexity classes
- Complexity gaps
- Complexity Zoo
- Compositions
- Compound pendulums
- Compressibility
- Computable functions
- Computable numbers
- Computation and physics
- Computation models
- Computational complexity
- Computational universality
- Computer viruses
- Concatenation
- Co-NP
- Consciousness and sentience
- Consistency of axioms
- Constructions
- Context free grammars
- Context free languages
- Context sensitive grammars
- Context sensitive languages
- Continuity
- Continuum hypothesis
- Contradiction
- Contrapositive
- Cooks theorem
- Countability
- Counter automata
- Counter example
- Crossing sequences
- Cross-product construction
- Cryptography
- DARPA Grand Challenge
- DARPA Math Challenges
- De Morgans law
- Decidability
- Deciders vs. recognizers
- Decimal number system
- Decision vs. optimization
- Dedekind cut
- Denseness of hierarchies
- Derivations
- Descriptive complexity
- Diagonalization
- Digital circuits
- Diophantine equations
- Disorder
- DNA computing
23Concepts, Techniques, Ideas Proofs
- Dovetailing
- DSPACE
- DTIME
- EDVAC
- Elegance in proof
- Encodings
- Enigma cipher
- Entropy
- Enumeration
- Epsilon transitions
- Equivalence relation
- Euclids Elements
- Euclids axioms
- Euclidean geometry
- Eulers formula
- Eulers identity
- Eulerian tour
- Existence proofs
- Exoskeletons
- Exponentiation
- EXPSPACE
- EXPSPACE
- EXPSPACE complete
- EXPTIME
- EXPTIME complete
- Extended Chomsky hierarchy
- Fermats last theorem
- Fibonacci numbers
- Final states
- Finite automata
- Finite automata minimization
- Fixed-point theorem
- Formal languages
- Formalizations
- Four color problem
- Fractal art
- Fractals
- Functional programming
- Fundamental thm of Arith.
- Gadget-based proofs
- Game of life
- Game theory
- Game trees
- Gap theorems
- Garey Johnson
- General grammars
- Generalized colorability
- Generalized finite automata
- Generalized numbers
- Generalized venn diagrams
- Generative grammars
- Genetic algorithms
- Geometric / picture proofs
- Godel numbering
- Godels theorem
- Goldbachs conjecture
- Golden ratio
24Concepts, Techniques, Ideas Proofs
- Grammars
- Grammars as computers
- Graph cliques
- Graph colorability
- Graph isomorphism
- Graph theory
- Graphs
- Graphs as relations
- Gravitational systems
- Greibach normal form
- Grey goo
- Guess-and-verify
- Halting problem
- Hamiltonian cycle
- Hardness
- Heuristics
- Hierarchy theorems
- Hilberts 23 problems
- Hilberts program
- Historical perspectives
- Household robots
- Hung state
- Hydraulic computers
- Hyper computation
- Hyperbolic geometry
- Hypernumbers
- Identities
- Immermans Theorem
- Incompleteness
- Incompressibility
- Independence of axioms
- Independent set problem
- Induction its drawbacks
- Infinite hotels applications
- Infinite loops
- Infinity hierarchy
- Information theory
- Inherent ambiguity
- Intelligence and mind
- Interactive proofs
- Intractability
- Irrational numbers
- JFLAP
- Karps paper
- Kissing number
- Kleene closure
- Knapsack problem
- Lambda calculus
- Language equivalence
- Law of accelerating returns
- Law of the excluded middle
- Lego computers
- Lexicographic order
- Linear-bounded automata
- Local minima
- LOGSPACE
- Low-deg graph colorability
25Concepts, Techniques, Ideas Proofs
- Machine equivalence
- Mandelbrot set
- Manhattan project
- Many-one reduction
- Matiyasevichs theorem
- Mechanical calculator
- Mechanical computers
- Memes
- Mental poker
- Meta-mathematics
- Millennium Prize
- Minimal grammars
- Minimum cut
- Modeling
- Multiple heads
- Multiple tapes
- Mu-recursive functions
- MAD policy
- Nanotechnology
- Navier-Stokes equations
- Neural networks
- Newtonian mechanics
- NLOGSPACE
- Non-approximability
- Non-closures
- Non-determinism
- Non-Euclidean geometry
- Non-existence proofs
- NP
- NP completeness
- NP-hard
- NSPACE
- NTIME
- Occams razor
- Octonions
- One-to-one correspondence
- Open problems
- Oracles
- P vs. NP
- Parallel postulate
- Parallel simulation
- Dovetailing simulation
- Parallelism
- Parity
- Parsing
- Partition problem
- Paths in graphs
- Peano arithmetic
- Penrose tilings
- Physics analogies
- Pi formulas
- Pigeon-hole principle
- Pilotless planes
- Pinwheel tilings
- Planar graph colorability
- Planarity testing
- Polyas How to Solve It
26Concepts, Techniques, Ideas Proofs
- Polynomial hierarchy
- Polynomial-time
- P-time reductions
- Positional system
- Power sets
- Powerset construction
- Predicate calculus
- Predicate logic
- Prime numbers
- Principia Mathematica
- Probabilistic TMs
- Proof theory
- Propositional logic
- PSPACE
- PSPACE completeness
- Public-key cryptography
- Pumping theorems
- Pushdown automata
- Puzzle solvers
- Quantifiers
- Quantum computing
- Quantum mechanics
- Quaternions
- Queue automata
- Quine
- Ramanujan identities
- Ramsey theory
- Randomness
- Rational numbers
- Real numbers
- Reality surpassing Sci-Fi
- Recognition and enumeration
- Recursion theorem
- Recursive function theory
- Recursive functions
- Reducibilities
- Reductions
- Regular expressions
- Rejection
- Relations
- Relativity theory
- Relativization
- Resource-bounded comput.
- Respect for the definitions
- Reusability of space
- Reversal
- Reverse Turing test
- Rices Theorem
- Riemann hypothesis
- Riemanns zeta function
- Robots in fiction
- Robustness of P and NP
- Russells paradox
- Satisfiability
- Savitchs theorem
- Schmitt-Conway biprism
- Scientific method
27Concepts, Techniques, Ideas Proofs
- Self compilation
- Self reproduction
- Set cover problem
- Set difference
- Set identities
- Set theory
- Shannon limit
- Sieve of Eratosthenes
- Simulated annealing
- Simulation
- Skepticism
- Soundness
- Space filling polyhedra
- Space hierarchy
- Spanning trees
- Speedup theorems
- Sphere packing
- Spherical geometry
- Standard model
- Steiner tree
- Stirlings formula
- Stored progam
- String theory
- Strings
- Strong AI hypothesis
- Superposition
- Super-states
- Surcomplex numbers
- Surreal numbers
- Symbolic logic
- Symmetric closure
- Symmetric venn diagrams
- Technological singularity
- Theory-reality chasms
- Thermodynamics
- Time hierarchy
- Time/space tradeoff
- Tinker Toy computers
- Tradeoffs
- Transcendental numbers
- Transfinite arithmetic
- Transformations
- Transition function
- Transitive closure
- Transitivity
- Traveling salesperson
- Triangle inequality
- Turbulance
- Turing complete
- Turing degrees
- Turing jump
- Turing machines
- Turing recognizable
- Turing reduction
- Turing test
- Two-way automata
- Type errors
28Concepts, Techniques, Ideas Proofs
- Uncomputable functions
- Uncomputable numbers
- Uncountability
- Undecidability
- Universal Turing machine
- Venn diagrams
- Vertex cover
- Von Neumann architecture
- Von Neumann bottleneck
- Wang tiles cubes
- Zero-knowledge protocols
- .
- .
- .
- .
Make everything as simple as possible, but not
simpler. - Albert Einstein (1879-1955)