Theory of Computation

1
Theory of Computation

• Shyh-Kang Jeng
• Department of Electrical Engineering/
• Graduate Institute of Communication Engineering
• National Taiwan University

2
Chapter Goal

• Investigate the capabilities of computers
• Understand what machines can and cannot do and
  what features are required for machines to reach
  their full potential

3
Functions and Their Computation

• Function
• A correspondence between a collection of possible
  input values and a collection of output values
• Each possible input is assigned a unique output
• Computing the function
• Process of determining the particular output
  value that assigns to a given input

4
Function Computing through Table Lookup
5
Computable and Non-computable Functions

• Computable functions
• Functions whose output values can be determined
  algorithmically from their input values
• Non-computable functions
• Functions that are so complex that there is no
  well-defined, step-by-step process for
  determining their output based on their input
  values
• The computation of these functions lies beyond
  the abilities of any algorithmic system

6
A Turing Machine
7
A Turing Machine

• Control unit
• Read/write head
• Tape and cells
• Alphabet and symbols
• States
• Start state
• Halt state

8
A Turing Machine

• Steps executed by the control unit
• Observe the symbol in the current tape cell
• Write a symbol in the current tape cell
• Move the read-write head one cell to the left
• Move the read-write head one cell to the right
• Can be implemented in a variety of forms
• Todays general-purpose computer (CPU, memory,
  bit patterns of registers, 0 and 1)

9
A Turing Machine for Incrementing a Value
1


1
0
state START
1


1
0
state ADD
10
A Turing Machine for Incrementing a Value
0


1
0
state CARRY
0


1
1
state RETURN
11
A Turing Machine for Incrementing a Value
0


1
1
state RETURN
0


1
1
state HALT
12
A Turing Machine for Incrementing a Value
13
Church-Turing Thesis

• Turing Computable Functions
• Functions that can be computed by a Turing
  machine
• Church-Turing Thesis
• The computable functions and the Turing
  computable functions are considered one and the
  same
• Widely accepted conjecture

14
A Turing Machine

• Captures the essence of computational process
• Its computational power is as great as any
  algorithmic system
• If a problem can not be solved by a Turing
  machine, it can not be solved by any algorithmic
  system
• Represents a theoretical bound on the
  capabilities of actual machines

15
Universal Programming Language

• Most features in high-level languages merely
  enhance convenience rather than contribute to the
  fundamental power of the language
• Universal programming language
• Rich enough to express programs for computing all
  computable functions
• If a problem can not be solved using this
  language, it will be that there is not an
  algorithm for solving the problems

16
A Bare BonesProgramming Language

• Language that isolates a minimal set of
  requirements of a Universal programming
• Data Description Statements
• bit pattern of any length
• variable name and value
• Imperative Statements
• clear name
• incr name
• decr name
• while name not 0 do
• end

17
Copying Today to Tomorrow
18
Computing Z X Y

• copy Y to Z
• while X not 0 do
• incr Z
• decr X
• end

19
Computing XY
20
Universality of Bare Bones

• Using Turing machine to compute
• incr X
• Bare Bones program for the addition function
• Bare Bones language can be used to express
  algorithms for computing all Turing-computable
  functions
• Any computable function can be computed by a
  program written in Bare Bones

21
Halting Problem

• Problem of trying to predict in advance whether a
  program will terminate (or halt) if started under
  certain conditions
• Example
• while X not 0 do
• incr X
• end

22
Testing a Program for Self-Termination
23
The Halting Problem

• Define a function
• Input of encoded self-terminating programs
  produce the output 1
• Input of encoded non-self-terminating programs
  produce the output 0
• Halting problem
• Calculating whether programs ultimately terminate
  after being started from a particular initial
  state

24
The Halting Problem
25
The Halting Problem
26
The Halting Problem
27
Complexity of a Problem

• Considered to be the complexity of the best
  algorithm for solving the problem
• Algorithms time complexity
• Time required for its execution
• Proportional to the number of steps
• Not the same as number of instructions in the
  program
• Algorithms space complexity

28
Time Complexity of a Problem

• Big theta notation for algorithm efficiency
• Big oh notation
• The problem is in O(f(n)) if it has a solution
  whose complexity is Q(f(n)) but could possibly
  have a better solution
• Searching and sorting a list of length n O(lg n)
• Binary search is an optimal solution
• Insertion sort is not an optimal solution

29
Algorithm for Merging Lists
30
Algorithm MergeSort
31
Hierarchy by MergeSort
32
Polynomial Problems

• Problem in O(f(n)), where f(n) is either a
  polynomial or bounded by a polynomial
• The collection of all polynomial problems P
  (Having practical solutions)
• Problems outside P have extremely long execution
  time
• e.g. O(2n) problems listing all subcommittees
  that can be formed from a group of n people

33
Checking Bounds
34
Traveling Salesman Problem
Find a path whose total length does not exceed
his allowed mileage
35
Traveling Salesman Problem

• Traditional solution
• List and compare paths systematically
• Not produce a polynomial time solution
• Nondeterministic algorithm
• pick one of the possible paths, and compute its
  total distance
• if this distance is not greater than the
  allowable mileage then declare a success else
  declare nothing

36
NP Problem

• A problem that can be solved in polynomial time
  by a nondeterministic algorithm
• The class of NP problems NP
• All problems in P are also in NP
• NP-complete problems
• Problems having the property that a polynomial
  time solution for any of them would provide a
  polynomial time solution for all the other
  problems in NP as well

37
Problem Classification
38
Public Key Cryptography

• Keys used for data encryption are not the same as
  those used to decode the encrypted information
• Encryption key
• Can be widely distributed
• Decoding key
• Required to decode encrypted message

39
Public key cryptography

• Key specially generated set of values used for
  encryption
• Public key used to encrypt messages
• Private key used to decrypt messages
• RSA a popular public key cryptographic
  algorithm
• Relies on the (presumed) intractability of the
  problem of factoring large numbers

40
Encrypting the message 10111

• Encrypting keys n 91 and e 5
• 10111two 23ten
• 23e 235 6,436,343
• 6,436,343 91 has a remainder of 4
• 4ten 100two
• Therefore, encrypted version of 10111 is 100.

41
Decrypting the message 100

• Decrypting keys d 29, n 91
• 100two 4ten
• 4d 429 288,230,376,151,711,744
• 288,230,376,151,711,744 91 has a remainder of
  23
• 23ten 10111two
• Therefore, decrypted version of 100 is 10111.

42
Public key cryptography
43
Establishing a RSA public key encryption system