CMPT 285 Discrete Mathematics

1
CMPT 285Discrete Mathematics

• by Kossi Edoh
• Department of Computer Science
• Montclair State University

2
Section 2.1 Algorithms
An algorithm is a finite set of precise
instructions for performing a computation or
solving a problem. A pseudocode is an
intermediate step between English and a
programming language description of an algorithm
3
Example pseudocode
Example Find the largest value in a finite
sequence of integers. Procedure
max(a1, a2, , an integers) max a1 for i
2 to n if max lt ai then max ai
max is the largest element
4
Some common properties of algorithms

• Input Input values
• Output Output results
• Definiteness Steps must be defined precisely
  
• Correctness Correct output for different set
  of inputs
• Finiteness Produce result after finite
  steps
• Effectiveness Each step of the algorithm
  performed
• exactly in a finite
  time
• Generality The procedure must be
  applicable to
• problems of the same
  form

5
Linear search
procedure linear_search(x integer, a1, a2, ,
an
distinct integers) i 1 while( i ? n and x ?
ai) i i 1 if i ? n then
location i else location 0 location is
the subscript of term that equals x, or is 0 if
x is not found
6
The Binary Search Algorithm
Procedure binary_search(x integer, a1, a2, ,
an increasing
integers) i 1 i is left endpoint of
search interval j n j is right endpoint
of search interval while i ? j begin
m ?( i j ) /2? if x ? am
then i m 1 else j m
end if x ai then location i
else location 0
7
Insertion Sort
Procedure insertion_sort(a1, a2, , an real
numbers with n ? 2) for j
2 to n begin i 1 while aj gt
ai i i 1 m aj
for k 0 to j i -1 aj-k
aj-k-1 aj m end a1, a2, , an are
sorted
8
Section 2.2 Growth functions

• The concept that g grows at least as fast as f.
• In comparing the complexity of algorithms
• We only care about the behaviour for large n
• Even bad algorithms can be used to solve small
• problems
• Ignore implementation details such as loop
• counter problems increments, etc.

9
The big O notation
The Big-O Notation Definition
Let f and g be functions from N to R. Then g
grows asymptotically and dominates f, denoted
f is O(g) or
f is big-O of g, or
f is of order g, iff ?k ?C ?n
n gt k ? f(n) ? Cg(n)
10
Example
Example Show that f(x) 2x2 3x 1 is
O(x2) 0 ? 2x2 3x 1 lt 2x2 3x2 x2 6x2
whenever x gt 1 with C 6. Note the value of k
and C is not unique Example 7x2 is O(x3)
with k 7 and C 1
11
Little o

• f(n)
• If lim ----- 0 then f is o(g) (called little-o
  of g)
• n?? g(n)
• Theorem If f is o(g) then f is O(g)
• It is usually easier to prove f is o(g)
• using the theory of limits
• using LHospitals rule
• using the properties of logarithms
• etc.

Example 3n 5 is O(n2). Lim (3n5)/n2 0 Hence
3n 5 is o(n2) and so it is O(n2)
12
Complexity class

• Note that O(g) is a set called a complexity
  class.
• It contains all the functions which g dominates
• Theorem f is o(g) means f ? O(g)
• Properties of Big O
• f is O(g) iff O(f) ? O(g)
• if f is O(g) and g is O(f) then O(f) O(g)
• the set O(g) is closed under addition
• the set O(g) is closed under multiplication by
  a scalar
• if f is O(g) and g is O(h), then f is O(h)

13
Important Complexity Classes

• O(1) ? O(log n) ? O(n) ? O(n log n) ? O(n2)
• O(nj) ? O(cn) ? O(n!) where j gt 2 and c gt 1.
• Suppose that f1(x) is O(g1(x)) and f2(x) is
  O(g2(x)).
• Then (f1 f2)(x) is O(max(g1(x),g2(x))
• (f1f2)(x) is O(g1(x)g2(x))

14
Examples
Example Suppose algorithm 1 has complexity n2
n 1 algorithm 2 has
complexity n2/2 3n 2 then both are O(n2).
Algorithm 2 has a smaller leading coefficient
and will be faster for large problems. Hence we
write Algorithm 1 has complexity n2
O(n) Algorithm 2 has complexity n2/2 O(n)
15
Example

• Example Find the complexity class of the
  function
• (n n! 3n2 3n100)(nn n2n)
• Solution
• This means to simplify the expression
• Throw out stuff which you does not grow fast
• Eliminate 3n2 and 3n100 since they grow
  slower
• than n!
• Eliminate n2n since it grows slower than
  nn
• Hence the complexity class is O(n n! nn)

16
Big Omega and Theta
f(x) is ?(g(x)) if the are positive constants C
and k such that f(x) ?C g(x) whenever
x gt k. f(x) is ?(g(x)) if f(x) is ?(g(x)) and
f(x) is O(g(x)) and we say f(x) is of order
g(x).
17
Section 2.3
Efficiency of algorithms are determined by Time
Complexity The approximate number of
operations required to solve a problem of size
n. Space Complexity The approximate memory
required to solve a problem of
size n.
18
Time complexity

• Time Complexity
• Use the Big-O notation
• Ignore house keeping
• Count the expensive operations only
• Basic operations
• searching algorithms key comparisons
• sorting algorithms list component comparisons
• numerical algorithms floating point operations
  

19
Space Complexity

• Space Complexity
• This is tied with the data structures used to
• implement the algorithm

20
Examples complexity
Example The time complexity of the algorithm for
finding the largest element is f(n) 2(n-1)
1 comparisons O(n) Example The time complexity
of the linear search algorithm is f(n) 2n 1
1(if element is not in the list) The two
examples are for the worst case
maximum number of operations
21
Binary search
Assume n 2k elements k log
n There are 2k 2 comparisons, the time
complexity is 2 log n 2
O(log n)
22
Example
If A is upper triangular then number of
matrix-vector multiplications is 1 2 3
n n2/2 n/2
n2/2 O(n) Bubble
sort procedure bubble(n,L) for i
from n-1 to 1 by 1 do for j from 1
to I do if L(j) gt L(j1) do
swap L(j1) L(j1) L(j)
L(j) swap end do
end do end do
23
Average case
Average Case mean number of operations assuming
input probability
distribution. Example Linear search If
x is the first element in the list then
with have 3 comparisons If x is the nth
element in the list then with have
2n 1 comparisons Average is 3 5 7
2n 1/2 2(1 2 n) n/2
2 n(n1)/2n
n 2
which is O(n)
24
Example continued
Average Case mean number of operations assuming
input probability distribution. Example
Multiply an n x n matrix A by a scalar c to
Produce the matrix B procedure(n,
c, A, B) for i from 1 to n do
for j from 1 to n do
B(i,j) c A(i,j)
end do end
do n2 multiplications, the complexity is O(n2)
25
Table
Complexity Terminology O(1)
Constant complexity O(log n) Logarithmic
complexity O(n) Linear complexity O(n log
n) n log n complexity O(nk) Polynomial
complexity O(bn), where bgt1 Exponential
complexity O(n!) Factorial complexity
26
Bubble sort
Bubbles the largest element to the top by
starting from the bottom swap elements until
the largest is at the top. Bubble the second
largest to the position below the top continue
until the list is sorted. n- 1 comparisons on
the first pass n- 2 comparisons on the second
pass 1 comparison
on the last pass Total (n-1)(n-2)(n-3) 1
O(n2).
27
Problem types
problem
solvable unsolvable
no algorithm, halting tractable
intractable problem (at
most O(nk) with higher type
algorithm, complexity class
NP) Class NP-complete problems have the property
that if any of them can be solved in O(nk) then
all can be solved in O(nk) worst time.
28
Section 2.7 Matrices
Definition A matrix is a rectangular array of
numbers. A matrix with m rows and n columns is
called an m x n matrix. An n x n matrix is
called a square matrix. Let a11 a12 ? a1n
a21 a22 ? a2n A
an1 an2 ? ann The (i,j)th element or entry of
A is the element aij, that is, the number in the
ith row and the jth column of A. A shorthand
notation is A aij
29
Operations on Matrices
Definition Let A aij and b bij be mxn
matrices A B aij
bij Definition Let A be an mxk matrix and B be
a kxn matrix. The product of A and B denoted AB,
is an mxn matrix with (i,j)th entry as

k AB cij, cij ai1b1j ai2b2j
aikbkj ? aitbtk

t1 Transpose of a matrix The Identity
matrix Symmetric matrices
30
Zero-one Matrices
A matrix with entries that are either 0 or 1 is
called a zero-one matrix Let A aij and b
bij 1 if aij
bij 1 A ? B aij ? bij 0
otherwise 1 if aij1 or bij 1
A ? B aij ? bij 0 otherwise
31
Boolean Product
Let A aij be an m x k zero-one matrix and B
bij be A k x n zero-one matrix. The Boolean
product of A and B, denoted by A?B , is the m x
n matrix with (i,j)th entry cij where
cij (ai1?b1j)?(ai2? b2j) ? ?(aik? bkj).