The Growth of Functions

1
The Growth of Functions

• Rosen 2.2

2
Basic Rules of Logarithms
If logz (x) a, then x za

• logz (xy)
• logz (x/y)
• logz (xy)
• If x y
• If x lt y
• logz (-x) is undefined

• logz (x) logz (y)
• logz (x) - logz (y)
• ylogz (x)
• then logz (x) logz (y)
• then logz (x) lt logz (y)

3
Growth

• If f is a function from Z or R to R, how can we
  quantify the rate of growth and compare rates of
  growth of different functions?
• Possible problem Whether f(n) or g(n) is larger
  at any point may depend on value of n.
• For example n2 gt 100n if n gt 100

4
How to quantify growth as n gets bigger?
5
Big-O Notation

• Let f and g be functions from the set of integers
  or the set of real numbers to the set of real
  numbers. We say that f(x) is O(g(x)) if there
  are constants C?N and k?R such that
• f(x) ? Cg(x) whenever x gt k.
• We say f(x) is big-oh of g(x).
• The intuitive meaning is that as x gets large,
  the values of f(x) are no larger than a constant
  time the values of g(x), or f(x) is growing no
  faster than g(x).
• The supposition is that x gets large, it will
  approach a simplified limit.

6
Show that 3x32x27x9 is O(x3)

• Proof We must show that ? constants C?N and k?R
  such that 3x32x27x9 ? Cx3 whenever x gt k.
• Choose k 1 then
• 3x32x27x9 ? 3x32x37x39x3 21x3
• So let C 21.
• Then 3x32x27x9 ? 21 x3 when x ? 1.

7
Show that n! is O(nn)

• Proof We must show that ? constants C?N and k?R
  such that n! ? Cnn whenever n gt k.
• n! n(n-1)(n-2)(n-3)(3)(2)(1)
• ? n(n)(n)(n)(n)(n)(n) n times
• nn
• So choose k 0 and C 1

8
General Rules

• Multiplication by a constant does not change the
  rate of growth. If f(n) kg(n) where k is a
  constant, then f is O(g) and g is O(f).
• The above means that there are an infinite number
  of pairs C,k that satisfy the Big-O definition.
• Addition of smaller terms does not change the
  rate of growth. If f(n) g(n) smaller order
  terms, then f is O(g) and g is O(f).
• Ex. f(n) 4n6 3n5 100n2 2 is O(n6).

9
General Rules (cont.)

• If f1(x) is O(g1(x)) and f2(x) is O(g2(x)), then
  f1(x)f2(x) is O(g1(x)g2(x)).
• Examples
• 10xlog2x is O(xlog2x)
• n!6n3 is O(n!n3)
• O(nn3)

10
Examples

• f(x) 10 is O(1)
• f(x) x2 x 1 is O(x2)
• f(x) 2x5 100 x3 xlogx is O(x5)
• f(x) 2n n10 is O(2n)
• How would you prove this?

11
Prove that n10 is O(2n )

• Proof We must show that ? constants C?N and k?R
  such that n10 ? C2n whenever n gt k.
• Take log2 of both expressions.
• log22n nlog22 n, log2n10 10log2n
• When is 10log2n lt n? or n/log2n gt 10?
• 2/1 2, 4/2 2, 8/3 ? 2.67, 16/4 4
• 32/5 6.2, 64/6 ? 10.67
• For n 64, 264 gt 6410. So, if we choose then k
  64, C 1, then n10 ? 12n whenever n gt 64.

12
Example Big-Oh Not Symmetric

• Order matters in big-oh. Sometimes f is O(g) and
  g is O(f), but in general big-oh is not
  symmetric.
• Consider f(n) 4n and g(n) n2. f is O(g).
• Can we prove that g is O(f)? Formally, ?
  constants C?N and k?R such that n2 ? C4n
  whenever n gt k?
• No. To show this, we must prove that negation is
  true for all C and k. ?C?N, ?k?R, ?ngtk such that
  n2 gt C4n.

13
?C?N, ?k?R, ?ngtk such that n2 gt 4nC.

• To prove that negation is true, start with
  arbitrary C and k. Must show/construct an ngtk
  such that n2 gt 4nC
• Easy to satisfy n gt k, then
• To satisfy n2gt4nC, divide both sides by n to get
  ngt4C. Pick n max(4C1,k1), which proves the
  negation.

14
(No Transcript)
15
Is 2n O(n!)?

• We must show that ? constants C?N and k?Z such
  that 2n ? Cn! whenever n gt k.
• 2n 2(2)(2)(2)(2)(2) n times
• ? n(n-1)(n-2)(3)(2)(1) n! if n 4
• So let C 1 and k 3.

16
Is 2n O(n!)?

• Note that we could also choose k 1 and C 2
• Since
• 20 ? 20! 2
• 21 ? 21! 2
• 22 ? 22! 4
• 23 ? 23! 12

17
Is f(x)(x21)/(x1) O(x)?

• We must show that ? constants C?N and k?R such
  that f(x) ? Cx whenever x gt k.

When x gt 1 (Why?) Therefore let k1, C 1
(x21)/(x1) ? x when x gt 1
18
Hierarchy of functions
n?n
n
3?n
nlog2n
?n
log2n
2n
1
n!
n2
nn
n3
19
Hierarchy of functions
1
n?n
n
3?n
nlog2n
?n
log2n
2n
n!
n2
nn
n3
20
Hierarchy of functions
1, log2n
n?n
n
3?n
nlog2n
?n
2n
n!
n2
nn
n3
21
Hierarchy of functions
1, log2n, 3?n
n?n
n
nlog2n
?n
2n
n!
n2
nn
n3
22
Hierarchy of functions
1, log2n, 3?n, ?n
n?n
n
nlog2n
2n
n!
n2
nn
n3
23
Hierarchy of functions
1, log2n, 3?n, ?n, n
n?n
nlog2n
2n
n!
n2
nn
n3
24
Hierarchy of functions
1, log2n, 3?n, ?n, n, nlog2n
n?n
2n
n!
n2
nn
n3
25
Hierarchy of functions
1, log2n, 3?n, ?n, n, nlog2n, n?n
2n
n!
n2
nn
n3
26
Hierarchy of functions
1, log2n, 3?n, ?n, n, nlog2n, n?n, n2
2n
n!
nn
n3
27
Hierarchy of functions
1, log2n, 3?n, ?n, n, nlog2n, n?n, n2, n3
2n
n!
nn
28
Hierarchy of functions
1, log2n, 3?n, ?n, n, nlog2n, n?n, n2, n3, 2n
n!
nn
29
Hierarchy of functions
1, log2n, 3?n, ?n, n, nlog2n, n?n, n2, n3, 2n,
n!, nn
Each one is Big-Oh of any function to its right
30
Prove that log10x is O(log2x)

• First we will prove the following lemma
• Lemma log10x clog2x where c is a constant.
• Proof
• Let y log2x.
• Then 2y x and log102y log10x.
• log102y ylog102 log10x. But since y log2x,
  this means that
• log2xlog102 log10x. Therefore c log102

31
To Prove that log10x is O(log2x)

• We must show that ? constants C?N and k?R such
  that log10x ? Clog2x whenever x gt k.
• From the lemma log102log2x log10x so choose C
  log102, k0

32
Prove log(n!) is O(nlogn)

• We must show that ? constants C?N and k?R such
  that logn! ? Cnlogn whenever x gt k.
• We know that n! ? nn so
• log(n!) ? log(nn) nlogn
• So choose k 1, C 1

33
Time Complexity

• We can use Big-O to find the time complexity of
  algorithms (i.e., how long they take in terms of
  the number of operations performed).
• There are two types of complexity normally
  considered.
• Worst-case complexity. The largest number of
  operations needed for a problem of a given size
  in the worst case. The number of operations that
  will guarantee a solution.
• Average-case complexity. The average number of
  operations used to solve a problem over all
  inputs of a given size.

34
Complexity of the Linear Search Algorithm

• Worst-case complexity
• The algorithm loops over all the values in a list
  of n values.
• At each step, two comparisons are made. One to
  see whether the end of the loop is reached, and
  one to compare the search element x with the
  element in the list.
• If x is equal to list element ai, then 2i
  comparisons are made.
• If x is not in the list, then 2n comparisons are
  made.
• The worst-case complexity is thus 2n and is O(n).

35
Complexity of the Linear Search Algorithm

• Average-case complexity
• The algorithm loops over all the values in a list
  of n values.
• At each step, two comparisons are again made.
• On average, the number of comparisons is
• 2 4 6 . (2n)
• n
• Whats the numerator?

Average case is O(n)
36
Complexity of Pair-wise Correlation

• Assume that there are n elements to correlate