O-Notation

1
O-Notation

• April 23, 2003
• Prepared by Doug Hogan
• CSE 260

2
O-notation The Idea

• Big-O notation is a way of ranking about how much
  time it takes for an algorithm to execute
• How many operations will be done when the
  program is executed?
• Find a bound on the running time, i.e. functions
  that are on the same order.
• We care about what happens for large amounts of
  data ? asymptotic order.

3
O-notation The Idea

• Use mathematical tools to find asymptotic order.
• Real functions to approximate integer functions.
• Depends on some variable, like n or X, which is
  usually the size of an array or how much data is
  going to be processed

4
O-notationThe complicated math behind it all

• Given f and g, real functions of variable x
• First form
• g provides an upper bound for f graph of f
  lies closer to the x axis than g
• More general form
• g provides an upper bound for f graph of f
  lies closer to the x axis than some positive
  multiple (M) of g after some minimum value of x
  (x0).

5
O-notation A graphical view
6
So what does closer to the x-axis MEAN?

• -M g(x) f(x) M g(x)
• But thats absolute value
• f(x) M g(x)

7
Another graphical view
y
M g
f
g
After x0, f(x) M g(x)
Before x0, nothing claimed about fs growth
x
x0
8
Formal Definition

• Let f and g be real-valued functions defined on
  the same set of reals.
• f is of order g, written f(x) O(g(x)), iff
  there exists
• a positive real number M (multiple)
• a real number x0 (starting point)
• such that for all x in the domain of f and g,
  f(x) M g(x) when x gt x0

9
Example

• Use the definition of O-notation to express17x6
  3x3 2x 8 30x6
  for all x gt 1
• M 30
• x0 1
• 17x6 3x3 2x 8 is O(x6)

10
Graphically

• 17x6 3x3 2x 8 is O(x6) M 30 x0 1

30x6
17x6 3x3 2x 8
x6
11
Problem

• Use the definition of O-notation to express
  for all x
  gt 6
• M 45
• x0 6

12
Graphically

• M 45 x0 6

13
Another graphical example
12x3

• f(x) 7x3 - 2x 3

x3
M 12, x0 1 7x3 - 2x 3 is O(x3)
14
Using O-notation

• Order of Power Functions
• For any rational numbers r and s, if r lt s,
• xr is O(xs)
• Order of Polynomial Functions
• If a0, a1,, an are real numbers and an ? 0
• anxn an-1xn-1 a1x a0 is O(xm)
  for
  all m n

15
Examples

• Example
• Find an order for
• f(x) 7x5 5x3 x 4 (all reals x)
• O(x5)
• Is that the only answer?
• No
• But its the best

16
Showing that a function is NOT Big-O of another

• Show that x2 is not O(x).
• Arguing by contradiction. Suppose not, that x2
  is O(x).
• By definition of O(), then there exist
• a positive real number M
• a real number x0
• such that x2 M x for all x gt x0 (1)

17
Showing that a function is NOT Big-O of another,
ctd.

• Let x be a positive real number greater than both
  M and x0, i.e. xgtM and xgtx0.
• Then by multiplying both sides of xgtM by x,
  xxgtMx.
• Since x is positive, x2gtMx.
• So there is a real number xgtx0 s.t. x2gtMx.
• This contradicts (1) above. So, the supposition
  is false and thus x2 is not O(x). ?

18
Generalization

• If a0, a1,, an are real numbers and an ? 0
• anxn an-1xn-1 a1x a0 is NOT O(xm)
  
  for all m lt n

19
Best approximation

• Definition
• Suppose S is a set of functions from a subset of
  R to R and f is a R-gtR function.
• Function g is a best big-O approximation for f in
  S iff
• f(x) is O(g(x))
• for any h in S, if f(x) is O(h(x)), then g(x) is
  O(h(x)).

20
Problem

• Find best big-O approx for f(x) 5x3 2x 1
• By thm. on polynomial orders,
• f(x) is O(xn) for all n 3
• By previous property,
• f(x) is NOT O(xm) for all m lt 3
• So O(x3) is the best approximation.

21
O-Arithmetic

• Let f and g be functions and k be a constant.
• O(kf) O(f)
• O(f g) O(f) O(g)
• O(f/g) O(f) / O(g)
• O(f) O(g) iff f dominates g
• O(f g) MaxO(f), O(g)

(Headington 546)
22
Dominance

• Let f and g be functions and a,b,m,n be
  constants.
• xx dominates x!
• x! dominates ax
• ax dominates bx if a gt b
• ax dominates xn if n gt m
• x dominates logax if a gt 1
• logax dominates logbx if b gt a gt 1
• logax dominates 1 if a gt 1

(Headington 547)
23
Works Cited

• Epp, Susanna. Discrete Mathematics with
  Applications. 2nd Ed. Belmont, CA Brooks, 1995.
• Headington, Mark A., and David Riley. Data
  Abstraction and Structures using C. Lexington,
  MA Heath, 1994.