Definition: Given a function f(n),

1

• Definition Given a function f(n),
• O(f(n)) is the set of all functions g(n)
• such that for all n gt 0
• g(n) lt cf(n)
• for some constant c.
• Example
• Let T(n)10n23. Is T(n) in O(n2)?
• Is there a c such that 10n23 lt cn2?

2

• Is there a c such that for all n gt 0,
• 10n23 lt cn2?
• If n1, then 13 lt c1, 13 lt c
• If n2, then 43 lt c4, 10.75 lt c
• If n3, then 93 lt c9, 10.34 lt c
• If n4, then 163 lt c16, 10.19 lt c
• So perhaps 13 works -- construct a proof that it
  works for all n!
• Thm For all whole numbers n, if n gt 0 then
• 10n23 lt 13n2.

3

• Too painful to calculate whether a function g is
  in O(f(n)) every time, but fortunately, its
  pretty easy to determine
• 1. f(n)c1nk1 is in O(nk2) for any k2 gt k1.
• 10n2 is in O(n2), also O(n3), O(n4)
• 13n0 is in O(1), also O(n), O(n2), etc.
• 2. If T(n)f(n)g(n) and both f and g are in
  O(h(n)) then T is in O(h(n)).
• T(n)10n2 13
• f(n)10n2 is in O(n2)
• g(n)13 is in O(n2)
• So T(n) is in O(n2)

4

• So a polynomial
• P(n) c0 c1n c2n2 cknk
• is in O(nk).
• g(n)156n13256n7123468n3?
• 3. O(c1) O(c2)
• 4. O(cf(n)) O(f(n))
• 5. O(f(n)c) O(f(n))
• 6. O(log_k(n)) O(log_c(n))
• log_2(n) log_10(n)log_2(10)
• Summary constants dont matter

5
(define (times-1 a n) (if ( n 0) 0
( a (times-1 a (- n 1)))))) T(0)
c T(n1) T(n) d T(0) O(1)
T(n1) T(n) O(1)
6
T(0) c T(n1) T(n) d T(n) T(n-1) d
T(n-2) d d ... T(n-n) d
d ... d (n d's) T(0) dn c
dn O(n)
7
(define (fast-times a n) (if ( n 0) 0 (if
(even? n) (fast-times (double a) (halve n)))
( a (fast-times a (- n 1)))))) T(0) c T(n)
T(n/2) c if n is even, n gt 0 T(n) T(n-1)
c if n is odd
8
Easy case b is a power of 2 T(0) c T(n)
T(n/2) c if n is even, n gt 0 T(n) T(n-1)
c if n is odd n 2(log n) T(n) T(n/2)
c T(n/4) c c T(n/8) c c
c ... T(1) (log n) c T(0)
(1 log n) c (2 log n) c
O(log n)
9
n times-1 fast-times 2 0.017
0.0 10 0.017 0.033 100 0.25
0.05 1000 2.45 0.067 10000
24.283 0.067
10
Some Common Patterns (k,c,d are constants) T(0)
c T(n) T(n-k) d T(n) is O(n) T(0)
c T(n) T(n-k) dn T(n) is O(n2) T(0)
c T(n) T(n/k) d T(n) is O(log n) T(0)
c T(n) kT(n/k) d T(n) is O(n) T(0)
c T(n) kT(n/k) dn T(n) is O(n log n)
11

• (define (merge list1 list2)
• (if (empty? list1) list2
• (if (empty? list2) list1
• (let ((h1 (head list1))
• (h2 (head list2))
• (if (lt h1 h2)
• (cons h1 (merge (tail list1) list2))
• (cons h2 (merge list1 (tail list2))))))))
• T(0,m) c
• T(n,0) c
• T(n,m) k T(n-1,m) if h1 lt h2
• T(n,m) k T(n,m-1) if h2 lt h1
• The best-case constant time
• The worst-case???

12

• Worst case all but the last element in list1 is
  less than the first element of list2, and it is
  greater than all of the elements in list2.
• T(n,m) k T(n-1,m)
• k k T(n-2,m)
• k k k T(1,m)
• k k k T(1,m)
• k k k k T(1,m-1)
• k k k k k T(1,m-2)
• k k k k k k T(1,1)
• k k k k k k k c
• T_merge(n,m) k(nm)c O(nm)

m
n-1
13

• (define (split x1 x2 x3)
• (if (empty? x1)
• (merge (sort x1) (sort x2)
• (split (tail x1) x3 (cons (head x1) x2))))
• (define (sort x)
• (if (or (empty? x) (empty? (tail x)) x
• (split x empty empty)))
• T_sort(n) c (n0,1)
• T_sort(n) T_split(n,0,0) (ngt1)
• T_split(0,m,p) k T_mergesort(m)
• T_mergesort(p) T_merge(m,p)
• T_split(n,m,p) k T_split(n-1,p,m1)

14

• T_merge(n,m) O(nm)
• T_sort(n) O(T_split(n,0,0))
• T_split(0,m,p) O(T_sort(m)T_sort(p)T_merge(m,p
  ))
• T_split(n,m,p) kT_split(n-1,p,m1)
• T_split(n,0,0) k T_split(n-1,0,1)
• k k T_split(n-2,1,1)
• k k k T_split(n-3,1,2)
• k k k k T_split(n-4,2,2)
• k k k k k
• T_split(0,n/2,n/2)
• nk T_split(0,n/2,n/2)
• nk 2T_sort(n/2)T_merge(n/2,n
  /2)
• O(n) O(T_sort(n/2)) O(n)
• O(n) O(T_sort(n/2))

15

• T_sort(n) c (n0,1)
• T_sort(n) kn T_sort(n/2) (n gt 1)
• kn kn T_sort(n/4)
• kn kn kn T_sort(n/8)
• kn kn kn kn T_sort(1)
• kn kn kn kn c
• log_2(n)
• knlog_2(n)c
• O(nlog(n))
• Turns out this is as good as you can do for
  sorting.

16

• Hint for homework (encoding pairs)
• How can we encode t, f, and If?
• t (lambda (x y) x)
• f (lambda (x y) y)
• (if e1 e2 e3) (e1 e2 e3)
• If e1 evaluates to t
• ((lambda (x y) x) e2 e3) gt
• xe2/x,e3/y e2
• If e1 evaluates to f
• ((lambda (x y) y) e2 e3) gt
• ye2/x,e3/y e3

17

• Attempt 2
• t (lambda (x y) (x))
• f (lambda (x y) (y))
• (if e1 e2 e3)
• (e1 (lambda () e2) (lambda () e3))
• If e1 evaluates to t
• ((lambda (x y) (x))
• (lambda () e2) (lambda () e3)) gt
• ((lambda () e2)) gt e2

18

• We can also encode let
• (let ((x1 e1) (xn en)) e)
• ((lambda (x1 xn) e) e1 en)
• We can also encode multiple arguments
• (lambda (x1 x2 xn) e) gt
• (lambda (x1)
• (lambda (x2)
• (lambda (xn) e)))

19

• Turns out we can also encode
• pairs, first, second
• lists empty, cons, head, tail
• numbers (0empty,n1(cons empty n)
• , , , -,
• letrec
• trees, records, other data structures, etc.
• Only need
• e x (lambda (x) e) (e1 e2)
• to get a complete language!!!