MS 101: Algorithms - PowerPoint PPT Presentation

About This Presentation

Title:

MS 101: Algorithms

Description:

MS 101: Algorithms – PowerPoint PPT presentation

Number of Views:31

Avg rating:3.0/5.0

Slides: 42

Provided by: nee107

Category:

more less

Transcript and Presenter's Notes

Title: MS 101: Algorithms

1
MS 101 Algorithms

Instructor
Neelima Gupta
ngupta_at_cs.du.ac.in

2
Table Of Contents

Mathematical Induction Review
Growth Functions

3
Review Mathematical Induction

Suppose
S(k) is true for fixed constant k
Often k 0
S(n) ? S(n1) for all n gt k
Then S(n) is true for all n gt k

4
Proof By Mathematical Induction

ClaimS(n) is true for all n gt k
Basis
Show formula is true when n k
Inductive hypothesis
Assume formula is true for an arbitrary n
Step
Show that formula is then true for n1

5
Strong Induction

Strong induction also holds
Basis show S(0)
Hypothesis assume S(k) holds for arbitrary k lt
n
Step Show S(n1) follows
Another variation
Basis show S(0), S(1)
Hypothesis assume S(n) and S(n1) are true
Step show S(n2) follows

6
Lets do it

Prove 1 2 3 n n(n1) / 2
Prove a0 a1 an (an1 - 1)/(a - 1) for
all a ? 1

7
Growth Functions

Big O Notation
In general a function
f(n) is O(g(n)) if there exist positive constants
c and n0 such that f(n) ? c ? g(n) for all n ? n0
Formally
O(g(n)) f(n) ? positive constants c and n0
such that f(n) ? c ? g(n) ? n ? n0
Intuitively, it means f(n) grows no faster than
g(n).
Examples
n2, n2 n
n3, n3 n2 n

f(n) n2 g(n) n2 n Is f(n) O(g(n))?
Sol g(n) n2 n
n2/2 n2/2 - n
n2/2 for n 2
½ f(n)
f(n) 2g(n) for n 2
Hence, f(n) O(g(n))

f(n) n3 g(n) n3 - n2 n Is f(n)
O(g(n))?
Sol g(n) n3 - n2 n
n3/2 n3/2 - n2 - n
n3/2 n3/2 - n3/4 - n3/4 for n
4
½ f(n)
f(n) 2g(n) for n 2
Hence, f(n) O(g(n))

f(n) n3 g(n) n3 - n2 n Is g(n)
O(f(n))?
Sol Clearly, g(n) n3 - n2 n
3 n3 for all n 1
3 f(n)
g(n) 3f(n) V n 1
Hence, g(n) O(f(n)).

11
Omega Notation

In general a function
f(n) is ?(g(n)) if ? positive constants c and n0
such that 0 ? c?g(n) ? f(n) ? n ? n0
Intuitively, it means f(n) grows at least as fast
as g(n).
Examples
n2, n2 n
n3, n3 n2 n

Ques f(n) n2, g(n) n2 n
Is f(n) ?(g(n))?
Sol n2 ½ (n2 n2 )
½ (n2 n )
f(n) c g(n)
for c ½ m 1

f(n) n3 g(n) n3 4n2 - 5n Is g(n)
?(f(n))?
Sol g(n) n3 4n2 - 5n
n3 /2 n3 /2 - 4 n2 - 5n2 for all
n 1
n3 /2 n2 (n/2 - 9)
n3 /2 for n 18
1/2 f(n)
Hence, g(n) ? (f(n)).

14
Theta Notation

A function f(n) is ?(g(n)) if ? positive
constants c1, c2, and n0 such that c1 g(n) ?
f(n) ? c2 g(n) ? n ? n0

15
Relations Between Q, W, O
Theorem For any two functions g(n) and f(n),
f(n) ?(g(n)) iff
f(n) O(g(n)) and f(n) ?(g(n)).

Proof
Simple. Do it.

16
Assignment No 1

Q1
Already proved
a0 a1 an (an1 - 1)/(a - 1) for all a
? 1
What is the sum for a 2/3 as n ? infinity? Is
it O(1)? Is it big or small?
For a 2, is the sum O(2n)? Is it big or
small?
Q2
Show that a polynomial of degree k theta(nk).

17
Other Asymptotic Notations

A function f(n) is o(g(n)) if for every positive
constant c, there exists a constant n0 gt 0 such
that f(n) lt c g(n) ? n ? n0
A function f(n) is ?(g(n)) if for every positive
constant c, there exists a constant n0 gt 0 such
that c g(n) lt f(n) ? n ? n0
Intuitively,

?() is like gt
?() is like ?

?() is like

o() is like lt
O() is like ?

18
Arrange some functions

f(n) O(g(n)) gt f(n) o(g(n)) ?
Is the converse true?
Let us arrange the following functions in
ascending order (assume log n o(n) is known)
n, n2, n3, sqrt(n), nepsilon, log n, log2 n,
n log n, n/log n, 2n, 3n

19
Relation between n n2

Solution
let c gt 0 be any constant such that
n lt c n2
i.e. 1 lt c n
i.e. n gt 1 / c
Hence n lt c n2 for n gt 1 / c
i.e. n o( n2)
we can also write it as n lt n2.

20
Relation between n2 n3

Solution
let cgt 0 be any constant such that
n2 lt c n3
Þ 1 lt c n
Þ n gt 1 / c
Hence n2 lt c n3 " n gt 1 / c
i.e. n2 o( n3)
we can also write it as n2 lt n3.
Combining the previous result we
can write
n lt n2
lt n3

21
Relation between n n1/2

Solution let cgt 0 be any constant such that
n1/2 lt c n
i.e n lt c2 n2
i.e 1 lt c2 n
i.e n gt 1/ c2
i.e. n1/2 lt c n for n gt 1/c2
i.e. n2 o( n3)
we can also write it as n1/2 lt n.
Combining the previous result
n1/2 lt n lt n2 lt n3

22
Relation between n log n

For the time being we can assume the result
log ( n ) o(n)
Þ log ( n ) lt n
we will prove it later.

23
Relation between n1/2 log n

Assume log n o(n)
let c gt 0 be any constant
for c/2 gt 0 there exists m gt 0 such that
log n lt (c/2) n for n gt m
changing variables from n to n1/2 we get
log(n1/2 ) lt (c/2) n1/2 for n1/2 gt m
½ log( n ) lt (c/2) n1/2 for n gt m2

24
Contd..

let m2 k
log( n ) lt c n1/2 for n gt k
Since c gt 0 was chosen arbitrarily hence
log n o( n1/2 )
or
log n lt n1/2
Combining the results we get
log n lt n1/2 lt n lt n2 lt n3

25
Relation between n2 nlog n

Since log n o(n)
for c gt 0, ? n0 gt 0 such that ? n ? n0, we have
log n lt c n
Multiplying by n on both sides we get
n log( n ) lt c n2 ? n ? n0
nlog n o( n2 )
nlog n lt n2

26
Relation between n nlog n

Solution let cgt 0 be any constant such
that
n lt c n log (n)
Þ 1 lt c log( n )
Þ log( n) gt 1 / c
Þ n gt e1/c
i.e. n lt c n log n " n gt e1/c
Since c was chosen arbitrarily
\ n o( n log n ) or n lt n log
n
Combining the results we can get
log n lt n1/2 lt n lt n logn lt n2 lt
n3

27
Relation between n n/log n

We know that n o(nlogn)
for c gt 0, ? n0 gt 0 such that ? n ? n0, we have
n lt c n log n
dividing both sides by log n we get
n/ log( n) lt c n ? n ? n0
Þ n / logn o(n)
i.e. n / logn lt n

28
Assignment No 2

Show that logM n o(nepsilon) for all
constants Mgt0 and epsilon gt 0. Assume that log n
o(n). Also prove the following
Corollary log n o(n/log n)
Show that nepsilon o(n/logn ) for every 0 lt
epsilon lt 1 .

Hence we have,
log n lt n/log n lt n1/2 lt n lt n logn lt n2 lt n3

30
Assignment No 3

Show that
lim f(n)/g(n) 0 gt f(n) o(g(n)).
n ? 8
lim f(n)/g(n) c gt f(n) ?(g(n)).
n ? 8, where c is a positive constant.
Show that log n o(n).
Show that nk o(2n) for every positive
constant k.

Show by definition of small o that
an o(bn) whenever a lt b , a and b are
positive constants.
Hence we have,
log n lt n/log n lt n1/2 lt n lt n logn lt n2 lt n3
lt2n lt 3n

32
Why the constants c and m?

Suppose we have two algorithms to solve the
problem say sorting Insertion Sort and Merge
sort for eg.
Why should we have more than one algorithm to
solve the same problem?
Ans efficiency.
Whats the measure of efficiency?
Ans System resources for example time.
How do we measure time?

33
Contd..

IS(n) O(n2)
MS(n) O(nlog n)
MS(n) is faster than IS(n).
Suppose we run IS on a fast machine and MS on a
slow machine and measure the time (since they
were developed by two different people living in
different part of the globe), we may get less
time for IS and more for MSwrong analysis
Solution count the number of steps on a generic
computational model

34
Computational Model Analysis of Algorithms

Analysis is performed with respect to a
computational model
We will usually use a generic uniprocessor
random-access machine (RAM)
All memory equally expensive to access
No concurrent operations
All reasonable instructions take unit time
Except, of course, function calls
Constant word size
Unless we are explicitly manipulating bits

35
Running Time

Number of primitive steps that are executed
Except for time of executing a function call, in
this model most statements roughly require the
same amount of time
y m x b
c 5 / 9 (t - 32 )
z f(x) g(y)
We can be more exact if need be

36
But why c and m?

Because
We compare two algorithms on the basis of their
number of steps and
the actual time taken by an algorithm is c
times the number of steps.

37
Why m?

We need efficient algorithms and computational
tools to solve problems on big data. For example,
it is not very difficult to sort a pack of 52
cards manually. However, to sort all the books in
a library on their accession number might be
tedious if done manually.
So we want to compare algorithms for large input.

38
An Example Insertion Sort