Title: Introduction to asymptotic complexity Search algorithms
1Introduction to asymptotic complexity Search
algorithms
- You are responsible for Weiss, chapter 5, as
follows - 5.1 What is algorithmic analysis?
- 5.2 Examples of running time
- 5.3 NO, not responsible for this No.
- 5.4 Definition of Big-oh and Big-theta.
- 5.5 Everything except harmonic numbers. This
includes the repeated doubling and repeated
having stuff. - 5.6. No, not responsible, No. Instead, you should
know the following algorithms as presented in the
handout on correctness of algorithms and
elsewhere and be able to determine their
worst-case order of execution time linear
search, finding the min, binary search,
partition, insertion sort, selection sort, merge
sort, quick sort. - 5.7 Checking an algorithm analysis.
- 5.8 Limitations of big-oh analysis
2Organization
- Searching in arrays
- Linear search
- Binary search
- Asymptotic complexity of algorithms
3 / index of first occ. of v in b (b.length is
v not in b) / public static boolean
linearSearch(Comparable a, Object v) int i
0 // invariant v is not in b0..i-1
while (i lt a.length) if
(ai.compareTo(v) 0) return true i
i1 return false
7 4 6 19 3 7 8 10 32 54 67 98
Linear search
4 /b is sorted. Return a value k such that
b0..k v lt bk1.. / public static boolean
binarySearch(Comparable b, Object v) int
k -1 int j b.length // invariant
b0..k v lt bj.. while (j ! k1)
int e (kj)/ 2 // -1 lt k lt e lt j lt
b.length if (bk.compareTo(v) lt 0) k
e else j e return k
Each iteration performs one comparison and cuts
bk1..j-1 is half
0 k e
j
binary search
lt v
gt v
5Comparison of linear and binary search
- binary search runs much faster than linear
search. - Stating this precisely can be quite subtle.
- One approach asymptotic complexity of programs
- big-O notation
- Two steps
- Compute running time of program
- Running time ? asymptotic running time
- Asymptotic running time gives you a formula that
- tells you something about the running time
- on LARGE arrays.
6Running time of algorithms
- In general, running time of a program such as
linear search depends on many factors - machine on which program is executed
- laptop vs. supercomputer
- size of input (array A)
- big array vs. small array
- values of input
- v is first element in array vs. v is not in array
- To talk precisely about running times of
programs, we must specify all three factors
above.
7Defining running time of programs
- Machine on which programs are executed.
- Random-access Memory (RAM) model of computing
- Measure of running time number of operations
executed - Other models used in CS Turing machine, Parallel
RAM model, - Simplified RAM model for now
- Each data comparison is one operation.
- All other operations are free.
- Evaluate searching/sorting algorithms by
estimating number of comparisons they make
It can be shown that for searching and sorting
algorithms, total number of operations executed
on RAM model is proportional to number of data
comparisons executed.
8Defining running time (contd.)
- Dependence on size of input
- Rather than compute a single number, we compute a
function from problem size to number of
comparisons. - (e.g.) f(n) 32n2 2n 23
- where is problem size
- Each program has its own measure of problem size.
- For searching/sorting, natural measure is size of
array being searched/sorted. -
9Define running time (contd.)
- Dependence of running time on input values
- Consider set In of possible
- inputs of size n.
- Find number of comparisons
- for each possible input in this set.
- Compute
- Average harder to compute
- Worst-case easier to compute
- We will use worst-case complexity.
Possible inputs of size 2 for linear/binary search
10Computing running times
7 4 6 19 3 7 8 10 32 54 67 98
Linear search
Assume array is of size n. Worst-case number of
comparisons v is not in array. Number of
comparisons n. Running time of linear search
TL(n) n
Binary search sorted array of size n
-2
0
6
8
9
11
13
22
34
45
56
78
Worst-case number of comparisons v is not in
array. TB(n) log2(n)
1
11Base-2 logarithms
k 0 1 2 3 4 5 6 15
2k 1 2 4 8 16 32 64 32768
log(2k) 0 1 2 3 4 5 6 15
binary rep of 2k 00000000000000001 0000000000000
0010 00000000000000100 00000000000001000 000000000
00010000 00000000000100000 00000000001000000 01000
000000000000
If n 2k, then k is called the logarithm (to
the base 2) of n If n is a power of 2, then
log(n) is number of 0s following the 1.
12Running time ? Asymptotic running time
Linear search TL(n) n Binary search
TB(n)
log2(n)
1
We are really interested in comparing running
times only for large problem sizes. For small
problem sizes, running time is small enough that
we may not care which algorithm we use.
For large values of n, we can drop the 1 term
and the floor operation, and keep only the
leading term, and say that TB(n) ? log2(n) as n
gets larger. Formally, TB(n) O(log2(n)) and
TL(n) O(n)
13Rules for computing asymptotic running time
- Compute running time as a function of input size.
- Drop lower order terms.
- From the term that remains, drop floors/ceilings
as well as any constant multipliers. - Result usually something like O(n), O(n2),
O(nlog(n)), O(2n)
14Summary of informal introduction
- Asymptotic running time of a program
- Running time compute worst-case number of
operations required to execute program on RAM
model as a function of input size. - for searching/sorting algorithms, we will compute
only the number of comparisons - Running time ? asymptotic running time keep only
the leading term(s) in this function.
15Finding the minimum
- Function min(b, h, k) returns the index of the
minimum value of bh..k. - It performs a linear search, comparing each
element of bh1..k to the minimum of the
previous segment. - To find the minimum of a segment of size n takes
n-1 comparisons O(n).
h i
k
b bj is min of these
16Selection sort O(n2) running time
- / Sort b /
- public static void selectionsort(Comparable b)
- // inv b0..k-1 sorted, b0..k-1 bk..
- for (int k 0 k ! b.length k k1)
- int t min(b,k,b.length)
- Swap bk and bt
- k k1
-
-
-
k 0 1 2 3 n-1
comp. n-1 n-2 n-3 n-4 0
0 k
n
(n-1)n/2 n2 - n / 2
b sorted, lt gt